Abstract
Background and objectives
Novel AKI biomarkers carry variable performance for prediction of AKI in patients with heterogeneous illness. Until utility is demonstrated in critically ill patients outside of the cardiopulmonary bypass population, AKI biomarkers are unlikely to gain widespread implementation. Operationalization of an AKI risk stratification methodology, termed renal angina, was recently reported to enhance prediction at the time of intensive care unit admission for persistent severe AKI. The renal angina index (RAI) was developed to provide the clinical context to direct AKI biomarker testing. This study tested the hypothesis that incorporation of AKI biomarkers in patients fulfilling renal angina improves the prediction of persistent severe AKI.
Design, setting, participants, & measurements
In a multicenter study of 214 patients admitted to the pediatric intensive care unit with sepsis, the discrimination of plasma neutrophil gelatinase–associated lipocalin (NGAL), matrix metalloproteinase-8 (MMP-8), and neutrophil elastase-2 (Ela-2) were determined individually and in combination with the RAI for severe AKI. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated.
Results
Individual biomarkers demonstrated marginal discrimination for severe AKI (area under curve [AUC]: NGAL, 0.72; MMP-8, 0.68; Ela-2, 0.72), inferior to prediction by the clinical model of the RAI (AUC=0.80). Incorporation of each biomarker significantly added to the renal angina model AKI prediction (AUC=0.80, increased to 0.84–0.88; P<0.05 for each). The inclusion of each biomarker with the RAI demonstrated NRI (0.512, 0.428, and 0.545 for NGAL, MMP-8, and Ela-2, respectively; all P<0.03) and IDI (0.075 for Ela-2). The inclusion of both Ela-2 and NGAL with RAI demonstrated an NRI of 0.871 (P<0.001) and an IDI of 0.1 (P=0.01).
Conclusions
This study shows that incorporation of AKI biomarkers into the RAI improves discrimination for severe AKI. The RAI optimizes the utility of AKI biomarkers in a heterogeneous, critically ill patient population.
Introduction
AKI affects a significant proportion of children in the pediatric intensive care unit (PICU) and is independently associated with worsened morbidity and increased mortality. Unfortunately, timely recognition of AKI is hampered by classic functional markers such as serum creatinine and urine output, both highly confounded and delayed to change after injury (1). Extensive research efforts have been expended to discover and validate biomarkers that detect AKI before functional marker change and predict AKI severity.
Novel AKI biomarkers have not demonstrated consistent predictive performance outside of the neonatal cardiopulmonary bypass (CPB) surgery population. AKI biomarkers demonstrate exceptional performance in these patients, largely because of demographic homogeneity, lack of comorbidities, and a known onset and duration of ischemic injury (2,3). However, in noncardiac PICU patients, performance of these same biomarkers is variable, with area under the curve receiver-operating characteristic (AUC-ROC) values ranging from 0.54 to 0.85 (Table 1) (4–7). Our previous data indicate that children suffering AKI>48 hours after admission are at highest risk for poor clinical outcome, including prolonged length of stay, need for RRT, or mortality (5,8). Thus, identification of patients at risk of AKI early in their intensive care unit (ICU) admission may help clinicians optimize the timing of resuscitative and supportive therapies. Identification of patients at intermediate risk of disease is equally important. For the purpose of risk stratification, we derived and validated the empirical clinical model of renal angina for prediction of subsequent severe AKI (9). The renal angina index (RAI) is a composite of patient AKI risk and early signs of injury, created to risk stratify patients for whom biomarkers testing would be most optimal (Figure 1). In short, fulfillment of renal angina informs the pretest probability of any AKI diagnostic test. Just as in the scenario of a potential heart attack, in which troponin levels are tested in patients with cardiac angina (thereby optimizing its use using clinical context), we propose that in scenarios of potential kidney attacks, AKI biomarkers will have greater utility and predictive value in patients who have renal angina fulfillment. Repeated evidence highlights the erosion of troponin performance when measured in patients without chest pain or at low clinical risk of myocardial infarction from coronary disease (10–16). In addition, independent of troponin, the absence of cardiac angina carries high negative predictive value for the diagnosis of a heart attack (17,18). Capricious, undirected use of any biomarker degrades its effectiveness.
Table 1.
Biomarker performance for AKI prediction in noncardiopulmonary bypass pediatric ICU patients
| Author | Year | N | Patients | Sepsis (%) | AKI Outcome | Biomarker | Sensitivity | Specificity | PPV | NPV | AUC-ROC |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Herrero (4) | 2007 | 25 | Foley | 28 | Detection | Serum cystatin C (0.6 mg/L) | 85 | 63 | 2.3 (+LR) | NR | 0.85 |
| Serum β2-microglobulin (1.5 mg/L) | 85 | 54 | 1.8 (+LR) | NR | 0.82 | ||||||
| Zappitelli (5) | 2007 | 140 | MV Foley | 22.9 | Persistent AKI at 48 h | Urinary NGAL (0.2 ng/mg creatinine) | 78 | 67 | NR | NR | 0.63 |
| Urinary NGAL (0.4 ng/mg creatinine) | 67 | 78 | NR | NR | |||||||
| Washburn (6) | 2008 | 137 | MV Foley | 21 | Prediction at 24 h | Urinary IL-18 (100 pg/ml) | 25 | 81 | 22 | 83 | 0.54 |
| Urinary IL-18 (200 pg/ml) | 13 | 89 | 20 | 82 | |||||||
| Wheeler (7) | 2008 | 143 | Sepsis | 100 | Prediction of AKI within 7 d | Serum NGAL (139 ng/ml) | 86 | 39 | 39 | 94 | 0.68 |
| Basu (24) | 2011 | 70 | Sepsis | 100 | Persistent AKI at 7 d | MMP-8 | 100 | 41 | 24 | 100 | 0.66 |
| Ela-2 | 100 | 49 | 27 | 100 | 0.69 |
Above are the most prominent pediatric studies examining biomarker performance for AKI in critically ill pediatric patients who were not subjected to cardiopulmonary bypass surgery. AUC-ROC, area under the curve receiver-operating characteristic; Ela-2, neutrophil elastase-2; Foley, indwelling Foley catheter; ICU, intensive care unit; +LR, positive likelihood ratio; MMP-8, matrix metalloproteinase-8; MV, mechanical ventilation; NGAL, neutrophil gelatinase-associated lipocalin; NPV, negative predictive value; NR, not reported; PPV, positive predictive value.
Figure 1.
Renal angina. (A) The renal angina construct. The juxtaposed graphs depict risk of AKI versus decrease in estimated creatinine clearance from baseline (↓eCrCl) and versus increase in the percentage of intensive care unit fluid overload (% ↑ICU FO). There are three risk groups defined for the pediatric intensive care unit population (tranches): very high risk (intubated plus the presence of at least one vasopressor or inotrope), high risk (history of solid organ or bone marrow transplant), and moderate (ICU admission). The construct is created such that less sign of injury (eCrCl change or FO change) is required for the higher risk tranches to fulfill renal angina (solid red slope line). Adapted with permission from Goldstein and Chawla (9). (B) The renal angina index (RAI). On the basis of existing pediatric AKI literature, tiered AKI risk strata were assigned point values for risk and signs of injury. The worse parameter between change in eCrCl from baseline and the percentage of fluid overload (% FO) was used to yield an injury score. The full description of the derivation appears in Supplemental Material A. The resultant RAI score can range from 1 to 40. A cut-off of ≥8 is used to determine renal angina fulfillment.
In this study, we test the hypothesis that plasma AKI biomarker incorporation into the RAI significantly improves prediction of subsequent severe AKI in patients with septic shock compared with either the clinical model or AKI biomarkers.
Materials and Methods
Patients
Included patients were admitted to 17 different PICUs across the United States between 2006 and 2011 and were enrolled in a study investigating the genomic profile of children admitted with a diagnosis of severe sepsis and/or septic shock (19). The patients were enrolled under informed consent for the genomics work, allowing measurement of the biomarkers. In the original study, blood samples were obtained within 24 hours of initial presentation to the PICU with septic shock. Patients who presented with preexisting CKD or immediately after cardiac surgery were excluded. The institutional review boards approved secondary use/analysis of biologic samples and clinical data. All patients in the study were aged 28 days to 12 years. The patients analyzed in this study formerly comprised one of the validation cohorts in the initial renal angina study (CCHMC Sepsis 2, Validation 3) (20).
Data Collection
We defined the following data collection points. Day 0 was the first calendar day of PICU admission. Day 3 consisted of the time period between 72 and 96 hours after PICU admission. Baseline data included demographic information, admission diagnoses, Pediatric Risk of Mortality (PRISM-II) scores (21), and the first serum creatinine measurement (SCr) of the PICU admission. Day 0 data used for determination of renal angina included use of vasopressor and inotropic support and mechanical ventilatory support. We recorded changes in kidney function, based on estimated creatinine clearance (eCrCl) calculated by the Schwartz equation (22), for determination of the RAI. We reviewed the electronic health record for the lowest SCr up to 3 months before PICU admission to establish a reference eCrCl. If no SCr was available, we used a reference eCrCl of 120 ml/min per 1.73 m2 (23).
Renal Angina
All patients were classified on day 0 as fulfilling criteria for renal angina [i.e., ANG(+) versus ANG(−)] using the RAI. Renal angina describes risk of AKI as a balance between risk and injury such that less early injury is required if a patient is at high risk for AKI compared with patients who are at low risk (Figure 1A). The RAI used to determine fulfillment or absence of renal angina is calculated as a composite of risk of AKI (based on published pediatric epidemiologic data) and initial injury based on changes from baseline in creatinine clearance and percent fluid overload (Figure 1B). A RAI score of ≥8 defined ANG(+) (20). Sensitivity analyses and breakdown of RAI derivation are presented in Supplemental Material A. Only eCrCl was used as an injury threshold for RAI calculation because fluid balance data were not available for these patients.
AKI Biomarkers
Plasma neutrophil gelatinase–associated lipocalin (NGAL), matrix metalloproteinase-8 (MMP-8), and neutrophil elastase-2 (Ela-2) were evaluated using xMAP multiplex array system technology (Luminex, Austin, TX). We chose these three biomarkers because NGAL is a well studied AKI biomarker, and MMP-8 and Ela-2 were identified as potential AKI biomarkers in the related genomic profile study (7,24). The biomarker measurements were performed before use in this study and investigators measuring these biomarker levels had no knowledge of the patient outcomes in the original study. The a priori determined cut-off values for each biomarker were determined by sensitivity analyses in the initial data sets. The cut-offs for MMP-8 and Ela-2 were ≥11 and ≥235 ng/ml, respectively (24). The cut-off used for serum NGAL was ≥150 ng/ml (7). Per the conditions of the original study, biomarkers were measured only in the first 24 hours of ICU admission. Blood was not drawn for testing after the first 24 hours.
Outcomes
The primary outcome was presence of severe AKI on day 3 of PICU admission (day 3 AKI). The presence of AKI on days 1 and 2 was not considered a positive AKI outcome. The presence of AKI on day 3, denoted as subsequent severe AKI, was considered a positive outcome. Day 3 was chosen because most PICU patients develop AKI within this timeframe, the time highlights the advantage of biomarkers on the PICU admission day to predict the outcome, and day 3 is a clinically relevant time frame for AKI management (20). Day 3 is also beyond the period of time that AKI may be functional and is more likely to be damage-associated AKI (25). Severe AKI was defined by the Kidney Disease Improving Global Outcomes AKI classification stage≥2: serum creatinine of 200% baseline (a decrease in eCrCl of ≥50% from baseline) (26). Urine output was not available in these patients. Secondary outcomes were PICU length of stay and in-hospital mortality. Data on RRT use were unavailable.
Statistical Analyses
All statistical analyses were performed using STATA version 12 (StataCorp, College Station, TX), SAS version 9.3 (SAS Institute, Cary, NC), and R version 2.14.1 (R Development Core Team, Vienna, Austria). Continuous variables were reported as the median with interquartile range and were compared using the Mann–Whitney test. Categorical variables were summarized using frequency and proportion and compared by chi-squared or Fisher’s exact tests. An RAI cut-off of ≥8 was used to define renal angina fulfillment [ANG(+)] and this cut-off was used for operative characteristics (20). Simple and multivariable logistic models were used to predict day 3 AKI using RAI and its combinations with AKI biomarker(s) concentration levels. Area under the curve (AUC) values were calculated for each prediction model (RAI and biomarker concentrations used as continuous variables) and compared using DeLong’s method (27). Besides the increase in AUC, the performance improvement of the prediction models with AKI biomarkers added were also evaluated by the Akaike information criterion (AIC), net reclassification improvement (NRI), and integrated discrimination improvement (IDI) (28,29). No risk categories were selected for the calculation of NRI. R package Hmisc was used for the calculation of NRI and IDI. In all analyses, a P value <0.05 was considered statistically significant.
Results
Group Characteristics
Demographics are shown in Table 2. The incidence of severe AKI at 3 days was 13.6% (29 of 214) in all patients, although the incidence was significantly higher in patients with RAI≥8 [ANG(+)] on admission (27 of 145 [18.6%] versus 2 of 69 [2.9%]; P=0.003). Age and PRISM-II scores were evaluated using multivariable logistic regression. Although younger age was independently associated with day 3 AKI, PRISM-II on admission was not. Fulfillment of renal angina was an independent predictor of day 3 AKI (Table 3).
Table 2.
Demographics of cohort versus fulfillment of renal angina
| Demographic | ANG(−) | ANG(+) | P Value |
|---|---|---|---|
| Patients | 69 (32.2) | 145 (67.8) | – |
| Age, yr | 3.8 (1.6, 6.8) | 1.7 (0.5, 5) | <0.001 |
| Boys | 40 (57.9) | 94 (44) | 0.41 |
| PRISM-II score | 10 (5, 15) | 16 (10, 24) | <0.001 |
| Day 3 AKI | 2 (2.9) | 27 (19) | 0.003 |
| Length of stay, d | 11 (8,16) | 15 (7,27) | 0.03 |
| Mortality | 0 (0) | 23 (16) | 0.001 |
Data are expressed as n (%) or median (interquartile range). Select descriptive characteristics and outcomes for each cohort of patients are listed above. P values compare ANG(−) versus ANG(+) for each individual cohort. ANG(−), no renal angina fulfillment; ANG(+), renal angina fulfillment; Day 3 AKI, Kidney Disease Improving Global Outcomes stage 2 or 3 at day 3 of pediatric intensive care unit admission; PRISM-II, Pediatric Risk of Mortality score.
Table 3.
Predictors of day 3 AKI using multivariable logistic regression
| Variable | Odds Ratio (95% Confidence Interval) | P Value |
|---|---|---|
| Renal angina | 1.09 (1.06 to 1.13) | <0.001 |
| Age | 1.19 (1.03 to 1.37) | 0.02 |
| PRISM-II | 1.02 (0.98 to 1.07) | 0.30 |
Renal angina score (by RAI ≥8) is an independent predictor of day 3 AKI, whereas PRISM-II score is not (odds ratio for PRISM-II crosses boundary of 1.0). RAI, renal angina index.
Novel AKI Biomarkers Are Not Superior to the RAI for Prediction of Severe AKI
The comparison of AUC point estimates taken on day 0 showed that all of the plasma biomarkers demonstrated equivalent performance and none of them were superior to the RAI for prediction of day 3 AKI (Table 4).
Table 4.
Performance of the RAI and serum biomarkers for prediction of subsequent severe AKI
| Test | RAI | NGAL | MMP-8 | Ela-2 |
|---|---|---|---|---|
| Sensitivity | 93 (76 to 99) | 90 (72 to 97) | 86 (69 to 95) | 90 (73 to 97) |
| Specificity | 36 (33 to 37) | 34 (28 to 42) | 33 (30 to 34) | 44 (42 to 46) |
| PPV | 18 (15 to 19) | 18 (12 to 25) | 16 (13 to 18) | 20 (16 to 21) |
| NPV | 97 (90 to 99) | 97 (87 to 99) | 94 (87 to 98) | 97 (91 to 99) |
| AUC-ROC | 0.80 (0.75 to 0.86) | 0.72 (0.63 to 0.82) | 0.68 (0.59 to 0.78) | 0.72 (0.62 to 0.82) |
Data are presented as the percentage (95% confidence interval). A RAI score of ≥8 is considered fulfillment of renal angina. The fulfillment of renal angina on day 0 was compared with the presence of AKI on day 3 of admission for the sensitivity, specificity, PPV, and NPV. The absolute value of the RAI (range, 1–40) was used to derive the AUC-ROC.
Incorporation of AKI Biomarkers Augments the Predictive Performance of the RAI
The probability of AKI was calculated by determining the rate of AKI per subpopulation of biomarker and RAI combination, demonstrating a marked increase after incorporation of each biomarker (Table 5). Supplemental Material B depicts a patient tree, separating patients based on a priori biomarker decision rules. The predictive performance of RAI for day 3 AKI improved with the incorporation of each plasma biomarker separately (Figure 2, Table 6). Improvement in RAI predictive performance was greatest when both NGAL and Ela-2 were added (AUC=0.88; P value for increase from RAI AUC=0.02). The quality of the models (AIC) also improved with the incorporation of the biomarkers, decreasing by almost 13 points with the inclusion of NGAL and Ela-2 to the RAI (from 135.39 to 123.45).
Table 5.
Probability of AKI increases with incorporation of biomarkers into the RAI
| RAI | Ela-2 | MMP-8 | NGAL | AKI Probability (%) |
|---|---|---|---|---|
| − | − | a | a | 0 |
| − | + | a | a | 5 |
| + | − | a | a | 5.3 |
| + | + | a | a | 27.3 |
| − | a | − | a | 0 |
| − | a | + | a | 4.3 |
| + | a | − | a | 4.9 |
| + | a | + | a | 24.1 |
| − | a | a | − | 0 |
| − | a | a | + | 4.4 |
| + | a | a | − | 6.8 |
| + | a | a | + | 23.8 |
| − | − | − | a | 0 |
| − | − | + | a | 0 |
| − | + | − | a | 0 |
| − | + | + | a | 7.4 |
| + | − | − | a | 0 |
| + | − | + | a | 8.8 |
| + | + | − | a | 11.1 |
| + | + | + | a | 31.4 |
The probability of day 3 AKI, determined as a percentage of patients with AKI in a given biomarker delineated subpopulation is shown for each biomarker profile. Individual rates and patient flow maps are depicted in Supplemental Material C.
Biomarker not included for the particular analysis.
Figure 2.
ROC curves. The four plots represent the ROC curves from the following four prediction models for day 3 AKI: RAI (A), NGAL and RAI (B), MMP-8 and RAI (C), and Ela-2 and RAI (D). The curves in B–D are overlaid with the ROC curve with RAI only (curve in A). Ela-2, neutrophil elastase-2; MMP-8, matrix metalloproteinase-8; NGAL, neutrophil gelatinase–associated lipocalin; ROC, receiver-operating characteristic.
Table 6.
Inclusion of biomarkers increases the predictive performance of the RAI
| Model | AUC | Increase in AUC (95% Confidence Interval) | P Value | AIC |
|---|---|---|---|---|
| RAI | 0.80 | 135.39 | ||
| RAI+NGAL | 0.85 | 0.05 (0.01 to 0.09) | 0.01 | 132.77 |
| RAI+MMP-8 | 0.84 | 0.04 (0.001 to 0.07) | 0.04 | 133.23 |
| RAI+Ela-2 | 0.87 | 0.06 (0.01 to 0.12) | 0.03 | 125.14 |
| RAI+NGAL+MMP-8 | 0.86 | 0.06 (0.01 to 0.11) | 0.03 | 128.97 |
| RAI+NGAL+Ela-2 | 0.88 | 0.07 (0.01 to 0.14) | 0.02 | 123.45 |
| RAI+MMP-8+Ela-2 | 0.87 | 0.06 (0.01 to 0.12) | 0.03 | 126.93 |
The prediction model of the RAI for day 3 AKI was used as the reference model for evaluating the models incorporating biomarkers NGAL, MMP-8, and Ela-2 individually and in combination. The AUC (c-statistic) and the Akaike information criterion (AIC) were calculated for each model.
AKI Biomarkers Augment the Classification Accuracy of AKI Prediction
Inclusion of all biomarkers, individually and in combinations of two, improved the classification accuracy of AKI prediction (Table 7). Although the NRI conferred by biomarker incorporation was most augmented by reclassification of nonevents (i.e., patients without day 3 AKI), improvement was also shown with event reclassification (i.e., patients with day 3 AKI). Weighted reclassification, measured by IDI demonstrated statistical significance with the incorporation of Ela-2 alone or with a combination of two biomarkers and the RAI.
Table 7.
Inclusion of biomarkers significantly improves discrimination of the RAI for AKI
| Model | NRI | IDI | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NRI Event | P Value | NRI Nonevent | P Value | Total NRI | P Value | IS | P Value | IP | P Value | Total IDI | P Value | |
| RAI+NGAL | −0.035 | 0.85 | 0.546 | <0.001 | 0.512 | 0.01 | 0.023 | 0.13 | −0.004 | 0.32 | 0.027 | 0.08 |
| RAI+MMP-8 | −0.172 | 0.35 | 0.600 | <0.001 | 0.428 | 0.03 | 0.025 | 0.30 | −0.004 | 0.15 | 0.029 | 0.23 |
| RAI+Ela-2 | 0.172 | 0.35 | 0.373 | <0.001 | 0.545 | 0.01 | 0.065 | 0.07 | −0.010 | 0.05 | 0.075 | 0.03 |
| RAI+NGAL+MMP-8 | 0.103 | 0.58 | 0.503 | <0.001 | 0.606 | 0.002 | 0.058 | 0.06 | −0.009 | 0.05 | 0.067 | 0.02 |
| RAI+NGAL+Ela-2 | 0.379 | 0.04 | 0.492 | <0.001 | 0.871 | <0.001 | 0.086 | 0.02 | 0.014 | 0.03 | 0.100 | 0.01 |
| RAI+MMP-8+Ela-2 | 0.172 | 0.35 | 0.459 | <0.001 | 0.632 | 0.002 | 0.067 | 0.06 | −0.011 | 0.04 | 0.078 | 0.02 |
The prediction model of the RAI for day 3 AKI was used as the reference model for evaluating the models incorporating biomarkers NGAL, MMP-8, and Ela-2 individually and in combination. NRI and IDI values shown represent the total effect for both events (with day 3 AKI) and nonevents (without day 3 AKI). IP, difference of integral of 1-specificity between models; IS, difference of integral of sensitivity between models.
Discussion
In this study, we confirm the hypothesis that incorporation of biomarkers into a risk-stratified clinical model for AKI improves predictive performance for subsequent severe injury. The data presented are directly complementary to, and a logical follow-up of, our original analysis of renal angina. In the initial study, we demonstrated that the renal angina construct was a valid screening model to identify patients at highest risk for AKI. In this study, we demonstrate that a positive biomarker result in the population with an RAI≥8 significantly increases the predictive discrimination for subsequent severe AKI. Although the clinical management of AKI in critically ill children who are either premoribund or obviously recovering is unlikely to change based on the assessment of renal angina, we propose that this systematic approach to evaluating AKI risk carries the greatest clinical relevance for patients with an uncertain trajectory of illness. RAI assessment is early (day of admission), easy to practice (calculating RAI and measuring biomarkers is relatively simple compared with calculation of severity of illness scores), useful (high negative predictive value), and could potentially change initial resuscitation targets (fluid goals), use of medications (nephrotoxins), and timing of adjunctive therapies (RRT).
Judicious assessment of AKI biomarkers will increase their predictive performance and clinical effectiveness (30,31). AKI biomarkers need to demonstrate the appropriate balance of diagnostic performance and cost-effectiveness in order to gain widespread acceptance leading to implementation at the bedside. Indiscriminate biomarker measurement in every patient, regardless of size, age, and comorbidities, for a given injury or syndrome will render any test virtually useless. We have used the performance of troponin levels to detect acute coronary syndrome to provide an apt example of directed, optimized biomarker testing. Troponin measured in patients who exhibit cardiac angina, a combination of clinical signs, and known coronary disease risk factors allows practitioners to “rule in” heart attack. Troponin loses specificity when measured in patients without cardiac angina risk stratification (12). Unfortunately, unlike a heart attack, AKI does not carry an easily identifiable physical prodrome like cardiac angina. Simply put, a kidney attack does not “hurt” (32). Therefore, in order to optimize performance, care providers must seek novel ways of directing AKI biomarker use. The results of this proof-of-concept analysis demonstrate that incorporation of a positive AKI biomarker test (e.g., a renal troponin) in the novel RAI AKI risk stratification system (kidney pain) heightens the sensitivity and discrimination for a kidney attack.
Improvement of AKI prediction by incorporation of biomarkers into the RAI occurs via correct classification of disease. A recent study from our center demonstrated that each of these urinary biomarkers (NGAL, kidney injury molecule-1, IL-18, and liver-type fatty acid binding protein) added alone and in combination with the clinical model of patient age and CPB duration, improved prediction of AKI development and severity (33). The results presented here parallel the findings of this CPB study, demonstrating improvement in discrimination, evident by higher AUC (i.e., c-statistic) values, for AKI after incorporation of biomarkers. AUC is affected by sample and effect sizes. The metric of AIC evaluates the statistical weight of loss of information due to sample size. The lower AIC upon incorporation of AKI biomarkers into renal angina indicates that less information was lost when the goodness of fit of the AUC was balanced with the complexity of the model (Table 6). The NRI and IDI metrics recalibrate the AUC for weighted reclassification of both sample size and effect size, leading to a more accurate assessment of predictive and discriminative ability of a model (34). The significant changes in NRI and IDI after incorporation of biomarkers into the renal angina model occurred via weighting of the negative effect size such as the nonevents (i.e., no day 3 AKI) (Table 7). Finally, just as in the troponin paradigm, biomarker positivity in patients without renal angina does not improve prediction of day 3 AKI. The rate of AKI in patients with RAI<8 was 3%. A positive biomarker in those patients only increased the probability of AKI from 4.3% to 5% (Supplemental Material B).
Risk stratification using the RAI provides a clinically relevant and potentially useful tool to guide care. By quickly calculating the RAI and detecting renal angina, a clinician can identify the absence or fulfillment of renal angina in any patient on admission and then appropriately allocate the use of an AKI biomarker test to those in whom the test may yield the greatest predictive benefit. The excellent negative predictive value (>90%) of the RAI allows a clinician to reliably rule out AKI outside the window of functional AKI (25) and may afford more freedom of management for the acutely ill patient (i.e., higher volume resuscitation) with fewer concerns about developing the severe fluid overload consistently associated with poor outcomes in critically ill children (35–37).
Our study has several strengths. We examined a large multicenter cohort of critically ill children, overcoming potential single-center bias for initiation of mechanical ventilation and/or inotropic support. Although all participants were included in an original study of sepsis, our study includes a wide-ranging array of patient age and illness severity. Compared with previous work (Table 1), our study demonstrates a significant improvement in AKI prediction using novel biomarkers outside of the CPB population. Although RAI was not tested on ICU admission days other than day 0 or to predict AKI persistence before or after ICU day 3, the timeframe chosen is clinically relevant with regard to discriminating between reversible (functional) and damage-associated AKI (25).
There are several potential limitations to our study. The retrospective analysis of patients is subject to the inherent limitations of data extraction, correlative analyses, and conclusions associated with data not collected with the exclusive purpose of testing our hypothesis. We did not have access to fluid balance data or urine output data, both incorporated in the initial renal angina study and used as a metric of AKI outcome. In the initial validation of the RAI, the additive effect of fluid overload to the model was minimal (20). In addition, because the population studied in this analysis was a validation cohort used in the original study, we did not feel that it was prudent to recalibrate the RAI due to the lack of fluid data. We chose to study patients at day 3 because of the clinical significance of AKI lasting longer than the traditional reversible period. We did not extend the AKI analysis past 3 days because the number of patients remaining in the ICU until day 7 was too small to afford meaningful statistical results. The paucity of high-risk renal angina strata patients in our cohort (history of transplant) supports the call for larger pediatric population database studies. We did not have data regarding the use of RRT because the data studied were taken from a study directed at deriving biomarkers for sepsis and sepsis-associated morbidity (not for management variables). Of the biomarkers presented here, only NGAL is traditionally considered an AKI biomarker and was studied secondary to previous experience investigating NGAL as a septic shock–associated AKI biomarker in the pediatric population (7). Ela-2 and MMP-8 carry mild biologic plausibility for involvement in sepsis-associated AKI, but have not otherwise been described or analyzed as AKI biomarkers aside from our initial derivation study. We did not have urine AKI biomarkers available for this study, which could yield different performance after incorporation in the clinical model.
We conclude that risk stratification using the RAI in conjunction with AKI biomarker measurement augments prediction of subsequent severe AKI. We have now derived and validated the RAI in multiple populations from multiple medical centers and demonstrated initial proof-of-concept data—biomarker incorporation into a risk-stratified model bolsters prediction of AKI. The renal angina concept will require further, prospective analysis and may require recalibration in the lower-risk populations. Context provided by the RAI to direct testing of biomarkers may help differentiate damage-associated AKI from functional AKI (25). We believe that the RAI is a clinical adjunct that will lead to the optimization of AKI biomarker performance across the wide-ranging heterogeneity that exists across the general pediatric PICU population.
Disclosures
L.S.C. is a consultant for Alere, Abbott, and Astute.
Supplementary Material
Acknowledgments
We thank the following investigators that contributed biological samples and patient data for the database: Natalie Z. Cvijanovich (Children’s Hospital Oakland), Mark Hall (Nationwide Children’s Hospital), Geoffrey L. Allen (Children’s Mercy Hospital), Neal J. Thomas (Hershey Children’s Hospital), Robert J. Freishtat (Children’s National Medical Center), Nick Anas (Children’s Hospital of Orange County), Keith Meyer (Miami Children’s Hospital), Paul A. Checchia (Texas Children’s Hospital), Richard Lin (Children’s Hospital of Philadelphia), Michael T. Bigham (Akron Children’s Hospital), Anita Sen (Morgan Stanley Children’s Hospital), Jeffrey Nowak (Children’s Hospital and Clinics of Minnesota), Michael Quasney (Children’s Hospital of Wisconsin), Jared W. Henricksen (St. Christopher’s Hospital for Children), Arun Chopra (C.S. Mott Children’s Hospital), Sharon Banschbach (Cincinnati Children's Hospital Medical Center), Eileen Beckman (Cincinnati Children's Hospital Medical Center), Kelli Harmon (Cincinnati Children's Hospital Medical Center), Patrick Lahni (Cincinnati Children's Hospital Medical Center), and Thomas P. Shanley (C.S. Mott Children’s Hopsital).
Patients were enrolled in a study supported by grants from the National Institutes of Health (R01-GM064619, RC1-HL100474, and R01-GM099773).
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
This article contains supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.09720913/-/DCSupplemental.
See related editorial, “Renal Angina: Right Concept…Wrong Name?,” on pages 633–634.
References
- 1.McCullough PA, Shaw AD, Haase M, Bouchard J, Waikar SS, Siew ED, Murray PT, Mehta RL, Ronco C: Diagnosis of acute kidney injury using functional and injury biomarkers: Workgroup statements from the tenth Acute Dialysis Quality Initiative Consensus Conference. Contrib Nephrol 182: 13–29, 2013 [DOI] [PubMed] [Google Scholar]
- 2.Kwiatkowski DM, Goldstein SL, Krawczeski CD: Biomarkers of acute kidney injury in pediatric cardiac patients. Biomarkers Med 6: 273–282, 2012 [DOI] [PubMed] [Google Scholar]
- 3.Parikh CR, Devarajan P, Zappitelli M, Sint K, Thiessen-Philbrook H, Li S, Kim RW, Koyner JL, Coca SG, Edelstein CL, Shlipak MG, Garg AX, Krawczeski CD, TRIBE-AKI Consortium : Postoperative biomarkers predict acute kidney injury and poor outcomes after pediatric cardiac surgery. J Am Soc Nephrol 22: 1737–1747, 2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Herrero-Morín JD, Málaga S, Fernández N, Rey C, Diéguez MA, Solís G, Concha A, Medina A: Cystatin C and beta2-microglobulin: Markers of glomerular filtration in critically ill children. Crit Care 11: R59, 2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Washburn KK, Zappitelli M, Arikan AA, Loftis L, Yalavarthy R, Parikh CR, Edelstein CL, Goldstein SL: Urinary interleukin-18 is an acute kidney injury biomarker in critically ill children. Nephrol Dial Transplant 23: 566–572, 2008 [DOI] [PubMed] [Google Scholar]
- 6.Zappitelli M, Washburn KK, Arikan AA, Loftis L, Ma Q, Devarajan P, Parikh CR, Goldstein SL: Urine neutrophil gelatinase-associated lipocalin is an early marker of acute kidney injury in critically ill children: A prospective cohort study. Crit Care 11: R84, 2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Wheeler DS, Devarajan P, Ma Q, Harmon K, Monaco M, Cvijanovich N, Wong HR: Serum neutrophil gelatinase-associated lipocalin (NGAL) as a marker of acute kidney injury in critically ill children with septic shock. Crit Care Med 36: 1297–1303, 2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Akcan-Arikan A, Zappitelli M, Loftis LL, Washburn KK, Jefferson LS, Goldstein SL: Modified RIFLE criteria in critically ill children with acute kidney injury. Kidney Int 71: 1028–1035, 2007 [DOI] [PubMed] [Google Scholar]
- 9.Goldstein SL, Chawla LS: Renal angina. Clin J Am Soc Nephrol 5: 943–949, 2010 [DOI] [PubMed] [Google Scholar]
- 10.Meyer M, Fink C, Roeger S, Apfaltrer P, Haghi D, Kaminski WE, Neumaier M, Schoenberg SO, Henzler T: Benefit of combining quantitative cardiac CT parameters with troponin I for predicting right ventricular dysfunction and adverse clinical events in patients with acute pulmonary embolism. Eur J Radiol 81: 3294–3299, 2012 [DOI] [PubMed] [Google Scholar]
- 11.Lim W, Whitlock R, Khera V, Devereaux PJ, Tkaczyk A, Heels-Ansdell D, Jacka M, Cook D: Etiology of troponin elevation in critically ill patients. J Crit Care 25: 322–328, 2010 [DOI] [PubMed] [Google Scholar]
- 12.Stein R, Gupta B, Agarwal S, Golub J, Bhutani D, Rosman A, Eng C: Prognostic implications of normal (<0.10 ng/ml) and borderline (0.10 to 1.49 ng/ml) troponin elevation levels in critically ill patients without acute coronary syndrome. Am J Cardiol 102: 509–512, 2008 [DOI] [PubMed] [Google Scholar]
- 13.Agewall S, Giannitsis E, Jernberg T, Katus H: Troponin elevation in coronary vs. non-coronary disease. Eur Heart J 32: 404–411, 2011 [DOI] [PubMed] [Google Scholar]
- 14.Agewall S, Tjora S: Physical exertion may cause high troponin levels. Tidsskr Nor Laegeforen 131: 2226, 2011 [DOI] [PubMed] [Google Scholar]
- 15.Tjora S, Gjestland H, Mordal S, Agewall S: Troponin rise in healthy subjects during exercise test. Int J Cardiol 151: 375–376, 2011 [DOI] [PubMed] [Google Scholar]
- 16.Ammann P, Fehr T, Minder EI, Günter C, Bertel O: Elevation of troponin I in sepsis and septic shock. Intensive Care Med 27: 965–969, 2001 [DOI] [PubMed] [Google Scholar]
- 17.Bassan R, Pimenta L, Scofano M, Gamarski R, Volschan A, Chest Pain Project investigators : Probability stratification and systematic diagnostic approach for chest pain patients in the emergency department. Crit Pathw Cardiol 3: 1–7, 2004 [DOI] [PubMed] [Google Scholar]
- 18.Bassan R, Pimenta L, Scofano M, Soares JF, Chest Pain Project Investigators : Accuracy of a neural diagnostic tree for the identification of acute coronary syndrome in patients with chest pain and no ST-segment elevation. Crit Pathw Cardiol 3: 72–78, 2004 [DOI] [PubMed] [Google Scholar]
- 19.Wong HR, Salisbury S, Xiao Q, Cvijanovich NZ, Hall M, Allen GL, Thomas NJ, Freishtat RJ, Anas N, Meyer K, Checchia PA, Lin R, Shanley TP, Bigham MT, Sen A, Nowak J, Quasney M, Henricksen JW, Chopra A, Banschbach S, Beckman E, Harmon K, Lahni P, Lindsell CJ: The pediatric sepsis biomarker risk model. Crit Care 16: R174, 2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Basu RK, Zappitelli M, Brunner L, Wang Y, Wong HR, Chawla LS, Wheeler DS, Goldstein SL: Derivation and validation of the renal angina index to improve the prediction of acute kidney injury in critically ill children [published online ahead of print September 18, 2013]. Kidney Int 10.1038/ki.2013.349 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Pollack MM, Ruttimann UE, Getson PR: Pediatric risk of mortality (PRISM) score. Crit Care Med 16: 1110–1116, 1988 [DOI] [PubMed] [Google Scholar]
- 22.Schwartz GJ, Work DF: Measurement and estimation of GFR in children and adolescents. Clin J Am Soc Nephrol 4: 1832–1843, 2009 [DOI] [PubMed] [Google Scholar]
- 23.Zappitelli M, Parikh CR, Akcan-Arikan A, Washburn KK, Moffett BS, Goldstein SL: Ascertainment and epidemiology of acute kidney injury varies with definition interpretation. Clin J Am Soc Nephrol 3: 948–954, 2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Basu RK, Standage SW, Cvijanovich NZ, Allen GL, Thomas NJ, Freishtat RJ, Anas N, Meyer K, Checchia PA, Lin R, Shanley TP, Bigham MT, Wheeler DS, Devarajan P, Goldstein SL, Wong HR: Identification of candidate serum biomarkers for severe septic shock-associated kidney injury via microarray. Crit Care 15: R273, 2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Endre ZH, Kellum JA, Di Somma S, Doi K, Goldstein SL, Koyner JL, Macedo E, Mehta RL, Murray PT: Differential diagnosis of AKI in clinical practice by functional and damage biomarkers: Workgroup statements from the tenth Acute Dialysis Quality Initiative Consensus Conference. Contrib Nephrol 182: 30–44, 2013 [DOI] [PubMed] [Google Scholar]
- 26.KDIGO Work Group : KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int Suppl 2: 1–138, 2012 [Google Scholar]
- 27.DeLong ER, DeLong DM, Clarke-Pearson DL: Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 44: 837–845, 1988 [PubMed] [Google Scholar]
- 28.Pencina MJ, D’Agostino RB, Sr, Steyerberg EW: Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 30: 11–21, 2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Pencina MJ, D'Agostino RB, Sr, D'Agostino RB, Jr, Vasan RS: Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat Med 27: 157–172; discussion 207–212, 2008 [DOI] [PubMed] [Google Scholar]
- 30.Al-Ismaili Z, Palijan A, Zappitelli M: Biomarkers of acute kidney injury in children: Discovery, evaluation, and clinical application. Pediatr Nephrol 26: 29–40, 2011 [DOI] [PubMed] [Google Scholar]
- 31.Glassford NJ, Eastwood GM, Young H, Peck L, Schneider AG, Bellomo R: Rationalizing the use of NGAL in the intensive care unit. Am J Respir Crit Care Med 184: 142–, author reply 143., 2011 [DOI] [PubMed] [Google Scholar]
- 32.Kellum JA, Bellomo R, Ronco C: Kidney attack. JAMA 307: 2265–2266, 2012 [DOI] [PubMed] [Google Scholar]
- 33.Krawczeski CD, Goldstein SL, Woo JG, Wang Y, Piyaphanee N, Ma Q, Bennett M, Devarajan P: Temporal relationship and predictive value of urinary acute kidney injury biomarkers after pediatric cardiopulmonary bypass. J Am Coll Cardiol 58: 2301–2309, 2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Pencina MJ, D’Agostino RB, Vasan RS: Statistical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med 48: 1703–1711, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Arikan AA, Zappitelli M, Goldstein SL, Naipaul A, Jefferson LS, Loftis LL: Fluid overload is associated with impaired oxygenation and morbidity in critically ill children. Pediatr Crit Care Med 13: 253–258, 2012 [DOI] [PubMed] [Google Scholar]
- 36.Valentine SL, Sapru A, Higgerson RA, Spinella PC, Flori HR, Graham DA, Brett M, Convery M, Christie LM, Karamessinis L, Randolph AG, Pediatric Acute Lung Injury and Sepsis Investigator’s (PALISI) Network. Acute Respiratory Distress Syndrome Clinical Research Network (ARDSNet) : Fluid balance in critically ill children with acute lung injury. Crit Care Med 40: 2883–2889, 2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Goldstein SL, Somers MJ, Baum MA, Symons JM, Brophy PD, Blowey D, Bunchman TE, Baker C, Mottes T, McAfee N, Barnett J, Morrison G, Rogers K, Fortenberry JD: Pediatric patients with multi-organ dysfunction syndrome receiving continuous renal replacement therapy. Kidney Int 67: 653–658, 2005 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.