Derivation and validation of the renal angina index to improve the prediction of acute kidney injury in critically ill children

Abstract

Reliable prediction of severe acute kidney injury (AKI) has the potential to optimize treatment. Here we operationalized the empiric concept of renal angina with a renal angina index (RAI) and determined the predictive performance of RAI. This was assessed on admission to the pediatric intensive care unit, for subsequent severe AKI (over 200% rise in serum creatinine) 72 h later (Day-3 AKI). In a multicenter four cohort appraisal (one derivation and three validation), incidence rates for a Day 0 RAI of 8 or more were 15–68% and Day-3 AKI was 13–21%. In all cohorts, Day-3 AKI rates were higher in patients with an RAI of 8 or more with the area under the curve of RAI for predicting Day-3 AKI of 0.74–0.81. An RAI under 8 had high negative predictive values (92–99%) for Day-3 AKI. RAI outperformed traditional markers of pediatric severity of illness (Pediatric Risk of Mortality-II) and AKI risk factors alone for prediction of Day-3 AKI. Additionally, the RAI outperformed all KDIGO stages for prediction of Day-3 AKI. Thus, we operationalized the renal angina concept by deriving and validating the RAI for prediction of subsequent severe AKI. The RAI provides a clinically feasible and applicable methodology to identify critically ill children at risk of severe AKI lasting beyond functional injury. The RAI may potentially reduce capricious AKI biomarker use by identifying patients in whom further testing would be most beneficial.

Approximately 10% of all children admitted to an intensive care unit (ICU) develop acute kidney injury (AKI), and this rate increases up to 82% with increasing patient severity of illness.^1,2 Increasing AKI severity, characterized by serum creatinine (SCr)- and urine output (UOP)-based stratifications of AKI, is associated with increased mortality in adults³ and children.⁴ Even small increases in SCr (0.3 mg/dl) reflect significant kidney damage and are associated with poor patient outcome.^5,6 The well-recognized limitations of SCr for real-time accurate AKI diagnosis have prevented timely therapeutic interventions.⁷ Thus, extensive research efforts have been expended to find earlier, more sensitive biomarkers for AKI.

Several AKI biomarkers have demonstrated promising results for the identification and prediction of AKI in children. However, most have been validated only in the cardiopulmonary bypass (CPB) setting, where demographic homogeneity, lack of comorbidities, and a known onset and duration of ischemic injury provide an ideal biomarker validation environment.^8,9 Demographic heterogeneity likely contributes to the poor discriminatory performance of these biomarkers in non-cardiac pediatric intensive care unit (PICU) patients (area under the curve (AUC) values range from 0.54 to 0.78).^10–13 We previously found that children with persistent AKI at PICU admission (AKI after 48 h) were at the highest risk for requiring renal replacement therapy (RRT).² Identifying patients at risk for severe and long-lasting AKI in the PICU, and as importantly, identifying patients unlikely to be at risk, is imperative as risk stratification could allow more judicious AKI biomarker assessment to drive therapeutic intervention, increasing their predictive performance and cost-effectiveness.^14,15 Along these lines, the recent 10th Acute Dialysis Quality Initiative Conference (ADQI-X) issued a directive to use combinations of biomarkers to identify and differentiate functional AKI (‘pre-renal’ or ‘reversible’) from kidney damage (persistent).¹⁶

The context-based disparity of biomarker efficacy for acute coronary syndrome provides important lessons for the AKI field; troponin demonstrates suboptimal efficacy with capricious, undirected use.^17,18 Although the ability to detect and subsequently expeditiously treat myocardial infarction was augmented with the discovery and incorporation of troponin into the clinical context of cardiac angina, repeated evidence highlights the erosion of troponin performance when measured in patients at low demographic and/or clinical risk of myocardial infarction from coronary disease.^17–23 In addition, independent of troponin, the absence of cardiac angina carries high negative predictive value (NPV) for the diagnosis of a heart attack.^24,25

To that end, we recently proposed the empiric clinical model of renal angina to identify which critically ill patients would be at the greatest risk of AKI.²⁶ Using patient demographic factors and early signs of injury, renal angina aims to delineate patients at risk for subsequent severe AKI (AKI beyond the period of functional injury) versus those at low risk (Figure 1a). In the current study, we operationalize renal angina fulfillment by deriving an index (renal angina index: RAI) and, in separate derivation and validation cohorts, test the hypotheses that: (1) renal angina fulfillment using a RAI threshold improves prediction of subsequent severe AKI over severity of illness or risk factors alone and (2) RAI prediction of AKI outperforms currently used clinical thresholds for early signs of kidney injury.

Figure 1. Renal angina.

(a) The renal angina construct. The juxtaposed graphs depict risk of acute kidney injury (AKI) versus decrease in estimated creatinine clearance from baseline (↓eCCl) and increase in % intensive care unit (ICU) fluid overload (% ↑ICU FO). There are three risk groups defined for the pediatric ICU population (tranches): very high risk (intubated + presence of at least one vasopressor or inotrope), high risk (history of solid organ or bone marrow transplant), and moderate risk (ICU admission). The construct is created such that less sign of injury (estimated creatinine clearance (eCCl) change or FO change) is required for the higher risk tranches to fulfill renal angina (solid red slope line). (Adapted with permission from Goldstein and Chawla.²⁶) (b) The renal angina index. On the basis of existing pediatric AKI literature, tiered AKI risk strata were assigned point values for ‘risk’ and ‘signs’ of injury. The worse parameter between change in eCCl from baseline and % FO was used to yield an injury score. The full description of the derivation appears in Supplementary A online. The resultant renal angina index score can range from 1 to 40. A cutoff of ≥8 is used to determine renal angina fulfillment.

RESULTS

Group characteristics

Demographics for each cohort (C1 (n = 144): derivation; C2–C4 (n = 118, 108, and 214, respectively): validation) are shown in Table 1. Other than the absence of transplant patients, there were no significant demographic differences between C1 and C3. C4 patients were more severely ill (Pediatric Risk of Mortality II (PRISM-II) score²⁷) and had higher use of inotropy and mechanical ventilation than the other cohorts. The overall incidence of the subsequent severe AKI outcome 72–96 h from PICU admission (Day-3 AKI) in the cohorts was 10–20% (C1: 19%, C2: 10.2%, C3: 10.2%, and C4: 13.6%). The optimal RAI cutoff for fulfillment of renal angina (ANG(+), defined by RAI ≥8) was derived by studying patients from cohort 1 (Supplementary A online).

Table 1.

Demographic and clinical data for all cohorts

	Cohort 1: CCHMC sepsis 1 derivation	Cohort 2: MCH pro validation 1	Cohort 3: MCH retro validation 2	Cohort 4: CCHMC sepsis 2 validation 3
N	144	118	108	214
Very high risk	34	19	27	184
High risk	32	0	0	9
Moderate risk	78	99	81	21
Age, years	3.8 (1.2, 12.5)	3.0 (0.2, 11.7)	1.5 (0.3, 10.6)	2.2 (0.8, 5.9)
Male, n (%)	83 (57.6)	74 (62.7)	64 (59.2)	134 (62.6)
PRISM-II	11 (7, 18)	6 (4, 10)	7 (4, 10)	14 (8, 21)*
Transplant, n (%)	39 (27.1)	0	0	9 (4.2)
Inotropy, n (%)	56 (38.9)	23 (19.5)	28 (25.9)	214 (100)*
MV, n (%)	69 (47.9)	87 (73.7)	83 (76.9)	184 (85.9)*
Day-3 AKI, n (%)	28 (19.4)	12 (10.2)	11 (10.2)	29 (13.6)
PICU LOS, days	5 (3, 13)	6 (4, 8)	9 (6, 13)	13 (8, 24)*
RRT, n (%)	13 (9.0)	3 (2.5)	3 (2.8)	N/A
Mortality, n (%)	13 (9.0)	7 (5.9)	4 (3.7)	23 (10.7)

Abbreviations: AKI, acute kidney disease; CCHMC, Cincinnati Children’s Hospital; LOS, length of stay; MCH, Montreal Children’s Hospital; MV, mechanical ventilation; N/A, not available; PICU, non-cardiac pediatric intensive care unit; PRISM-II, Pediatric Risk of Mortality score; pro, prospective; retro, retrospective; RRT, renal replacement therapy.

Descriptive characteristics for each cohort of patients are listed above. Transplant refers to solid organs and bone marrow. Day-3 AKI refers to KDIGO stage 2 or 3 at Day 3 of PICU admission. Age, PRISM-II, and LOS are expressed as median (interquartile range).

^*P-value <0.05 Cohort 4 versus Cohort 1.

Derivation cohort (C1)—Cincinnati sepsis #1

Day 0 (PICU admission day) ANG(+) occurred in 51/144 (35%) of patients. Compared with ANG(−) (RAI <8) patients, ANG(+) patients had higher Day-3 AKI rates, longer PICU length of stay (LOS), higher RRT provision, and higher hospital mortality rates (Table 2). Day 0 RAI predicted Day-3 AKI with an AUC of 0.77 (95% confidence interval (CI) = 0.68–0.86). RAI <8 had a high NPV of 92% (95% CI = 85–97%) (Table 3).

Table 2.

Demographics of each cohort by fulfillment of renal angina

	Cohort 1: CCHMC sepsis 1 derivation			Cohort 2: MCH pro validation 1			Cohort 3: MCH retro validation 2			Cohort 4 : CCHMC sepsis 2 validation 3
	ANG(−)	ANG(+)	P	ANG(−)	ANG(+)	P	ANG(−)	ANG(+)	P	ANG(−)	ANG(+)	P
N (%)	93 (65)	51 (35)		100 (84.7)	18 (15.3)		70 (64.8)	38 (35.2)		69 (32.2)	145 (67.8)
Age (years)	3.1 (1, 11)	5.4 (2, 14)	0.088	6.0 (0, 12)	5.9 (0.4, 10)	0.90	5.6 (0, 11.7)	4.5 (0, 10.7)	0.37	3.8 (1.6, 6.8)	1.7 (0.5, 5)	<0.001
Male, n (%)	53 (56.9)	30 (58.8)	0.997	62 (62.0)	12 (67.7)	0.71	44 (62.9)	20 (52.6)	0.30	40 (57.9)	94 (44)	0.413
PRISM-II	10 (5, 16)	15 (8, 22)	0.016	6.8 (1, 12)	7.9 (3, 13)	0.41	6.2 (1, 12)	8.9 (3, 15)	0.004	10 (5, 15)	16 (10, 24)	<0.001
Day-3 AKI, n (%)	7 (7.5)	21 (41.2)	<0.001	5 (5.0)	7 (38.9)	<0.001	0 (0)	11 (28.9)	<0.001	2 (2.9)	27 (19)	0.003
LOS, days	5 (2, 10)	9 (4, 15)	0.011	7 (2, 11)	6 (4, 9)	0.66	12.3 (4, 20)	10.1 (3, 17)	0.24	11 (8, 16)	15 (7, 27)	0.032
RRT, n (%)	4 (4.3)	9 (17.6)	0.02	2 (2.0)	1 (5.6)	0.39	0 (0)	3 (7.9)	0.04	N/A	N/A
Mortality, n (%)	4 (4.3)	9 (17.6)	0.02	6 (6.0)	1 (5.6)	1	3 (4.3)	1 (2.6)	0.56	0 (0)	23 (16)	<0.001

Abbreviations: AKI, acute kidney disease; ANG, renal angina; CCHMC, Cincinnati Children’s Hospital; LOS, length of stay; MCH, Montreal Children’s Hospital; N/A, not available; PRISM-II, Pediatric Risk of Mortality score; pro, prospective; retro, retrospective; RRT, renal replacement therapy.

Select descriptive characteristics and outcomes for each cohort of patients are listed above. Day-3 AKI refers to KDIGO stage 2 or 3 at Day 3 of PICU admission. Data are expressed as medians with interquartile ranges in parentheses. P-values compare ANG(−) versus ANG(+) for each individual cohort.

Table 3.

Performance of the renal angina index for prediction of subsequent severe AKI

	Cohort 1: CCHMC sepsis 1 derivation	Cohort 2: MCH pro validation 1	Cohort 3: MCH retro validation 2	Cohort 4: CCHMC sepsis 2 validation 3
ANG(+), n (%)	51 (35)	18 (15)	38 (35)	145 (68)
Day-3 AKI, n (%)	28 (19)	12 (10)	11 (10)	29 (13)
Sensitivity, % (95% CI)	75 (55–89)	58 (28–85)	91 (59–100)	93 (76–99)
Specificity, % (95% CI)	73 (64–81)	90 (82–95)	71 (61–80)	36 (33–37)
PPV, % (95% CI)	40 (27–55)	39 (17–64)	26 (13–43)	18 (15–19)
NPV, % (95% CI)	92 (85–97)	95 (89–98)	99 (92–100)	97 (90–99)
AUC, (95% CI)	0.77 (0.68–0.86)	0.74 (0.59–0.88)	0.81 (0.71–0.91)	0.80 (0.75–0.86)

Abbreviations: AKI, acute kidney disease; ANG, renal angina; AUC, area under the curve; CCHMC, Cincinnati Children’s Hospital; CI, confidence interval; MCH, Montreal Children’s Hospital; NPV, negative predictive value; PPV, positive predictive value; pro, prospective; retro, retrospective.

The performance of the renal angina index (RAI) for prediction of severe AKI is shown above. For each patient in each cohort, an RAI was derived (a score of ≥8 was considered fulfillment of renal angina). The predictive performance of fulfillment of ANG on day 0 for the presence of Day-3 AKI was evaluated, which comprised the following: sensitivity, specificity, PPV, and NPV. The absolute value of the RAI (range 1–40) was used to derive the AUC receiver operating characteristic. ANG(+)refers to patients who fulfilled angina. Sensitivity, specificity, NPV, PPV, and AUC are listed with 95% CI.

Validation cohorts (C2–C4)—Montreal retrospective, prospective, and Cincinnati sepsis #2

Day 0 ANG(+) occurred in 15.3% (C2), 35.2% (C3), and 67.8% (C4) of patients. ANG(+) patients had significantly higher Day-3 AKI rates than ANG(−) patients in all cohorts (Table 2). Day 0 RAI predicted Day-3 AKI with an AUC between 0.74 and 0.81 and RAI <8 had an NPV ≥95% for all three cohorts (Table 3). In addition, RRT provision rates were higher, PICU LOS was longer, and mortality was higher in ANG(+) than in ANG(−) patients (Table 2).

RAI prediction of AKI—by creatinine clearance and/or fluid overload criteria

Both the predictive variable (RAI) and the outcome variable (AKI) were broken down by composite factors of kidney injury. The discrimination of RAI for Day-3 AKI by change in creatinine clearance from baseline (ΔeCCl) resulted in AUC values consistently superior to the discrimination by percent fluid overload (FO). Although FO did not perform as well for prediction as ΔeCCl, the AUC for RAI for Day-3 AKI improved when RAI incorporated both day of admission ΔeCCl and FO (Table 4). The AUC values for RAI prediction of Day-3 AKI were not different for whichever outcome criterion was used for outcome (UOP or ΔeCCl).

Table 4.

Renal angina index performance broken down by individual criterion

	Day-3 AKI outcome
	UOP	Cr	Worse
RAI
ΔeCCl	0.81 (0.71–0.90)	0.73 (0.59–0.88)	0.78 (0.69–0.87)
FO	0.57 (0.44–0.71)	0.63 (0.49–0.76)	0.60 (0.49–0.71)
Worse	0.78 (0.68–0.88)	0.75 (0.62–0.89)	0.77 (0.68–0.86)
Illness score
PRISM-II	0.65 (0.52–0.79)	0.61 (0.45–0.79)	0.66 (0.54–0.79)

Abbreviations: ΔeCCl, estimated change in creatinine clearance from baseline; AKI, acute kidney disease; Cr, creatinine; FO, fluid overload; RAI, renal angina index; UOP, urine output.

The area under the curve (AUC) values were calculated for prediction of Day-3 AKI for the RAI broken down by individual components (ΔeCCl, FO, or the ‘worse’ of the two). The AUC values demonstrate discriminatory superiority of ΔeCCl-derived RAI over FO-derived RAI for Day-3 AKI by all metrics of outcome used (UOP, Cr, or the ‘‘worse’’ variable). Including the FO metric for RAI improved the AUC for AKI measured by creatinine at Day 3 from 0.73 (RAI derived solely from ΔeCCl) to 0.75 (RAI derived from ‘worse’). Renal angina outperforms severity of illness scores for prediction of Day-3 AKI. For patients in cohort 1, the discrimination of day of admission RAI for Day-3 AKI was compared against Pediatric Risk of Mortality-II (PRISM-II) scores. Although the performance of FO-derived RAI is not as robust as ΔeCCl-derived RAI, it is comparable to PRISM-II and in some cases (Day-3 AKI outcome measured by creatinine clearance change (Cr)) improved. AUC values are expressed with 95% confidence intervals.

RAI and baseline creatinine

Approximately 25% (35/144) of patients in C1 required an imputed baseline creatinine owing to a lack of a baseline creatinine from which to compute ΔeCCl on the day of admission for the RAI calculation. Only 9 of these 35 were ANG(+) on the day of admission and only 1 had Day-3 AKI (this patient was ANG(+)). For the remaining 109 patients who had a known baseline creatinine for calculation of ΔeCCl, the RAI discrimination for Day-3 AKI was greater than for PRISM-II values and similar to the cohort as a whole (Supplementary B online).

RAI versus KDIGO stage

RAI prediction of Day-3 AKI was superior to simply having Kidney Diseases Improving Global Outcomes (KDIGO) stage 1 injury on the day of admission; fulfillment of renal angina demonstrated higher positive predictive value (PPV), NPV and a higher Youden’s index²⁸ for severe subsequent AKI than KDIGO stage 1. ANG(+) demonstrated similar predictive efficacy for Day-3 AKI compared with KDIGO stages 2–3, but had higher sensitivity, higher NPV, and higher Youden’s index (Table 5).

Table 5.

Fulfillment of renal angina outperforms KDIGO stages of AKI for prediction of subsequent severe AKI

	N	Sensitivity	Specificity	PPV	NPV	Youden’s index
KDIGO 1	25	21 (8–41)	84 (76–90)	24 (9–45)	82 (73–88)	5
KDIGO 2–3	24	46 (28–66)	91 (84–95)	54 (33–74)	87 (80–93)	37
ANG(+)	52	75 (55–89)	73 (64–81)	40 (27–55)	92 (85–97)	48

Abbreviations: AKI, acute kidney disease; ANG, renal angina; KDIGO, Kidney Diseases Improving Global Outcomes; NPV, negative predictive value; PPV, positive predictive value; RAI, renal angina index.

Renal angina outperforms signs of early injury alone for prediction of Day-3 AKI. On the day of admission, patients in cohort 1 were assessed for ANG(+) by RAI ≥ 8 and compared with KDIGO stage 1 or KDIGO stages 2 and 3. The results demonstrate that ANG(+) is superior to KDIGO stage 1 for prediction of severe subsequent AKI and as effective as KDIGO stages 2 and 3. Both sensitivity and negative predictive value for ANG(+) are higher than for KDIGO stages 2 and 3. Results are shown as percentages with 95% confidence intervals.

RAI versus pediatric illness severity

Significant differences in PRISM-II scores between ANG(+) and ANG(−) patients were analyzed by risk tranche and found to be attributable to weighting of patients in each group (Supplementary C online). On the basis of the univariate associations observed for patients with or without Day-3 AKI in both C1 and C4 (cohorts with similar admission diagnoses), we constructed a multivariable predictive model for Day-3 AKI using fulfillment of ANG, patient age, and PRISM-II score. Independent predictors of Day-3 AKI were as follows: ANG(+) (odds ratio (OR) = 3.91, 95% CI = 1.89–8.2), age (OR = 1.01 for 1-year increase in age, 95% CI = 1.02–1.11), and PRISM-II score (OR = 1.04 for 1-unit increase in score, 95% CI = 1.0–1.1) (Supplementary C online). When compared directly, RAI outperformed PRISM-II for the prediction of AKI outcome measured either by UOP, change in creatinine, or the worse metric of the two (Table 4).

DISCUSSION

Renal angina fulfillment identifies children at the highest risk of suffering subsequent severe AKI. For a clinician, the ability to predict the presence of severe AKI 3 days in advance carries obvious benefit. The percentage of children suffering from severe AKI at 72 h (Day-3 AKI) in each cohort is indicative of the extent of the AKI burden in the PICU. The concomitant comorbidities of ANG(+)-associated AKI (increased duration of mechanical ventilation, inotropy, RRT use, and mortality) are clear.

AKI biomarkers need to demonstrate the appropriate balance of diagnostic performance and cost-effectiveness to gain widespread acceptance leading to implementation at the bedside. Indiscriminate testing for any condition (myocardial infarction, stroke, kidney injury, and so on) in every patient (regardless of size, age, and comorbidities) will render any biomarker virtually useless. Avoidance of such shotgun testing is only possible by providing direction (clinical context) for biomarker use. We have used the performance of troponin levels to detect acute coronary syndrome to provide an apt example of directed, optimized biomarker testing. Troponin measured in patients who exhibit cardiac angina, a combination of clinical signs and known coronary disease risk factors, allows practitioners to rule in myocardial infarction. In this select, risk-stratified population, troponin has great specificity and PPV. When measured in patients without cardiac angina, troponin loses performance. Unfortunately, unlike a heart attack, AKI does not carry an easily identifiable physical prodrome such as cardiac angina. Simply put, a kidney attack²⁹ does not ‘hurt’. Therefore, to optimize performance, clinicians must seek novel ways of directing AKI biomarker use.

We suggest that renal angina fulfills this prediction need, as it outperformed signs of injury alone for prediction of severe subsequent AKI. The derivation of the RAI (Figure 1b, Supplementary A online) was based on available AKI epidemiology reported in select pediatric populations: children admitted to the ICU carry increased risk over the general population (4.5–10%),^1,30 children receiving bone marrow transplantation have ~3× risk (11–21%),³¹ and those who are intubated and on vasopressor support carry nearly 5× risk versus the general ICU population (51%).² The ‘signs of injury’ (i.e., kidney pain) in the RAI include ΔeCCl and FO. The significance of FO on PICU morbidity has been recently documented, causing deleterious effects on oxygenation³² and increasing morbidity and mortality in children with lung injury.³³ When the RAI was derived based on either of the individual variables (ΔeCCl or FO) in isolation, the discrimination for Day-3 AKI remained robust, particularly for ΔeCCl (Table 4). Although FO for RAI calculation did not perform as well as ΔeCCl, FO alone was equal to PRISM-II scores for prediction of Day-3 AKI (Supplementary C online). More importantly, the prediction for Day-3 AKI when the worse value of FO or ΔeCCl was used improved over using only ΔeCCl (0.73–0.75). The RAI AUC values for Day-3 AKI as classified by ΔeCCl, UOP, or the worse of the two metrics were nearly identical (Table 4). Renal angina outperformed early signs of injury as stratified by the KDIGO stages. ANG(+) demonstrated a markedly more robust predictive performance compared with KDIGO stage 1 (higher Youden’s index, PPV and NPV) and was at least equal to, if not slightly superior to, KDIGO stages 2 and 3 (Table 5). It should be noted that the improvement over KDIGO stage 1 is the value of the RAI—the identification of the at-risk patient who requires further investigation and for whom an AKI biomarker may be most beneficial. The overweight smoker with sub-sternal chest pain, jaw claudication, and ST-segment changes on electrocardiogram (i.e., risk + KDIGO 2–3) gets a troponin as a confirmatory test while being prepped and draped in the interventional catheterization suite. A patient with cardiac risk factors, and mild chest pressure, benefits from troponin as a discriminatory test, as the result affects clinical decision-making.

Renal angina outperforms severity of illness scores for prediction of severe subsequent AKI. As mentioned earlier, FO used alone for calculation of the RAI was on par with PRISM-II discrimination of Day-3 AKI. When either eCCl or the worse metric (either eCCl or FO) was used for RAI calculation, the index scores’ AUC values far surpassed PRISM-II scores’ values (Table 4 and Supplementary C online). Analysis of RAI per risk tranche revealed that the PRISM-II score skewed toward those patients in the very high-risk tranche, likely leading to a significant difference in the overall PRISM-II scores between the ANG(+) and ANG(−) patients. The PRISM-II score consistency within the individual tranches taken together with the comparison with renal angina simultaneously highlights the limitations of traditional severity of illness scoring systems for AKI³⁴ and underscores the utility of renal angina for AKI prediction. The implied benefit of RAI having greater advocacy than PRISM-II at identifying patients at risk for AKI is the application of RAI in clinical trials of AKI. Identification of patients at a higher AKI risk using RAI stratification could theoretically guide the enrollment for a novel AKI biomarker or therapy trial, which could ultimately guide treatment strategy.

Our goal with the derivation of the RAI was to develop a simple score, which is easily calculable and can be used at the bedside of a critically ill patient. Although published prediction scores for organ failure, severity of illness, or mortality admittedly use more rigorous statistical methodologies for derivation, these models are constrained by a constant need of recalibration, potential over-fitting, and are intended to be used in population analyses, not for single patients. Thus, although our derivation of the RAI was semi-empiric and, by comparison, simple, we propose that this simplicity (and, based on this study, apparent validity) will enhance its use in clinical care and in future research. Another important point is that children admitted to the PICU most often do not have clearly identifiable risk factors for AKI (as a preoperative cardiac surgery adult patient may have); the RAI uses easily identifiable criteria within the first day of PICU admission to predict the highly clinically relevant outcome of subsequent severe AKI. Finally, previous studies of adult AKI prediction scores have had extremely large and comprehensive administrative databases available for deriving and validating scores; such databases are nonexistent in the PICU population. We propose that future research should aim at constructing and validating the use of large, multicenter PICU databases, to enable more refined and rigorous evaluation of AKI risk factors and evidence of injury, which will allow for validation and/or calibration of the RAI score if needed.

The performance of AKI biomarkers for diagnosis and prediction varies depending on the risk of AKI in the population studied. Candidate biomarkers including neutrophil gelatinase-associated lipocalin (NGAL), kidney injury molecule-1 (KIM-1), interleukin-18 (IL-18), and liver-type fatty acid–binding protein (L-FABP), initially discovered via proteomic data in murine models of kidney ischemia,³⁵ demonstrate good-to-excellent predictive performance (AUC = 0.75–0.95) for AKI in children after CPB.^36–38 A recent study from our center demonstrated that a clinical model of patient age and CPB duration yielded AUC values of 0.72 and 0.74 to predict a 50% SCr rise at 2 and 6 h after bypass.⁹ In our current study, the renal angina clinical model on Day 0 of PICU admission in all four cohorts studied yielded a similar predictive precision (moderate-to-good predictive range, with AUC values ranging from 0.73 to 0.81) for persistent severe AKI on Day 3. We also suggest that for ANG(−) children our model provides a powerful predictive tool to guide care. The NPVs >92% in all four cohorts and >95% in three of the cohorts indicate that Day 0 ANG(−) patients have a very low likelihood of having a 200% rise in SCr or prolonged oliguria on PICU Day 3. These data suggest that AKI biomarkers should not be obtained to predict severe AKI in Day 0 ANG(−) children, given the low likelihood of Day-3 AKI. For instance, although not tested in the current study, ANG(−) children with elevated Day 0 SCr may likely have a fluid responsive state (previously termed pre-renal azotemia), and therefore could likely receive significant volume without developing the severe FO consistently associated with poor outcomes in critically ill children.^32,33,39 In the CPB study conducted at our center, the prediction of AKI severity improved after urinary biomarkers were added alone and in combination; we also predict that inclusion of available plasma AKI biomarkers (i.e., directed biomarker testing) will improve upon the renal angina clinical model. Ideally, through a simple calculation of the RAI, a clinician can identify ANG fulfillment or absence in any patient on admission and then appropriately allocate the use of an AKI biomarker test to those in whom the test may yield the greatest predictive benefit. We suggest that renal angina allows biomarker testing to be appropriately directed. Directed use is a logical method that can be used to derive biomarker panels that classify and phenotype AKI types—the exact directive that ADQI-X set forth in the recent consensus statement.¹⁶ Such optimization eliminates shotgun use of AKI biomarkers, both cost prohibitive and nonscientific.¹⁵

We acknowledge that given our limited data in pediatrics, particularly in the general PICU, which cares for wide-ranging and heterogeneously ill children, an admission change in SCr from baseline (to any degree) carries an unknown prognosis. In our derivation cohort, roughly 25% of the patients did not have a baseline creatinine in our database, and we imputed baseline values for these patients based on height and normal creatinine clearance for age. The only effect this manipulation carried on our results was to potentially underestimate the change in eCCl from baseline (affecting both ANG diagnosis and AKI outcome diagnosis; Table 4); the imputed baseline creatinine levels were higher than baseline creatinine levels observed in comparable patients (based on size and age). Finally, although data from the adult AKI literature suggests that the outcome of AKI as measured by both creatinine and UOP is worse than creatinine-only AKI, this is unknown in pediatrics. There are, in fact, few studies that examine UOP-pRIFLE (pediatric-modified RIFLE),² and this limitation is outlined in the recent worldwide KDIGO AKI guidelines.^40–42 In our study, we found no significant differences in RAI prediction of AKI when the outcome was based on creatinine or UOP.

Our study has several strengths. We examined four independent cohorts with 100–200 critically ill children each, which in aggregate is a pediatric large AKI study. Our study includes a wide-ranging array of patient age, background history, and illness severity. The multicenter nature of our study overcomes single-center bias for the initiation of mechanical ventilation, inotropic support, and RRT provision. In addition, we have provided evidence that RAI outperforms severity of illness scores, risk factors, and degrees of injury alone for prediction of subsequent severe AKI. Our study may be useful in addressing the limitations and problems that are encountered in pediatric AKI study.⁴³ As we suggested earlier, the use of renal angina to stratify patients for enrollment in biomarker or therapy trials may create the uniformity required to properly analyze AKI in pediatrics. Such stratification and implementation of the ADQI-X directive of AKI classification may identify patients who have significant creatinine change as a result of functional injury, which was the case in Cohort 1 (Table 5 shows a low PPV for KDIGO 2–3 on admission).

There are several potential limitations to our study. The retrospective analyses of several cohorts is subject to the inherent limitations of data extraction, correlative analyses, and conclusions associated with data not collected with the exclusive purpose of testing our hypothesis. The number and the diverse range of illness in our patient population overcome some of these limitations by giving us a larger sample of patients who are at varying degrees of risk. Although the specificity of early changes in either creatinine clearance or % FO for persistent AKI is limited, this composite aspect of RAI incorporates clinical context to provide usable bedside information (e.g., when a doubling of SCr may reflect reversible AKI vs. persistent AKI). Although there seemed to be a paucity of high-risk patients in our cohorts (history of transplant), a substantial percentage of transplant patients were stratified as very high risk given the need for inotropy and mechanical ventilation. The apparent higher incidence of AKI in the high risk group versus the very high risk group is therefore slightly misleading. Patients with transplants have an increased risk of AKI, and we expect that with future study of renal angina, supported by larger pediatric population database study, more patients will fall directly in the high-risk tranche, allowing for refinement of the RAI score. We did not test the RAI on ICU admission days other than Day 0 or to predict AKI persistence after ICU Day 3 (primarily owing to the most logical predictive benefit for a clinician and sample size limitation), and acknowledge that the RAI performance may be better or worse than the clinical context we tested.

We conclude that risk stratification using renal angina will aid with the prediction of persistent severe AKI and that renal angina prodrome, in conjunction with AKI biomarker measurement, may reliably differentiate patients who will be responsive to appropriate restorative therapy from those will progress to severe subsequent AKI. We believe that renal angina is a clinical adjunct that will lead to the optimization of AKI biomarker performance across the wide-ranging heterogeneity that exists across the general pediatric PICU population.

MATERIALS AND METHODS

All patient data

We used data collected from four separate PICU cohorts from Cincinnati Children’s Hospital Medical Center (CCHMC) and Montreal Children’s Hospital (MCH). Each cohort is described below. For all cohorts, patients with preexisting chronic kidney disease or immediately after cardiac surgery were excluded (three patients were excluded from C1 because of baseline requirement of RRT). Each site’s Institutional Review Board waived the need for informed consent for the purposes of this study, but informed consent was obtained for C3 and C4 as part of their original study (see below). All patients in the study (all cohorts) were in the age range of 28 days to 25 years.

Individual cohort descriptions

‘Derivation’ cohort (C1): this was a retrospective cohort of patients admitted to the CCHMC PICU from 2009 to 2011 with an International Classification of Diagnosis (ICD-9) code of ‘sepsis’ or ‘septic shock’, as per international consensus guidelines for the diagnosis of sepsis.⁴⁴ The RAI was first derived and validated in this cohort, as described in Supplementary A online.

‘Validation’ cohorts (C2, C3, and C4): two separate observational cohorts of patients admitted to the MCH PICU were included as validation cohorts. The retrospective cohort (C2) consisted of all eligible children admitted to the MCH PICU from 2004 to 2007 for at least 4 days. The original study for C2 was performed to evaluate FO in critically ill children and has not previously been reported. UOP data were unavailable. The prospective cohort (C3) consisted of children admitted to the MCH PICU for at least 2 days from 2007 to 2010. These patients were recruited into a prospective AKI biomarker study and have not previously been reported. Both databases were reviewed in detail to ensure that relevant data for this report were appropriately collected, followed by extraction of necessary variables and analysis. The C4 cohort included patients admitted to 17 different PICUs across the United States from the years 2006–2011 and enrolled in a study investigating the genomic profile of children with sepsis.⁴⁵ Inclusion criteria of this study required an ICD-9 code of ‘septic shock’. Patients from C4, called ‘CCHMC sepsis 2’ for convenience, were not included in the C1 cohort above.

Data obtained

Obtained data were denoted as follows: Day 0 was the first calendar day of PICU admission. Day 3 consisted of the time period between 72 and 96 h after PICU admission. Baseline data included demographic information, admission diagnoses, PRISM-II scores,²⁷ and first SCr of the PICU admission. Day 0 data included for determination of RAI included the use of vasopressors/inotropy and the use of invasive mechanical ventilation (yes or no). Calculated variables for the determination of RAI included % FO⁴⁶ and changes in kidney function (based on estimated creatinine clearance (eCCl)). Percent FO on Day 0 was determined by assessing the first 8 h of admission in the ICU on Day 0. The time frame of 8 h was felt to be beyond the generally accepted window of ‘early goal-directed therapy’ (EGDT) of resuscitation,⁴⁷ allowing time for some diuresis, and also was sufficiently shorter than an actual full day. The electronic health record was reviewed for the lowest SCr up to 3 months before PICU admission to establish a reference eCCl. If no SCr was available, a reference eCCl of 120 ml/min per 1.73 m² was used.⁴⁸ A baseline creatinine was also imputed based on the eCCl and the patient height using the Schwartz correction. UOP was recorded hourly as ml/kg/h in 8-h blocks (except in C2, no urine output available).

Renal angina

All patients were classified on Day 0 as fulfilling criteria for renal angina (i.e., being ANG(+) vs. ANG(−)) using the RAI. An RAI score of ≥8 demonstrated the highest Youden’s index²⁸ and the highest NPV (Supplementary A online), and thus ANG(+) was defined as an RAI score ≥8.

KDIGO stage comparison

RAI scores for each patient in C1 were compared with the KDIGO AKI stages (1 or 2 and3)⁴⁰ for prediction of outcome.

Outcomes

The primary outcome was the presence of severe AKI 72 h after PICU admission (Day-3 AKI), denoted as ‘subsequent severe AKI’. Day 3 was chosen because of the following reasons: most PICU patients develop AKI within this time frame; there is enough time to develop the severe or persistent AKI outcome; and theoretically, 3 days surpasses the time frame of what would be considered reversible (or functional) AKI. Severe AKI was defined by the KDIGO AKI classification stage ≥2: SCr of 200% baseline (a decrease in eCCl of ≥50% from baseline) or ≤0.5 ml/kg/h of UOP for ≥8 h.⁴⁰ We chose KDIGO stage ≥2 as the primary outcome as it is associated with mortality and morbidity in multiple pediatric studies.⁴ The higher of the KDIGO strata (UOP or ΔeCCl) was used. For C2 (retrospective MCH), only SCr data were available. Secondary outcomes were PICU LOS, use of RRT, and in-hospital mortality.

Statistical analysis

All statistical analyses were performed using STATA version 12 (StataCorp, College Station, TX) and SAS version 9.3 (SAS Institute, Cary, NC). Continuous variables were reported as median with interquartile range and compared using the Mann–Whitney test. Categorical variables were summarized using frequency and proportion and compared by chi-square or Fisher’s exact tests. An RAI cutoff of ≥8 was used to analyze the predictive performance of RAI (sensitivity, specificity, NPV, and PPV). As Youden’s index should not be used as the sole index of performance, the cutoff value of ≥8 was chosen on the basis of most superior performance based on Youden’s and highest NPV (to optimize the ability of RAI to serve as a ‘screening test’). Multivariate regression was performed by comparing variables carrying univariate associations with the outcome and a P-value <0.20 (Supplementary C online). AUC values were calculated for each prediction model. In all analyses, a P-value <0.05 was considered statistically significant.