Recalibration of the Renal Angina Index for Pediatric Septic Shock

Abstract

Introduction

Sepsis-associated acute kidney injury (AKI) is a common diagnosis in children that is associated with poor outcomes. The lack of therapeutic options once present makes early identification of at-risk patients essential. The renal angina index (RAI) has been previously validated to predict severe AKI in heterogeneous populations of critically ill children. The performance of this score specifically in children with septic shock is unknown.

Methods

A secondary analysis of a multicenter, prospective, observational study of 379 children with septic shock to determine the ability of the RAI to predict severe AKI at day 3, and to assess for the potential need for recalibration of the RAI in this unique subset of patients.

Results

At the original cutoff of ≥8, the RAI predicted day 3 severe AKI with an area under the receiving operating characteristic (AUROC) curve 0.90 (95% confidence interval [CI]: 0.86 to 93), 95% sensitivity, and 54% specificity. A Youden’s index identified a higher optimal cutoff of ≥20 (sensitivity 83%, specificity 80%), and day 1 platelet count <150 × 10³/μl was an independent predictor of severe AKI (adjusted odds ratio: 3.2; 95% CI: 1.7 to 6.3; P < 0.001). Recalibration of the RAI to include platelet count and this new threshold restored the sensitivity of the original ≥8 threshold (95%), while improving its specificity (69%).

Conclusions

The RAI appears to be a sensitive and reliable tool for prediction of severe AKI in children with septic shock, although the use of a recalibrated sepsis-specific RAI using a higher cutoff and platelet count may be beneficial.

Graphical abstract

See Commentary on Page 1755

AKI occurs in up to half of all patients admitted to an intensive care unit (ICU) and is associated with increased morbidity and mortality.^1,2 Similarly, septic shock is a frequent diagnosis in critically ill patients, and is the most common cause of AKI in the ICU.3, 4, 5 Although both AKI and septic shock confer increased risk for poor outcomes alone, the coincidence of these diagnoses portends even worse outcomes, with some studies citing mortality rates as high as 70%.6, 7, 8, 9, 10, 11 Although these consequences of sepsis-associated AKI (SA-AKI) are now well-recognized, treatment remains limited to supportive care and renal protection strategies, with no effective disease-modifying treatments identified to date.^12,13

The lack of therapeutic options is not unique to SA-AKI, as therapies for AKI as a whole remain limited.¹⁴ As a result, significant focus has appropriately been directed on developing tools for early identification of patients at risk for severe, persistent AKI. Such a tool may allow for proactive intervention in these patients — such as avoiding unnecessary nephrotoxin exposure, excess chloride, and limiting exacerbating factors like fluid overload — that may improve outcomes, a hypothesis that has proven to be true in some patient populations.15, 16, 17 Although multiple AKI biomarkers18, 19, 20, 21, 22, 23, 24 and clinical prediction models25, 26, 27 have been developed and validated in a translational research setting, few of these tools are widely available for use in clinical practice at this time.

The RAI is a validated scoring tool for the prediction of severe AKI (≥100% increase in serum creatinine over baseline) in heterogeneous populations of critically ill children and adolescents.^26,27 Incorporating both AKI risk and injury criteria, the RAI is typically calculated 8 to 12 hours after pediatric ICU (PICU) admission, with a threshold score of 8 or higher defining fulfillment of renal angina (RAI+) (Figure 1). RAI+ outperforms both serum creatinine elevation and severity of illness scores for the prediction of severe AKI 72 hours later,²⁷ and given its relative ease of calculation, represents a feasible tool that can be incorporated into clinical practice in most PICUs.²⁸ Although patients with septic shock were included in both the derivation and validation cohorts, the RAI performance exclusively in this particularly high-risk subset of patients is unknown. Patients with septic shock often have high vasoactive and mechanical ventilation requirements, suffer high rates of early serum creatinine elevation, and require significant fluid resuscitation.²⁹ Because the RAI incorporates these variables (Figure 1), we hypothesized a priori that the RAI would likely require recalibration to be fully optimized for this subset of patients. We suspected that inclusion of decreased platelet count — which has a well-demonstrated association with SA-AKI — may improve the performance of this tool.^30,31

Figure 1.

The renal angina index. Calculated as the product of the highest risk and injury strata, a score of ≥8 has been previously defined as “fulfillment of renal angina,” and validated to predict the presence of severe acute kidney injury 3 days later. %FO, percent fluid overload from admission; RAI, renal angina index; SCr > baseline= elevation of serum creatinine above baseline value.

Therefore, we examined the RAI performance for prediction of severe SA-AKI at day 3 of septic shock (D3 severe SA-AKI) in a large cohort of pediatric patients, including an assessment of its predictive capacity in comparison to serum creatinine elevation alone, and severity of illness. Additionally, we sought to determine the optimal cutoff for defining RAI+ in this cohort, and to identify the potential benefit of inclusion of day 1 platelet count in a septic shock-specific RAI.

Methods

Study Design and Patient Selection

The reporting of this study follows the Strengthening the Reporting of Observational Studies in Epidemiology statement for cohort studies, which is included in the Supplementary Material. We performed a secondary analysis of a multicenter, prospective, observational cohort study of children aged 1 week to 18 years who were admitted to the PICU with septic shock.^29,32 The original study included patients from 14 PICUs across the United States and was conducted from January 2015 to December 2018.³³ After approval from the institutional review boards at each site, patients meeting criteria for pediatric septic shock were enrolled after obtaining informed consent from parents or legal guardians³²; the only exclusion criteria was the inability to obtain informed consent. The remainder of the original study protocol has been described in detail.^34,35 Once enrolled, clinical and laboratory data were collected daily during PICU admission for up to 7 days, and mortality was tracked for 28 days after enrollment. Patients from the original study (n = 461) were excluded from our analysis if they had a pre-existing history of kidney disease or were missing day 1 (D1) or D3 serum creatinine data (n = 82, final total cohort n = 379). Fifty-one of 379 patients included in our analysis were discharged from the PICU alive before D3 and without evidence of AKI, and these patients were presumed not to have D3 severe AKI. There were no other missing data for any included patient in our cohort.

RAI Calculation

The RAI was calculated for each patient based on the previously published and validated scoring system,^26,27 using clinical and demographic information from the first 24 hours of septic shock (Figure 1). Given the limitations of our dataset, a few modifications are worth noting. First, we lacked data specifically from the 8- to 12-hour time point that is typically used for calculation, and thus relied on data collected within the first 24 hours. Second, to avoid excluding patients receiving renal replacement therapy (RRT) — which can impact serum creatinine — at the time of enrollment (n = 24), we assigned these patients the highest possible injury score of 8. Based on previously published work, we defined RAI+ a priori as an RAI score of 8 or higher.^26,27

The Updated Pediatric Sepsis Biomarker Risk Model (PERSEVERE-II)

As part of the original study, each patient was assigned an updated Pediatric Sepsis Biomarker Risk Model (PERSEVERE-II) mortality probability. PERSEVERE-II is a validated tool incorporating prognostic biomarkers and platelet count for estimating the baseline mortality risk among children with septic shock.³³ Thus, we opted to include the PERSEVERE-II mortality probability to adjust for baseline mortality risk, in addition to the more widely used and validated updated Pediatric Risk of Mortality Score (PRISM III).³⁶

Outcomes and Definitions

The primary outcome of interest was the ability of the RAI to predict D3 severe SA-AKI, which was defined as Kidney Disease Improving Global Outcomes stage 2 AKI or higher (at least a doubling of serum creatinine from baseline).³⁷ Baseline serum creatinine values were unknown for all patients in our cohort, and thus estimated values were calculated for each patient using their body surface area (m²) and an estimated glomerular filtration rate of 120 ml/min per 1.73 m², as previously validated.^2,38 For patients without documented heights (n = 20), the age-based Pottel method was used, which has also been previously validated in our center’s RAI research.²⁸ Accurate urine output data were not available in this cohort.

As a secondary outcome, we compared the RAI to degree of serum creatinine elevation above baseline alone and severity of illness (by both PRISM III and PERSEVERE-II) for prediction of D3 severe SA-AKI. Additionally, based on our hypothesis that the RAI may need to be recalibrated for children with septic shock, we first performed an assessment of the optimal cutoff point to define renal angina fulfillment (RAI+) in these patients. Second, based on the a priori hypothesis that lower platelet counts would be associated with increased rates of severe AKI, we assessed both for this independent association and for potential improvement in the predictive capacity of the RAI by inclusion of this readily available laboratory value (platelet-modified RAI).

We also assessed the ability of the RAI to predict additional adverse renal outcomes, including need for RRT, the development of fluid overload (FO), and mortality. Daily percent FO was calculated using the previously published and commonly used formula³⁹:

$$\text{\%FO}=\left[\left(\text{Total}\text{Fluid}\text{In}\left(1\right)-\text{Total}\text{Fluid}\text{Out}\left(1\right)\right)/\text{Admission}\text{Weight}\left(\text{kg}\right)\right]\text{×}100\text{\%}$$

Statistical Analyses

Data were initially described using medians, interquartile ranges (IQRs), frequencies, and percentages. Comparisons between groups were performed using Wilcoxon rank sum, chi square, or Fisher exact test, as appropriate. Receiver operator characteristic (ROC) curves were generated and the AUROCs were compared to assess the predictive performance of both continuous and dichotomized predictor variables. Sensitivities, specificities, positive predictive values (PPVs), negative predictive values (NPVs), positive likelihood ratios, and negative likelihood ratios were generated both for specific cutoff values of continuous predictor variables, and for dichotomous predictor variables. Youden’s index was calculated for the RAI to determine the optimal cutoff point for prediction of D3 severe SA-AKI. Multivariable logistic regression was used to assess the independent association between RAI+, serum creatinine (SCr) elevation above baseline on D1 of septic shock (SCr > baseline, calculated as D1 SCr divided by baseline SCr), age, severity of illness (by both PRISM III and PERSEVERE-II) and D1 platelet count with the development of AKI-related outcomes and 28-day mortality. P < 0.05 was considered statistically significant. All statistical analyses were performed using Sigmaplot 14.0 (Systat Software Inc., San Jose, California) and SAS 14.0 (SAS Institute, Cary, North Carolina).

Two sensitivity analyses were also performed. First, we excluded patients who left the PICU before D3 and without evidence of SA-AKI, to account for potential bias via their inclusion. Additionally, given the lack of baseline SCr values and reliance on a normative glomerular filtration rate calculation, we also excluded patients younger than 1 year of age (n = 54) to assess for any differences in AKI incidence or RAI performance, given the potential to underestimate baseline creatinine in these very young patients.

Performance of the platelet-modified RAI (pltRAI) model described above was tested using a five-fold cross-validation procedure. Net reclassification improvement (NRI) was also used to estimate the incremental predictive ability of the pltRAI model, relative to the previously used RAI score ≥ 8.⁴⁰ The NRI was computed using R-packages pROC and Hmisc.

Results

Baseline Characteristics

The study cohort consisted of 379 patients, 207 (54.6%) of whom met criteria for renal angina fulfillment (RAI+) with an RAI score ≥8. A total of 65 of 379 (17.2%) had D3 severe SA-AKI. Table 1 describes the clinical, demographic, and outcome data according to the presence (RAI+) or absence (RAI-) of renal angina fulfillment. Patients who were RAI+ were younger, more likely to have a history of transplantation, had higher severity of illness by both PRISM-III score and PERSEVERE-II mortality probability, and were more likely to require vasoactive infusions, mechanical ventilation, and have >10% FO in the first 24 hours of septic shock. RAI+ patients also suffered worse outcomes than RAI- patients, including higher risk of D3 severe SA-AKI, RRT use, D3 FO, prolonged PICU length of stay, and 28-day mortality. Although there were no differences noted in the D1 platelet count between RAI+ and RAI- patients, platelet counts were lower in those who developed D3 severe SA-AKI versus those who did not (78 × 10³/μl vs. 167 × 10³/μl, P < 0.001).

Table 1.

Clinical, demographic and outcome variables by the presence of renal angina fulfillment (renal angina index ≥ 8) on day 1 of pediatric septic shock

	All	RAI-	RAI+	Comparison RAI+ to RAI-
N (%)	379	172 (45)	207 (55)	—
Male	195 (52)	86 (50)	109 (53)	P = 0.68
Age, years	6.3 [1.9 to 12.6]	8.1 [4.4 to 13.7]	4.9 [1.2 to 10.4]	P < 0.001
History of transplantation	47 (12)	13 (8)	34 (16)	P = 0.014
Severity of illness
PRISM III	10.2 [7 to 15]	10 [5 to 12]	12 [8 to 18]	P < 0.001
PERSEVERE-II	0.019 [0.007 to 0.189]	0.007 [0.007 to 0.167]	0.019 [0.007 to 0.189]	P < 0.001
D1 vasoactive use	332 (88)	129 (75)	203 (98)	P < 0.001
D1 mechanical ventilation	255 (67)	69 (40)	186 (90)	P < 0.001
D1 >10% fluid overload	48 (12.6)	7 (4.1)	41 (19.8)	P < 0.001
D1 platelet count, × 103/μl
All patients	153 [70, to 250]	165 [77 to 251]	129 [60 to 249]
Severe D3 SA-AKI	78 [26 to 154]a	—	76 [36 to 154]a	P = 0.25
No severe D3 SA-AKI	167 [86 to 263]a	—	170 [101 to 283]a	—
Causative organism
Gram positive	81 (21.4)	35 (20.3)	46 (22.2)	P = 0.75
Gram negative	96 (25.3)	47 (27.3)	49 (23.7)	P = 0.49
Viral	29 (7.7)	12 (7)	17 (8.2)	P = 0.80
Fungal	10 (2.6)	2 (1.2)	8 (3.9)	P = 0.19
None	163 (43)	76 (44.2)	87 (42)	P = 0.75
D3 SA-AKI
All stage	95 (25)	8 (4.7)	87 (42)	RR 9.0 (4.5 to 18.1, P < 0.001)
Severe	65 (17)	1 (0.6)	64 (31)	RR 53.2 (7.5 to 379, P < 0.001)
D3 fluid overload
>10%	104 (27)	26 (15)	78 (38)	RR 2.5 (1.7 to 3.7, P < 0.001)
>20%	28 (7.4)	6 (3.5)	22 (11)	RR 3.0 (1.3 to 7.3, P = 0.014)
RRT use	38 (7.4)	1 (0.6)	37 (18)	RR 30.7 (4.3 to 222, P < 0.001)
PICU LOS, days	7 [3 to 13]	4 [2 to 10]	8 [5 to 19]	P < 0.001
Mortality	42 (11)	7 (4.1)	35 (17)	RR 4.2 (1.9 to 9.1, P < 0.001)

D1/3, day 1/3; IQR, interquartile range; LOS, length of stay; PICU, pediatric intensive care unit; PERSEVERE-II= updated Pediatric Sepsis Biomarker Risk Model mortality probability; PRISM III, Pediatric Risk of Mortality Score III; RAI, renal angina index; RAI+, renal angina fulfillment; RR, relative risk; RRT, renal replacement therapy; SA-AKI, sepsis-associated acute kidney injury.

Values are n (%) unless otherwise noted. All continuous variables reported as median (IQR).

^aComparison in each group for D3 severe SA-AKI versus no D3 severe SA-AKI: p<0.001

Table S1 outlines the incidence of the components of the RAI by the presence or absence of D3 severe SA-AKI. Patients who went on to develop D3 severe SA-AKI had significantly higher incidence of a history of transplantation, D1 vasoactive and mechanical ventilation use, D1 FO >15%, and D1 SCr values more than twice baseline, compared to those who did not. In addition, Table S2 shows the percentage of patients in each RAI score category and their associated outcomes. Patients with the maximum RAI score of 40 (n = 55) suffered the highest incidence of all outcomes, including D3 severe SA-AKI (70.9%), RRT use (43.6%), and 28-day mortality (32.7%).

Assessment of the RAI for Prediction of Severe D3 SA-AKI

The RAI had an AUROC of 0.90 (95% CI: 0.86 to 0.93, P < 0.001) for discriminating between patients with versus without D3 severe SA-AKI. The test characteristics of this prediction at the previously defined cutoff for RAI+ of ≥8 are outlined in Table 2. At this cutoff, the RAI had a sensitivity of 98% (95% CI: 91% to 99%), NPV of 99% (95% CI: 96% to 99%), specificity of 54% (95% CI: 49% to 60%), and PPV of 31% (95% CI: 25% to 38%). Sensitivity analyses excluding patients who had left the PICU before D3 (n = 51), or those younger than 1 year of age (n = 54) showed similar performance of the RAI for D3 severe SA-AKI prediction, as outlined in Tables S3 and S4, respectively.

Table 2.

Comparison of renal angina fulfillment to initial serum creatinine above baseline on day 1 of pediatric septic shock

Characteristic	RAI+	SCr>Baseline	Comparison RAI+ to SCr>Baseline
N (%)	207 (55)	220 (58)	—
Male	109 (53)	112 (51)	P = 0.79
Age, years	4.9 [1.2 to 10.4]	6.3 [1.6 to 12.3]	P = 0.19
History of transplant	34 (16)	30 (14)	P = 0.50
Severity of illness
PRISM-III	12 [8 to 18]	12 [8 to 18]	P = 0.52
PERSEVERE-II	0.019 [0.007 to 0.189]	0.019 [0.007 to 0.189]	P = 0.84
RAI	20 [10 to 40]	10 [6 to 24]	P < 0.001
D3 SA-AKI
All stage	87 (42)	85 (39)	RR 1.1 (0.86 to 1.4, P = 0.54)
Severe	64 (31)	58 (26)	RR 1.2 (0.87 to 1.6, P = 0.35)
D3 Severe SA-AKI Prediction
AUROCa	0.90 (0.86 to 0.93)	0.85 (0.80 to 0.91)	P = 0.11
Sensitivity, %	98 (91 to 99)	89 (78 to 95)	—
Specificity, %	54 (49 to 60)	48 (43 to 54)	—
PPV, %	31 (25 to 38)	26 (21 to 33)	—
NPV, %	99 (96 to 99)	92 (91 to 98)	—
Positive likelihood ratio	2.2 (1.9 to 2.4)	1.7 (1.5 to 2.0)	—
Negative likelihood ratio	0.03 (0.004 to 0.20)	0.22 (0.11 to 0.45)	—
RRT use	37 (18)	32 (15)	RR 1.2 (0.8 to 1.9, P = 0.42)
PICU LOS, days	8 [5 to 19]	7 [3 to 13]	P = 0.005
Mortality	35 (17)	34 (16)	RR 1.1 (0.71 to 1.69, P = 0.78)

AUROC, area under the receiver operating curve; D, day; IQR, interquartile range; LOS, length of stay; NPV, negative predictive value; PERSEVERE-II, updated Pediatric Sepsis Biomarker Risk Model mortality probability; PICU, pediatric intensive care unit; PPV, positive predictive value; PRISM-III, Pediatric Risk of Mortality Score; RAI+, renal angina index ≥ 8; RRT, renal replacement therapy; SA-AKI, sepsis-associated acute kidney injury; SCr > baseline, serum creatinine above baseline

Values are n (%) unless otherwise noted. All continuous variables reported as median [IQR].

^aAUROC reported for RAI and SCr/Baseline as continuous variables

Comparison of the RAI to Severity of Illness and SCr Alone

We assessed RAI performance compared to D1 SCr > baseline alone because change in SCr is an important component of the RAI score (Figure 1). A comparison of clinical, demographic, and outcome variables for RAI+ and D1 SCr > baseline patients is shown in Table 2, along with a comparison of their D3 severe SA-AKI predictive performance. As noted previously, 207 patients (54.6%) were RAI+ on D1 of septic shock, whereas 220 patients (58%) had SCr > baseline on D1. There were no differences in sex distribution, age, history of transplantation, or severity of illness scores between these groups; however, RAI+ patients had higher median RAI values on the day of admission (20, IQR: 10 to 40 vs. 10, IQR: 6 to 24, P < 0.001). Compared to those with SCr > baseline, patients who were RAI+ had longer PICU lengths of stay (8 days [IQR: 5 to 19 days] vs. 7 days [IQR: 3 to 13 days], P = 0.005); no differences were noted in other outcomes between the two groups. The RAI AUROC did not differ from that of SCr > baseline (0.90; 95% CI: 0.86 to 0.93 vs. 0.85; 95% CI: 0.80 to 0.91; P = 0.11) for prediction of D3 severe SA-AKI. However, RAI+ failed to identify only 1 of 65 patients who had D3 severe SA-AKI. Conversely, having SCr > baseline failed to identify 7 of 65 patients who developed D3 severe SA-AKI, including 6 who went on to require RRT. When multivariable logistic regression analysis was performed adjusting for severity of illness (by both PERSEVERE-II and PRISM-III scores) and age, RAI+ status was independently associated with increased risk of D3 severe SA-AKI, RRT use and mortality (Table 3). By comparison, SCr > baseline was not associated with any of these outcomes.

Table 3.

Multivariable logistic regression testing for an association between renal angina fulfillment (RAI+), serum creatinine above baseline (SCr > baseline), age, and severity of illness with sepsis-associated acute kidney injury outcomes

Outcome	Variable	Adjusted OR	95% CI	P
D3 severe SA-AKI	PRISM-III	1.04	0.99 to 1.09	0.086
	PERSEVERE-IIa	1.46	1.18 to 1.80	<0.001
	Age, years	1.03	0.99 to 1.07	0.19
	RAI+	50.9	6.7 to 387	<0.001
	SCr > baseline	2.41	0.97 to 5.97	0.057
RRT use	PRISM-III	1.05	0.99 to 1.10	0.057
	PERSEVERE-IIa	1.66	1.31 to 2.11	<0.001
	Age, years	1.03	0.99 to 1.09	0.18
	RAI+	30.9	3.82 to 250	0.001
	SCr > baseline	1.09	0.39 to 3.07	0.88
28-day mortality	PRISM-III	1.03	0.98 to 1.08	0.22
	PERSEVERE-IIa	1.96	1.56 to 2.46	<0.001
	Age, years	1.03	0.98 to 1.08	0.30
	RAI+	3.40	1.22 to 9.45	0.019
	SCr > baseline	1.40	0.53 to 3.68	0.50

CI, confidence interval; D, days; OR, odds ratio; PERSEVERE-II, updated Pediatric Sepsis Biomarker Risk Model mortality probability; PRISM-III, Pediatric Risk of Mortality Score; RAI, renal angina index; RRT, renal replacement therapy; SCr, serum creatinine.

^aThe raw PERSEVERE-II mortality probability was transformed by a factor of 10 for the logistic regression analyses.

Recalibrating the RAI for Pediatric Septic Shock

Given the relatively low specificity of the RAI at a threshold score of 8 or higher (54%; 95% CI: 49% to 60%), we assessed different RAI cutoffs for prediction of D3 severe SA-AKI with Youden’s index, which identified a score of ≥20 (Youden’s index, 0.63) as most optimal. Table 4 outlines the test characteristics of the RAI at this cutoff; although the specificity and PPV of RAI+ improved at this cutoff (54% to 80% and 31% to 47%, respectively), this was at the cost of reduced sensitivity and NPV (98% to 83% and 99% to 96%, respectively). This reduction in sensitivity resulted in 11 missed patients with D3 severe SA-AKI, including 5 who went on to require RRT.

Table 4.

Comparison of three different definitions of renal angina fulfillment in pediatric septic shock for prediction of day 3 severe SA-AKI

Characteristic	RAI ≥ 8	RAI ≥ 20	pltRAI+	P
N (%)	207 (55)	116 (31)	160 (42)	—
Male	109 (53)	63 (54)	87 (54)	0.93
Age, years	4.9 [1.2 to 10.4]	4 [1.3 to 9.8]	4.2 [1.4 to 9.5]	0.93
History of transplantation	34 (16)	25 (22)	34 (21)	0.39
Severity of illness
PRISM-III	12 [8 to 18]	14 [10 to 19]	13 [9 to 19]	0.15
PERSEVERE-II	0.019 [0.007 to 0.189]a	0.178 [0.019 to 0.189]a	0.167 [0.007 to 0.189]	0.017
D1 vasoactive use	203 (98)	115 (99)	158 (99)	0.72
D1 mechanical ventilation use	186 (90)	108 (93)	145 (91)	0.62
D3 SA-AKI
All stage	87 (42)	70 (60)	80 (50)	0.007
Severe	64 (31)	54 (47)	62 (39)	0.018
D3 severe SA-AKI prediction
AUROC (dichotomous)b	0.76 (0.73 to 0.80)c	0.82 (0.77 to 0.87)c,d	0.82 (0.78 to 0.86)c	—
Sensitivity, %	98 (91 to 99)	83 (71 to 91)	95 (86 to 99)	—
Specificity, %	54 (49 to 60)	80 (75 to 84)	69 (63 to 74)	—
PPV, %	31 (25 to 38)	47 (37 to 56)	39 (31 to 47)	—
NPV, %	99 (96 to 99)	96 (92 to 98)	99 (96 to 99)	—
Positive likelihood ratio	2.2 (1. to 2.4)	4.2 (3.3 to 5.4)	3.1 (2.6 to 3.6)	—
Negative likelihood ratio	0.03 (0.004 to 0.20)	0.21 (0.12 to 0.36)	0.07 (0.02 to 0.20)	—
Youden’s index	0.52	0.63	0.64	—
RRT use	37 (18)	33 (28)	37 (23)	0.084
PICU LOS, days	8 [5 to 19]	9 [6 to 19]	9 [6 to 19]	0.82
Mortality	35 (17)	30 (26)	34 (21)	0.154

AUROC, area under the receiver operating curve; D, day; IQR, interquartile range; LOS, length of stay; NPV, negative predictive value; PERSEVERE-II, updated Pediatric Sepsis Biomarker Risk Model mortality probability; PICU, pediatric intensive care unit; pltRAI+, meets renal angina criteria by platelet count-modified RAI, which is defined as 1) RAI ≥ 20, or 2) RAI 8 to <20 with platelet count <150 × 10³/μl; PPV, positive predictive value; PRISM-III, Pediatric Risk of Mortality Score; RAI, renal angina index; RRT, renal replacement therapy; SA-AKI, sepsis-associated acute kidney injury.

Values are n (%) unless otherwise noted. All continuous variables reported as median [IQR].

^aThese two groups were different from each other on pairwise comparison.

^bAUROC reported for RAI ≥8, RAI ≥20, and pltRAI+ as dichotomous variables.

^cStatistically different groups on pairwise comparison.

^dRAI ≥ 8 was significantly different than both RAI ≥ 20 and pltRAI+.

In an effort to preserve the sensitivity of the RAI at the original cutoff of ≥8 and to improve its specificity, we further recalibrated the RAI for sepsis using platelet count information to generate a sepsis-specific pltRAI. As mentioned previously, patients who developed D3 severe SA-AKI had a lower D1 platelet count. Of the 11 patients with D3 severe SA-AKI who were missed by an RAI cutoff of ≥20, 9 (81.8%) had a D1 platelet count below 150 × 10³/μl. Furthermore, a D1 platelet count below this threshold was associated with D3 severe SA-AKI (adjusted odds ratio: 3.2; 95% CI: 1.7 to 6.3; P < 0.001), after adjustment for age, PERSEVERE-II and PRISM-III scores, SCr > baseline, and RAI. Thus, we redefined renal angina fulfillment (in this case, termed pltRAI+) by defining at-risk patients to be those with either: (1) an RAI score ≥20, or (2) an RAI score from 8 to <20 with a platelet count <150 × 10³/μl. This recalibration improved the sensitivity of prediction compared to the higher RAI ≥20 cutoff alone (83% to 95%), while increasing the specificity (54% to 69%) and PPV (31% from 39%) relative to the original cutoff of RAI ≥8. Furthermore, the now dichotomous pltRAI had an AUROC for prediction of D3 severe SA-AKI of 0.82 (95% CI: 0.78 to 0.86), which proved superior when compared to a dichotomized RAI ≥8 (AUROC: 0.76; 95% CI: 0.73 to 0.80; P = 0.0001). On five-fold cross-validation, the pltRAI had a summary AUROC of 0.82, with an average sensitivity of 95% and specificity of 69%. A complete comparison of the predictive capacity of all three definitions of renal angina fulfillment is outlined in Table 4. Additionally, a breakdown of patients by pltRAI designation (positive and predicted to have D3 severe SA-AKI, or negative and predicted not to have D3 severe SA-AKI) and their associated outcomes is shown in Table S2.

To further quantify the improvement in prediction of D3 severe SA-AKI seen when using the pltRAI model over the RAI at the original cutoff of 8, we performed an NRI analysis. The pltRAI model showed improved predictive capacity compared to RAI ≥ 8, with an NRI of 0.225 (95% CI: 0.11 to 0.34; P = 0.0001). This NRI reflected an NRI for events of 0.94 (95% CI: 0.85 to 1.02; P < 0.0001) and for nonevents of 0.71 (95% CI: 0.79 to 0.64; P < 0.0001).

Discussion

In this secondary analysis of a large prospective study of children with septic shock, we found the RAI reliably predicted the presence of D3 severe SA-AKI. Whereas the previously defined cutoff of 8 or higher used in general populations of critically ill children was extremely sensitive and superior to both SCr elevation alone and severity of illness, an optimal cutoff assessment indicated that a higher RAI score threshold of 20 may be more appropriate in these patients, although at the cost of reduced sensitivity. Additionally, we identified admission platelet count below 150 × 10³/μl to be an independent predictor of D3 severe SA-AKI in this cohort. Incorporation of this readily available variable and the newly derived septic shock–specific cutoff point allowed us to recalibrate the RAI in a manner that preserved the high sensitivity of the original threshold while improving its specificity and overall predictive ability.

Given the variables used to calculate an RAI score, the need to recalibrate this tool for patients with septic shock is not surprising. By definition, a diagnosis of septic shock requires significant circulatory dysfunction,²⁹ which often results in the need for vasoactive infusions and mechanical ventilation. Almost two-thirds of our cohort (n = 240, 63.3%) required both vasoactive medications and mechanical ventilation in the first 24 hours of septic shock, equating to the maximum RAI risk score of 5 (Figure 1). Additionally, given the pathophysiology of septic shock and the fact that fluid resuscitation is a mainstay of therapy, it is not uncommon for these patients to develop early (i.e., within the first 24 hours) SCr elevation, need for RRT,^7,8 and/or positive fluid balance,^41,42 all of which have the potential to increase the RAI injury score (Figure 1). These trends were seen in our cohort as 220 (58%) had SCr elevation, 24 (6.3%) required RRT, and 48 (12.6%) had greater than 10% fluid accumulation in the first 24 hours. The high incidence of significant RAI risk and injury criteria in our cohort resulted in more than half of patients fulfilling renal angina criteria at the previous cutoff of 8. Although this cutoff was highly sensitive (capturing all but one patient with D3 severe SA-AKI), it is not surprising that the use of a higher threshold to optimize specificity improved overall predictive performance in this cohort.

However, the RAI was intentionally designed to be a highly sensitive tool for ruling out AKI, particularly given the aforementioned dire consequences of D3 severe SA-AKI, as well as the reliance on early supportive care as the mainstay of therapy. Although added specificity of a tool allows for improved diagnostic capability, given what we now know about the complexity of SA-AKI pathophysiology,^12,13 it is unlikely that one tool alone will be able to predict severe SA-AKI with both excellent sensitivity and specificity. Ultimately, the most logical approach likely involves the initial use of a highly sensitive test such as the RAI for risk stratification, followed by refinement of that risk assessment via the targeted use of more specific SA-AKI prediction tools,^19,25 a step-wise approach that has been previously shown to be effective in general populations of critically ill children.^23,43 Additionally, different RAI cut-off values may be selectively used by clinicians depending on needs of the care team. If the goal is to rule out AKI, then a high-sensitivity cut-off can be selected (i.e., RAI ≥ 8). For patients in whom the risk of adding a nephrotoxic drug is being considered, a more specific cut-off may be contemplated (i.e., RAI ≥20). Thus, values across the RAI range can be harnessed to improve clinical decision-making on an individual patient level.

Although a staged approach to SA-AKI prediction such as the one outlined above would be ideal, the reality is that the cited biomarkers and tools have not yet been adequately studied in pediatric septic shock, nor are they widely available for use. As such, we recalibrated the RAI for septic shock using platelet count — a readily available clinical parameter identified in our cohort to be independently associated with D3 severe SA-AKI — to redefine renal angina fulfillment. Inclusion of decreased platelet count in this sepsis-specific RAI is also supported by evidence, as the association between thrombocytopenia and SA-AKI has both a plausible pathophysiology (i.e., driven by sepsis-induced endothelial dysfunction and microvascular thromboses similar to thrombotic microangiopathy), and has been previously documented in the literature.^30,31 Ultimately, the pltRAI was able to show improved specificity of severe SA-AKI prediction while maintaining high sensitivity. Although these improvements were small, in the absence of other approved bedside tools for SA-AKI prediction, the pltRAI represents an immediately feasible mechanism for identifying at-risk patients in the PICU.

The ability to accurately identify patients at risk for severe, persistent SA-AKI is important for several reasons. First, it is now well-documented that SA-AKI is common in both adults and children, impacting up to half of all patients admitted with septic shock.^6,7,9 Second, its consequences are severe, including higher odds of death in afflicted children^9,10 and serious health-related quality of life deterioration in those who survive.¹¹ Unfortunately, despite knowledge of its prevalence and consequences, we continue to lack definitive disease-modifying therapy for AKI once present, relying exclusively on supportive care and renal protective strategies. These treatment mainstays have been shown in some populations to be more effective when implemented early,^15,16 thus highlighting the need for reliable risk stratification tools for SA-AKI. Beyond merely identifying patients who may benefit from early, proactive intervention, such a tool may also facilitate the study of novel therapeutics, as it allows for identification of a high-risk subset of patients appropriate for informed clinical trial enrollment in the future.

This study has several strengths. Importantly, this is the first study examining the use of the RAI exclusively in a large population of children with septic shock. The study cohort was large, and comprised of patients from multiple centers across the United States, enhancing the generalizability of the results. We were also able to use the PERSEVERE-II mortality probability as an additional discriminator of septic shock illness severity, potentially enhancing the validity of our regression analyses.

Study Limitations

Our work also has important limitations. This is a secondary analysis of an observational study that was not intended to study SA-AKI or the RAI, and as such, there is a possibility of unintended bias. Additionally, the RAI had to be manually calculated from data collected at varying time points within the first 24 hours of septic shock, a slightly different timeframe from the previously published work in this area. Baseline SCr data were not available; thus, we relied on estimated values for all patients, potentially impacting the validity of the reported AKI rates in this cohort. Finally, accurate urine output data were also unavailable, raising the possibility that rates of SA-AKI, as defined by serum creatinine alone, were underestimated.

Conclusions

We conclude that the RAI is a feasible tool for the early and accurate estimation of D3 severe SA-AKI risk in critically ill children with septic shock. Although highly sensitive at its originally defined threshold of 8, recalibration of the RAI to incorporate the optimal septic shock-specific cutoff of 20, as well as decreased platelet count, appears to increase the specificity of prediction without sacrificing significant sensitivity. Pending prospective validation, the pltRAI has the potential to aid in the identification of patients most likely to benefit from early intervention to mitigate the risk of severe SA-AKI, and to consider for future enrollment in clinical trials aiming to identify novel therapies.

Disclosures

Supported by the National Institute of General Medical Sciences, R35GM126943 (HRW), and the National Center for Advancing Translational Sciences of the National Institutes of Health, 2KL2TR001426-05A1 (NLS). The content is solely the responsibility of the authors and does not necessary represent the views of the National Institutes of Health.

NLS and HRW are named as co-inventors on a provisional patent application submitted by the Cincinnati Children’s Research Foundation for the PERSEVERE-II Model cited in this manuscript. SLG and RKB are co-inventors of the Renal Angina Index.

Supplementary Material

Table S1. Incidence and comparison of Renal Angina Index components by the presence or absence of severe acute kidney injury at day 3 of septic shock.

Table S2. Distribution of outcomes by both Renal Angina Index (RAI) score and platelet-modified Renal Angina Index (pltRAI) designation.

Table S3. Comparison of demographic data, clinical outcomes and renal angina index (RAI) performance in patients still admitted to the pediatric intensive care unit at day 3 (n=328) compared to the entire included cohort (n=379).

Table S4. Comparison of demographic data, clinical outcomes and renal angina index (RA).

STROBE Statement