Risk prediction models for AKI in critically ill patients: opus in progressu

Abstract

Background

Acute kidney injury (AKI) is a complex systemic syndrome associated with high morbidity and mortality. Among critically ill patients admitted to intensive care units (ICUs), the incidence of AKI is as high as 50% and is associated with dismal outcomes. Thus, the development and validation of clinical risk prediction tools that accurately identify patients at high risk for AKI in the ICU is of paramount importance.

Summary

We provide a comprehensive review of three clinical risk prediction tools that have been developed for incident AKI occurring in the first few hours or days following admission to the ICU. We found substantial heterogeneity among the clinical variables that were examined and included as significant predictors of AKI in the final models. Area under the receiver operating characteristic curves was ~0.8 for all three models, indicating satisfactory model performance, though positive predictive values ranged from only 23 to 38%. Hence, further research is needed to develop more accurate and reproducible clinical risk prediction tools. Strategies for improved assessment of AKI susceptibility in the ICU include the incorporation of dynamic (time-varying) clinical parameters, as well as biomarker, functional, imaging, and genomic data.

INTRODUCTION

Approximately 5 million patients are admitted to intensive care units (ICUs) in the US each year, and as many as 50% of these patients suffer from acute kidney injury (AKI).^1–3 AKI is a complex systemic syndrome associated with high morbidity, mortality, resource utilization, and risk for kidney and non-kidney related complications.^3,4 Survivors of AKI are at higher risk for development or progression of chronic kidney disease (CKD),^5,6 end-stage renal disease (ESRD),⁷ and cardiovascular disease.^8,9 Early risk stratification for AKI in the ICU is challenging, but is essential in developing novel strategies for AKI prevention and treatment.¹⁰ In this article, we discuss the evolving field of clinical risk prediction tools for AKI in the ICU, including a review of existing models, as well as some of the challenges, limitations, and future directions for research in the field.

What has been done in relation to clinical risk prediction tools for AKI in the ICU?

Several investigators have developed and validated clinical risk prediction models for AKI among heterogeneous groups of critically ill patients admitted to the ICU. We focused on original manuscripts published during the last 10 years that included a sample size of more than 200 critically ill adults. Utilizing these criteria, we identified three studies (Tables 1A and 1B).

Table 1A.

Comparison of clinical risk prediction models for AKI in the ICU

	Kashani et al., 201311	Malhotra et al., 201713	Flechet et al., 201714
Sample size	n = 728	n = 2017	n = 4760
Single- or multi-center	Multi-center	Single-center	Multi-center
Type of ICU	Mixed	Mixed	Mixed
Validation cohort	No	External validation	Internal validation
Physiologic data available	No	Yes (limited)	Yes
Biochemical data available	Yes	Yes	Yes
#Risk factors in final model	9 (4 acute; 5 chronic)	10 (5 acute; 5 chronic)	13 (10 acute; 3 chronic)
Timeframe for examination of	Up to 5 days prior to study	Within 48h following ICU	Pre-admission; ICU
clinical variables	enrollment	admission	admission; Day 1
AKI definition	↑SCr ≥100% in 7d or UOP <0.5 ml/kg/h × >12h	↑SCr ≥0.3 mg/dl in 48h or ≥50% in 7d	↑SCr ≥0.3 mg/dl in 48h or ≥50% in 7d
RRT included in AKI definition	Yes	No	Yes
Timeframe for AKI determination	12h from biospecimen collection	7d from study enrollment	7d from ICU admission
Baseline SCr determination	See below*	Mean of all SCr measurements 7-365d prior to admission	See below**
How missing baseline SCr was addressed	SCr at the time of enrollment	SCr at ICU admission utilized	SCr back-calculated***
Incident AKI events (%)	14.0	37.2	28.5
PPV	NA	23% for risk score ≥5	38%
AUC	0.81	0.79 (internal validation) 0.81 (external validation)	0.86–0.88 (discovery) 0.80–0.84 (validation)

AKI = acute kidney injury; AUC = area under the receiver operating curve; ED = emergency department; eGFR = estimated glomerular filtration rate; ICU = intensive care unit; MDRD = modification of diet in renal disease; NA = not assessed; PPV = positive predictive value; RRT = renal replacement therapy; SCr = serum creatinine; UOP = urine output.

^*If at least 5 SCr values were available, the median of all values from 6 months to 6 days prior to enrollment was used. Otherwise, the lowest SCr value in the 5 days prior to enrollment was used.

^**The lowest SCr within 3 months prior to ICU admission was used for elective admissions, and the lowest SCr within 3 months to 1 week prior to ICU admission was used for patients transferred to the ICU from the emergency department.

^***Using an eGFR of 75 ml/min/1.73m² (MDRD equation).

Table 1B.

Variables included in clinical risk prediction models for AKI in the ICU: comparison across models

	Kashani et al., 201311	Malhotra et al., 201713	Flechet et al., 201714#
Demographics and comorbidities
Agea	X		X
Gender
Race
BMI >30 kg/m2
Baseline SCr			X
CKDb	X	X
Diabetes mellitus	X		X
Hypertension	X	X
Congestive heart failure		X
Chronic liver disease	X	X
Chronic lung disease
Cardiovascular disease		X
Cancer
HIV
Drug abuse
Acute severity-of-illness variables
Planned vs unplanned ICU admission			X
Surgical vs medical			X
Major surgery
Sepsis	X	X	X
Burns or trauma
Mechanical ventilation		X
Serum albumin
Anemiac		X
Nephrotoxin exposured	X	X
IV or IA radiocontrast exposure
Blood pH ≤7.30		X
Blood glucose on ICU admission			X
Hemodynamic supporte			X
SCr at time of enrollment/ICU Day 1	X		X
APACHE II or III score on Day 1	X		X
Serum lactate on Day 1			X
Serum bilirubin on Day 1			X
Total hours in ICU on Day 1			X

Blue and red boxes indicate variables that were assessed and were (blue) or were not (red) significant predictors of AKI in the final model; white boxes indicate that the variable was not assessed in the study. “X” denotes that the clinical parameter was included in the final model.

APACHE = Acute Physiologic Assessment and Chronic Health Evaluation; BMI = body mass index; CKD = chronic kidney disease; HIV = human immunodeficiency virus; IA = intra-arterial; ICU = intensive care unit; IV = intravenous; SCr = serum creatinine.

^#Based on “Day 1 model” only.

^aAge >70 years (Malhotra et al.); used as a continuous variable in the other 2 studies.

^bRecorded from the patient’s medical history (Kashani et al.); defined as eGFR <60 ml/min/1.73m², calculated using the CKD Epidemiology Collaboration equation (Malhotra et al.).

^cHematocrit <30% (Kashani et al.); Hemoglobin <9 mg/dL or hematocrit <27% (Malhotra et al.).

^dExposure to any of the following drugs within 5 days prior to and including the day of enrollment: NSAIDs, ACE inhibitors, Angiotensin II receptor blockers, calcineurin inhibitors, beta-lactam antibiotics, aminoglycosides, vancomycin, acyclovir, amphotericin, allopurinol, colistin (Kashani et al.); exposure within 7 days prior to ICU admission to IV or IA radiocontrast, ≥3 nephrotoxins (amphotericin, aminoglycosides, chemotherapy, anti-retroviral drugs, or NSAIDs), or ≥3 days of aminoglycosides (Malhotra et al.); exposure to nephrotoxins as a category was not evaluated by Flechet et al., but the following individual medication exposures during the first day of ICU stay were examined, and none was a significant predictor of AKI in the final model: vancomycin, aminoglycosides, anti-retrovirals, beta-lactam antibiotics, diuretics, calcineurin inhibitors, antifungals, NSAIDs, ACE inhibitors, and IV or IA radiocontrast.

^eHypotension (MAP <70 mmHg) within five days prior to and including the day of enrollment (Kashani et al.); MAP <70 mmHg or use of any vasopressor (Malhotra et al.); mechanical or pharmacological hemodynamic support (Flechet et al.).

Sapphire Cohort

Kashani and colleagues¹¹ conducted a prospective, multi-center cohort study in which they used a hypothesis-free approach to assess a panel of 340 candidate urinary and plasma biomarkers for the early detection of AKI among a heterogeneous group of mixed (medical and surgical) ICU patients. They found that urinary levels of tissue inhibitor of metalloproteinases 2 (TIMP2) and insulin-like growth factor-binding protein 7 (IGFBP7), which are markers of cell cycle arrest, were significant predictors of AKI. To assess the clinical utility of urinary TIMP2 and IGFBP7 in predicting AKI, they also developed a clinical model, and assessed its performance both with and without inclusion of the urinary biomarkers. Thus, the study was not designed specifically to develop a clinical risk prediction tool for AKI per se.

The clinical model tested 16 parameters characterized as exposures (acute risk factors) and susceptibilities (demographics and chronic risk factors). Exposures were evaluated up to 5 days prior to study enrollment. The final model included 9 variables (4 acute and 5 chronic; Table 1B). AKI was defined as stage ≥2 of the Kidney Disease: Improving Global Outcomes (KDIGO)-AKI criteria (SCr and urine output).¹² The timeframe for AKI assessment was the first 12 hours following study enrollment. This notably narrow window is a potentially significant limitation because many of the patients may have already been well on their way toward developing stage ≥2 AKI at the time of enrollment. Nonetheless, the final clinical model (without the inclusion of the urinary biomarkers of interest) exhibited an area under the receiving operating characteristic curve (AUC) of 0.81, with a 95% confidence interval (CI) of 0.76 to 0.85, for the prediction of AKI stage ≥2 (Table 1A).

UCSD-Mayo Cohort

Malhotra and colleagues¹³ developed a clinical risk prediction model for AKI in the ICU using two independent cohorts: a discovery cohort from the University of California San Diego (UCSD), and an external validation cohort from Mayo Clinic. They used forward elimination multiple logistic regression to select the variables for inclusion in their model, and developed a scoring system using regression coefficients. Candidate continuous variables were dichotomized based on optimal cutoffs (e.g., blood pH ≤7.30) using Youden’s index method.

The final model included 10 clinical risk factors for AKI (5 acute and 5 chronic; Table 1B). All variables were evaluated within 48 hours following ICU admission. Each risk factor was weighted by assigning a score of 0 (if absent) or 1, 2, or 3 points (if present). The minimum total score was 0 and the maximum total score was 21 points. AKI was defined as stage ≥1 of the KDIGO-AKI SCr-based criteria.¹² The timeframe for assessment of AKI was the first 7 days following study enrollment. The AUC for the discovery and external validation cohorts was 0.79 (95% CI, 0.69–0.88) and 0.81 (0.78–0.83), respectively, and the positive predictive value (PPV) was 23% for a risk score ≥5 (Table 1A). Notable limitations of the study were the lack of access to comprehensive physiologic data other than mean arterial pressure <70 mmHg, the absence of oliguria and need for renal replacement therapy in the definition of AKI, and the utilization of SCr at the time of ICU admission as the baseline SCr when pre-ICU SCr values were not available. The latter could have misclassified patients with ongoing AKI at the time of ICU admission as not having AKI.

The ‘AKI Predictor’

Flechet and colleagues¹⁴ conducted a retrospective study using the Early Parenteral Nutrition Completing Enteral Nutrition in Adult Critically Ill Patients (EPaNIC) database. They developed and validated a clinical risk stratification tool to predict AKI during the first 7 days of ICU stay. The candidate variable selection was performed using a random forest machine-learning scheme using correlation-based ranking algorithms. Clinical parameters were evaluated at three time points in relation to ICU admission: pre-admission, ICU admission, and Day 1 of ICU stay. In addition, the investigators divided the dataset into development and validation cohorts.

The final variable selection was determined incrementally for each time-based model via bootstrapped backward elimination and included 5 variables with data available at pre-admission, 3 variables with data available on ICU admission, and 5 variables examined during the first day (Day 1) of ICU stay (Table 1B). AKI was defined as stage ≥1 of KDIGO-AKI SCr-based criteria.¹² The timeframe for assessment of AKI was the first 7 days following ICU admission. The AUCs for the discovery cohort were 0.86 (95% CI, 0.86 to 0.86) and 0.88 (95% CI, 0.88 to 0.89) for AKI stage ≥1 and ≥2, respectively. These results were internally validated with repeated AUCs ≥0.80 (Table 1B). The reported PPV of this clinical risk prediction tool was 38%. Important limitations of this study were the absence of oliguria criterion in the AKI diagnosis and the assumption of a baseline estimated glomerular filtration rate (eGFR) of 75 ml/min/1.73m² for patients with missing baseline SCr value.

Comparison of clinical risk prediction models

Table 1A summarizes the design characteristics, definitions, and performance metrics among the three studies. Table 1B shows a comparison of the variables included in each of the models. As demonstrated, the only variables that replicated across more than one model were CKD, sepsis, and exposure to nephrotoxins, and no variable replicated across all three studies. Further, even among the variables that predicted AKI in more than one study, definitions used for these variables differed from study to study. For example, CKD was determined by Kashani et al. based on the patient’s medical history, while it was defined as baseline eGFR <60 ml/min/1.73m² by Malhotra et al. In contrast, Flechet et al. used baseline SCr rather than CKD in their risk prediction model. Similarly, the definition of nephrotoxin exposure varied widely across studies (Table 1B). Finally, many variables that predicted AKI in one study were simply not assessed by other studies.

What needs to be done to improve risk prediction tools for AKI in the ICU?

• Development of a large, heterogeneous, multi-center, and multi-ethnic cohort to refine, homogenize, and validate current clinical risk prediction models.

• Utilize advances in computer science and clinical informatics to incorporate dynamic (time-varying) clinical parameters that may be more informative for clinical risk prediction models than solely static clinical parameters.

• Incorporation of novel biomarkers of AKI into clinical risk prediction models. A suggested framework is provided in the Figure.

• Incorporation of functional studies (e.g., furosemide stress test), imaging studies, and genomic data into clinical risk prediction models for more individualized and precise detection of AKI susceptibility.

• Development of risk stratification tools for the prognostication of AKI recovery.

• Development of clinical decision support systems that enhance health-related decisions and timely interventions to prevent or attenuate AKI or promote its recovery.

• Strengthening of multidisciplinary collaborations among investigators from all latitudes, incorporating computer, bench, and clinical scientists.

Figure. Framework for implementation of biomarker testing into a clinical risk prediction tool for AKI in the ICU.

Abbreviations: ACE-I = angiotensin-converting enzyme inhibitor; ARB = angiotensin II receptor blocker; KDIGO = Kidney Disease: Improving Global Outcomes; RCTs = randomized controlled trials; SCr = serum creatinine, UOP = urine output.

Conclusions

It is essential to develop accurate and precise clinical risk prediction tools for AKI in the ICU, both for timely intervention that can prevent or mitigate AKI and for enrichment of patients in randomized controlled trials. We reviewed three existing AKI risk prediction tools and found substantial heterogeneity in the variables that were included in the final models. Further, although all models had satisfactory AUCs, PPV ranged from only 23 to 38%. Thus, additional research is needed to further validate, harmonize, and expand on existing clinical risk prediction tools for AKI in the ICU.

Key Messages.

• Accurate and precise clinical risk prediction of AKI in the ICU is essential for timely intervention and for enrichment in randomized clinical trials

• Existing clinical risk prediction tools for AKI in the ICU demonstrate substantial heterogeneity of included variables

• Additional research is needed to validate, harmonize, and expand on existing clinical risk prediction models for AKI in the ICU