Prediction Models for AKI

At present, AKI is defined by changes in serum creatinine or reduction in urine output (1). However, AKI is associated with numerous etiologies including sepsis, global hypoperfusion, and nephrotoxins. Current treatments for AKI are limited to supportive care. In this regard, AKI is similar to other critical illnesses like acute respiratory distress syndrome and sepsis, where targeted therapeutics have failed. One proposed reason for this failure is underlying disease heterogeneity (2). That is, therapies that are efficacious in certain subpopulations may appear ineffective if the intervention is tested in a larger, mixed cohort. Furthermore, it is often unclear how to subdivide disease populations into treatment-responsive subgroups. Recently, acute respiratory distress syndrome subphenotypes have been identified using novel biomarkers, and in reanalysis of completed clinical trials, have shown differential response to ventilatory (3), fluid management strategies (4), and statin use (5). Prediction models for AKI that incorporate novel biomarkers could similarly identify subpopulations that may benefit from effective targeted therapies to improve patient outcomes.

In this issue of the Clinical Journal of the American Society of Nephrology, Bhatraju et al. (6) introduce the “ACT” model, a three-variable equation comprising age, presence of cirrhosis, and plasma soluble TNF receptor-1 (sTNFR-1) level. This model predicts severe AKI in critically ill patients with two or more criteria for systemic inflammatory syndrome (SIRS) (7). A derivation cohort of 749 individuals from Harborview Medical Center was used to develop the model, which was subsequently internally validated in 326 patients and externally validated in 262 patients from Massachusetts General Hospital. The demographic distribution of the two cohorts were similar in age and sex. However, the Harborview participants were recruited from both medical (approximately 60%) and surgical intensive care units (ICUs), whereas Massachusetts General Hospital participants were mostly medical ICU patients (97%). As a result, the external validation cohort had more sepsis (90% versus 64%), higher illness severity (mean Acute Physiologic Assessment and Chronic Health Evaluation [APACHE] III score 73 versus 50), and greater vasopressor requirements (94% versus approximately 25%). The prevalence of CKD was unknown in the external validation cohort, but the two cohorts had similar mean admission creatinine (approximately 1.5 mg/dl and 1.2 mg/dl). Severe AKI was defined as a creatinine rise two times above baseline within 72 hours of enrollment. However, incident RRT was not included in the primary analysis as there was concern that dialysis might be initiated for reasons unrelated to worsening kidney function (e.g., toxin removal). The prevalence of AKI was 8% in the derivation and internal validation cohorts and 6% in the external validation cohort.

The ACT model was developed using nine clinical covariates and seven log-transformed novel biomarkers using least absolute shrinkage and selection operator regression (LASSO). LASSO is a type of machine learning that penalizes the magnitude of regression coefficients for less important factors and then excludes variables whose coefficients are penalized to zero. Such methods are useful when a large pool of predictors (such as biomarkers) have unknown significance. Rather than requiring a dictated hypothesis, LASSO analysis can help determine which predictors are most important. The authors then arrived at the three-variable ACT model by applying the “one standard deviation rule.” That is, the authors identified the simplest model where more complex regressions did not change the crossvalidation error by more than one SD.

To assess model performance, the authors compared the c-statistic of the ACT model to admission APACHE III scores and baseline serum creatinine. The c-statistic evaluates discrimination, with 0.5 indicating the model is no better than random chance and 1 indicating perfect prediction. The ACT model’s c-statistic was 0.95 (95% confidence interval [95% CI], 0.91 to 0.97) in the derivation cohort and remained strong in the internal (0.90; 95% CI, 0.82 to 0.96) and external validation (0.93; 95% CI, 0.89 to 0.97) cohorts. The ACT model performed statistically significantly better than the APACHE III score, but not better than baseline serum creatinine in the internal (0.85; 95% CI, 0.72 to 0.95) and external validation (0.93; 95% CI, 0.82 to 0.99) cohorts. This result may not be surprising because the APACHE III score was developed to predict mortality, not AKI. Furthermore, a potential challenge to the interpretation of the c-statistic for the ACT model is that the model likely predicted prevalent rather than incident AKI. In general, models that predict prevalent disease have higher c-statistics than models that predict incident disease (8). Specifically, biomarker data were collected within 24–48 hours of admission and the outcome was AKI within 72 hours of ICU admission. If those patients in whom severe AKI was observed within 48 hours were excluded, the c-statistic might be lower. In the subgroup with de novo severe AKI where the serum creatinine at study enrollment was ≤1.5 mg/dl, the c-statistic for the ACT model was 0.97 (95% CI, 0.92 to 1), which was higher than the c-statistic using baseline serum creatinine (0.65; 95% CI, 0.55 to 0.74). However, restriction of baseline serum creatinine to the ≤1.5 mg/dl range a priori limits the predictive value of a model that only contains serum creatinine. In addition, the lower baseline serum creatinine range likely identifies subjects where the model is predicting incident, not prevalent disease, and this may further “penalize” the baseline serum creatinine model.

To assess the clinical utility of the ACT model, cut-offs were determined where the negative predictive value (NPV) was 0.95 and positive predictive value (PPV) was 0.9 in the derivation cohort. Using the same cut-offs, specificity and NPV remained in the >0.9 in the internal and external validation cohorts. Although the authors suggest that the ACT model may be used to effectively eliminate low-risk patients for clinical trial enrollment, an NPV of approximately 95% may not be very effective when the incidence of severe AKI is 6%–8%. That is, the pretest probability of not developing AKI is 92%–94% for the overall cohort, and is only increased to approximately 95% by application of the model. Furthermore, the ACT model had limited sensitivity (as low as 0.07; 95% CI, 0.00 to 0.20) and PPV (as low as 0.50; 95% CI, 0.00 to 1.00) in the validation cohorts. Ultimately, the utility of the model in clinical trial enrollment would also require knowledge on the relative costs and benefits of false negatives and false positives of the intervention.

Historically, many epidemiologic studies have focused on risk factors for AKI. More recent novel biomarker studies have tested whether admission biomarkers may predict AKI. Although prior studies have included urinary biomarkers, the use of urinary biomarkers may be difficult in patients with anuria. This study is innovative in combining both traditional clinical risk factors and novel plasma biomarkers to evaluate AKI risk and using machine learning to identify a parsimonious model for risk prediction. The concept of using a prediction model to risk stratify patients for potential clinical trial enrollment is relatively novel in AKI.

The ACT model’s discriminatory ability for severe AKI suggests that sTNFR-1 levels, the presence of cirrhosis, and age may modify the risk of AKI. The high c-statistic, specificity, and NPV of the ACT model are impressive, but the clinical utility may be limited when the outcome is rare and the test has limited sensitivity and PPV. Regardless, this is an important article because it represents a key conceptual step toward identification of patients for clinical trials of AKI. Predictive models may allow for trial population enrichment to yield significant effects that are otherwise lost in a heterogenous environment. For low-risk/low-expense trials, the enrichment is less crucial, but may still provide efficiency of scale. However, for high-risk/high-expense trials, the enrichment would allow investigators to focus resources on those most likely to benefit. This study also demonstrates how machine learning techniques may eventually identify new directions for pathophysiologic investigations and allow for more personalized treatment for acute illness.

Furthermore, there is significant value in identifying sTNFR-1 as a key contributor to this model. Newer animal models suggest that sepsis associated AKI is a state of hyperperfusion and may be mediated by dysregulated toxic and inflammatory factors (9). In this cohort where 60%–90% of patients had sepsis and all had SIRS, elevated sTNFR-1 biomarker levels identified a subgroup at risk of severe AKI who may be also be responsive to targeted therapies. In fact, when applying latent class analysis to patients with AKI enrolled in the Vasopressin in Septic Shock Trial, the same group also found a subphenotype of sepsis-associated AKI marked by lower sTNFR-1 as well as other inflammatory factors that may particularly benefit from vasopressin (10). This potentially implies that therapies targeting TNF activation may also alter the incidence of severe AKI in SIRS.

Although at first glance, this article provides a simple predictive model, its incorporation of targeted biomarkers along the pathway of either endothelial dysfunction, apoptosis, and/or inflammation provides a glimpse into the future of precision medicine. In this future, one would be able to obtain a panel of point-of-care biomarkers to define unique patient profiles and rapidly determine risk factors, individualized response to different therapies, and ultimately alter patient outcomes.

Disclosures

K.D.L. has been a consultant for BioMerieux, Durect, Quark, Potrero Medical, and Theravance. She holds stock in Amgen. Y.D.K. has nothing to disclose.