Information Technology and AKI: Alerts, Alarms, Bells and Whistles

Summary

The goal of this review is to describe the rationale for alerting systems for acute kidney injury (AKI), the challenges associated with alert implementation, and the efficacy (or lack thereof) of AKI alerts to date.

Introduction

That Acute Kidney Injury (AKI) is a common, costly, and devastating condition is well-known to nephrologists.^1–5 Yet despite efforts by the nephrology community to bring AKI awareness and AKI research into the limelight⁶, non-nephrologist clinicians do not appear to have embraced the importance of the syndrome.

While lack of awareness of the definitions and outcomes of AKI may be to blame, it may equally be the result of a therapeutic nihilism that permeates AKI discussions, even among nephrologists. That there are "no treatments for AKI" is a common refrain parroted from the wards to medical conferences. While true in a narrow sense – there are no drug-based therapies that have been broadly shown to affect clinical outcomes across the spectrum of AKI – that there is "nothing to be done" when AKI develops is patently false.

Guidelines from the Kidney Disease: Improving Global Outcomes group, the UK National Institute for Health and Care Excellence, and the European Renal Association provide concrete recommendations for physicians faced with AKI (Table 1).^7,8 While these recommendations are often dismissed as merely supportive, the optimization of hemodynamics, avoidance of nephrotoxins, and attention to relevant diagnostic tests are broadly supported. While we lack clinical trial data suggesting that, for example, cessation of NSAIDs truly modifies the course of AKI, this is largely due to the perception that such a trial would be unethical, rather than due to lack of interest.

Table 1.

Guideline-based management of AKI.

KDIGO	NICE	UK Renal Association
Adjust drug dosing	Urinalysis	Adjust drug dosing
Fluids if volume depleted	Consider Ultrasound	Urinalysis
Pressors if Shock	Relieve Urological Obstruction	Contrast precautions
Protocolized Hemodynamic Management	Avoid diuretics	Avoid diuretics
Insulin Therapy in Critically Ill	Consider Renal Replacement Therapy
Nutritional Support	Consider Referral to Nephrology
Avoid diuretics
Avoid Nephrotoxins
Monitor Creatinine, Urine Output
Avoid subclavian catheters

KDIGO: Kidney Disease: Improving Global Outcomes. NICE: National Institute for Health and Care Excellence.

AKI Recognition

That something can and should be done for patients with AKI is clear. It is also clear that nothing can be done for these patients if AKI goes unnoticed. Several lines of research suggest that AKI is frequently "missed" by clinicians. First are studies that compare the documentation rate (usually assessed by billing codes) of AKI. These studies universally find low rates of diagnostic coding for AKI, often at levels lower than 50%.^9–11 A study by Grams et al examined 1,970 hospitalizations with AKI as defined by change in creatinine. Only 361 (18%) of those had evidence of AKI documentation based upon administrative coding.¹² While lack of diagnostic coding is not a perfect proxy for AKI recognition (clinicians may recognize and yet fail to document AKI), it is likely that a large number of patients with AKI may simply fly under the radar.

Further evidence supporting the hypothesis that some AKI events are "missed" can be found by examining the behavior of physicians in the setting of AKI with and without evidence of documentation. In a study of over 6000 patients with a doubling of creatinine, physicians who documented AKI in the medical record were more likely to discontinue nephrotoxic medications and engage in diagnostic testing compared to physicians who did not document AKI.¹³ Beyond increasing the rate of appropriate clinical behaviors in the setting of AKI, the study demonstrated that, after adjustment for AKI severity, individuals with AKI who also had formal documentation of AKI had a lower inpatient mortality than those individuals without formal documentation of AKI. Taken together, these lines of evidence suggest that a significant proportion of AKI goes unrecognized, that the opportunity for therapeutic actions is missed, and that these factors may contribute to poor outcomes.

Against this backdrop, the rationale for electronic alerting of AKI is clear. One hopes that alerts increase recognition, recognition changes management, and proper management improves outcomes. AKI alerts have proliferated in recent years^14–19, in large part due to the widespread adoption of the Electronic Health Record (EHR). In addition, given that AKI can be defined by the change in a single variable (creatinine) – AKI alerts are "low-hanging fruit" compared to more complicated alert systems (such as those for sepsis).^20–23 Far from a limitation, it seems imperative that nephrologists take the lead on rigorous evaluation of these alerts as they may not be entirely without harm.

AKI Alert Challenges

Despite the relative simplicity of detecting AKI via a change in creatinine, alerts can range from the very simple to quite complex. Urine output data has rarely been used in AKI alert studies²⁴, owing largely to the lack of rigorous standards of collection, though novel measurement devices for catheterized patients are being tested (clinicaltrials.gov NCT02195713).

Even if hospital systems adopt alerts based solely on changes in creatinine, several factors must be considered. Should alerts be generated for modest changes in creatinine (such as 0.3 mg/dl) which may include patients without "true" AKI? We previously demonstrated that, due to laboratory and eGFR-independent biological variation in creatinine, a significant portion of patients may be diagnosed with AKI due to random fluctuations in creatinine, rather than true injury.²⁵ This is particularly true for patients with higher baseline creatinine. For patients with a baseline creatinine of 3.0 mg/dl, we found that after four creatinine measurements, roughly 50% would be diagnosed with AKI even with a completely stable GFR.

Additionally, while the KDIGO-specified time frame for changes in creatinine ranges from 48 hours (for a 0.3 mg/dl change) to 7 days (for a 50% change), "rolling windows" may be difficult to program in the EHR, leading some alerts to adopt a "nadir-to-peak" approach which may increase the rate of alerting among those with longer lengths of stay.

Finally, how should pre-hospitalization creatinine concentrations be integrated? For those without a pre-hospitalization creatinine measurement, several studies have imputed a creatinine as if the patient had an eGFR of 75 ml/min/1.73m².²⁶ In the alert setting, such an imputation would dramatically increase the alerts among those with chronic kidney disease – a group already at risk for "false-positive" alerts as described above.

Decisions about alert triggers are, as is the case with all diagnostic tests, a tradeoff between sensitivity and specificity. This tradeoff is particularly problematic in the AKI alert paradigm, as more sensitive systems may be necessary to capture individuals with "subtle" AKI – those that might be missed clinically. At the same time, more sensitive systems create a greater number of false-positive alerts, which can be very frustrating for providers.

Once an alert is triggered, how should it be presented to the provider? Again, designers are faced with a tradeoff: this time between alert intrusiveness and potential effectiveness. In terms of pure effectiveness, "hard stop" alerts seem the natural choice. In the AKI framework, a hard stop alert would prevent a provider from, for example, ordering a contrast study while a patient has AKI. Providers would be forced to appeal through some mechanism to obtain the needed study. While an alert of this type would no doubt reduce the rate of contrast administration in AKI, it might engender revolt from providers who feel it too aggressively impedes their workflow. On the flip side, "soft" alerts, such as in-basket messages (that may or may not be read) hardly hinder workflow at all, but may not lead to substantive change in provider behavior. Striking the balance between these extremes is critical, and should be done with direct feedback from those who are likely to receive alerts. To date, there are no studies in AKI that have varied alert intrusiveness to elucidate the "sweet spot" between workflow impediment and patient benefit.

AKI Alert Effectiveness

Several studies have evaluated the effectiveness of alerts for AKI. Broadly speaking, the studies have shown that alerts change provider behaviors. A smaller subset of studies have suggested that alerts improve patient outcomes. A summary of studies, outcomes, and effects appears as Table 2.

Table 2.

Selected studies of Acute Kidney Injury Alerts and Outcomes

Study	N	Outcome	Effect (Alert vs. Control)
Rind 199427	562	Serious Renal Impairment	RR 0.45 (0.22 – 0.94) in intervention vs. control period
McCoy 201028	1,598	Change in nephrotoxic med	52.6 vs. 35.2 medication discontinuations per 100 AKI events (p<0.001)
Colpaert 201224	951	Resolution of AKI	65.9% vs. 63.1%, p=0.048.
Selby 201318	8,411	Survival	80.5% vs. 76.3%, p=0.006.
Wilson 201529	2,393	Change in creatinine, dialysis, and survival	No difference in composite endpoint (p=0.88)
Thomas 201530	408	Survival	No difference (log-rank p=0.38).

To date, only one randomized trial – performed by our group - has evaluated the efficacy of AKI alerts and failed to show a difference in the rates of change in creatinine, dialysis, or death.²⁹ While this may imply a true lack of efficacy, it should be noted that in this study alerts were sent only once per patient, required no acknowledgement, and did not carry any guiding information in terms of alert action.

The reasons for the varying effects of AKI alerts are multifactorial. Study design may play a role, of course, and heterogeneity of populations may also account for some differences. But it is likely that the mode and content of alerts may be the most salient factor. In the randomized trial of AKI alerts, no specific diagnostic or treatment recommendations were given. However, the Rind and Selby studies, which were both positive, tied alerts to actionable interventions (either a drug-dose recommendation or AKI "care bundle”). Several studies of clinical decision support systems outside of AKI have demonstrated that linking alerts to clear instructions and orders (especially with orders defaulted to the "preferred" state) can be highly effective at changing provider behaviors.³¹

Despite mixed effects in the adult population, pediatric AKI alerts appear to have a distinctly positive impact. At Cincinnati Children's hospital, Goldstein et al developed a real-time, prospective monitoring system for patients receiving nephrotoxic medications. The implementation of this system was associated with a 38% reduction in nephrotoxin exposure, and a 64% reduction in the rate of AKI.³² Among those who developed AKI, AKI duration was reduced after alerts were implemented. Again, it may be the presence of a clearly actionable factor – in this case nephrotoxic medication exposure – that allowed for favorable outcomes. It may also be due to the fact that pediatric AKI is more homogenous than adult AKI. In other words, the most likely cause of AKI in these children was exposure to nephrotoxic medication, as opposed to adults who may develop AKI through a myriad of disparate mechanisms.

AKI Alert Risks

Alerts are not a zero-risk intervention. While they lack the potential harms of drug- or device-based interventions, one must be cognizant of several mechanisms by which alerts can compromise patient care. Most widely studied is the concept of alert fatigue.³³ Alert fatigue is a condition whereby alerts, even important alerts, are ignored by providers due to the sheer number of alerts to which they are exposed. As the ease of creating alerts for a variety of conditions has increased, providers are exposed to ever more alerting. Even when, in isolation, alerts may be useful or desired by providers, they may be less so in aggregate.

Two strategies exist to combat alert fatigue. The first is rigorous alert evaluation. No alert should be implemented in the absence of a well-defined plan for monitoring its effects. Ideally, alerts should be implemented in a randomized fashion whenever feasible to assess their effects in an unbiased manner. Where not possible, rigorous before-after or before-during-after designs should be considered. Alerts that don't meaningfully affect patient outcomes should be abandoned.

The second method for avoiding alert fatigue is via alert targeting. Rather than targeting, for example, every patient with AKI, alerts can be targeted to just those who meet certain characteristics. These characteristics could be prognostic (eg – targeting patients at higher risk of dialysis or death) or predictive (eg – targeting patients at risk of recognition failure due to a slow rate of creatinine change). Predictive approaches could target subgroups that may specifically benefit from alerts (such as those receiving nephrotoxic medications). More complex predictive models, integrating multiple covariates, can be created through "uplift" modeling, a technique pioneered in the marketing literature to target interventions (such as coupons) to those for whom the intervention is most likely to change behavior.³⁴ Finally, alerts can be targeted to specific actions – firing only when a potential medical "error" is in the process of being made. Examples are further enumerated in Table 3.

Table 3.

Targeting strategies for Acute Kidney Injury Alerts

Alert Targeting Strategy	Explanation	Example
Prognostic	Target alerts to those at high risk of adverse outcomes such as dialysis or death.	Alerts are sent only for ICU patients, patients with severe electrolyte abnormalities, or very high serum creatinine concentrations.
Predictive (Subgroups)	Target alerts to those in whom alerts may have unique benefit	Alerts are sent for patients receiving nephrotoxic drugs.
Predictive (Uplift)	Target alerts to those in whom alerts may have unique benefit	Alerts targeted to those in whom a combination of factors suggests a differential benefit of alerting versus usual care.
Point of Potential Error	Target alerts to patients with AKI in whom an order that may worsen kidney function is about to be placed	Alert fires when a contrast study is ordered

Alerts may also endanger patients by leading to unnecessary interventions. Providers may (falsely) assume that alerts require some remedial action. In the randomized trial referenced above²⁹, among surgical ward patients the rate of renal consult was 2.5 times higher in the alert compared to the usual care arm. One can also imagine that alerts may prolong length of stay, as clinicians wait to see if creatinine improves, though this outcome has not been reported in any studies thus far.

In sum, electronic alerts for AKI are an attractive approach to improve and standardize care for this high-risk patient population. Despite their relative ease-of-implementation, researchers should be encouraged to evaluate alerts' clinical efficacy with robust and unbiased methods, ideally clinical trials. Without proven efficacy, AKI alerts (and all alerts that may impede usual workflow) should not be implemented.

Summary.

• -Electronic alerts for AKI and other conditions are relatively feasible to implement, but have unclear evidence of clinical benefit
• -Some possibility of harm associated with alerting exists -Clinical trials are the best mechanism to evaluate alerts, and alerts found not to improve patient outcomes should be abandoned
• -Alert targeting may help to both improve alert efficacy and decrease alert fatigue