Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Nov 16.
Published in final edited form as: Lancet. 2015 Feb 26;385(9981):1966–1974. doi: 10.1016/S0140-6736(15)60266-5

Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomised controlled trial

F Perry Wilson 1, Michael Shashaty 1, Jeffrey Testani 1, Iram Aqeel 1, Yuliya Borovskiy 1, Susan S Ellenberg 1, Harold I Feldman 1, Hilda Fernandez 1, Yevgeniy Gitelman 1, Jennie Lin 1, Dan Negoianu 1, Chirag R Parikh 1, Peter P Reese 1, Richard Urbani 1, Barry Fuchs 1
PMCID: PMC4475457  NIHMSID: NIHMS670794  PMID: 25726515

Summary

Background

Acute kidney injury often goes unrecognised in its early stages when effective treatment options might be available. We aimed to determine whether an automated electronic alert for acute kidney injury would reduce the severity of such injury and improve clinical outcomes in patients in hospital.

Methods

In this investigator-masked, parallel-group, randomised controlled trial, patients were recruited from the hospital of the University of Pennsylvania in Philadelphia, PA, USA. Eligible participants were adults aged 18 years or older who were in hospital with stage 1 or greater acute kidney injury as defined by Kidney Disease Improving Global Outcomes creatinine-based criteria. Exclusion criteria were initial hospital creatinine 4·0 mg/dL (to convert to μmol/L, multiply by 88·4) or greater, fewer than two creatinine values measured, inability to determine the covering provider, admission to hospice or the observation unit, previous randomisation, or end-stage renal disease. Patients were randomly assigned (1:1) via a computer-generated sequence to receive an acute kidney injury alert (a text-based alert sent to the covering provider and unit pharmacist indicating new acute kidney injury) or usual care, stratified by medical versus surgical admission and intensive care unit versus non-intensive care unit location in blocks of 4–8 participants. The primary outcome was a composite of relative maximum change in creatinine, dialysis, and death at 7 days after randomisation. All analyses were by intention to treat. This study is registered with ClinicalTrials.gov, number NCT01862419.

Findings

Between Sept 17, 2013, and April 14, 2014, 23 664 patients were screened. 1201 eligible participants were assigned to the acute kidney injury alert group and 1192 were assigned to the usual care group. Composite relative maximum change in creatinine, dialysis, and death at 7 days did not differ between the alert group and the usual care group (p=0·88), or within any of the four randomisation strata (all p>0·05). At 7 days after randomisation, median maximum relative change in creatinine concentrations was 0·0% (IQR 0·0–18·4) in the alert group and 0·6% (0·0–17·5) in the usual care group (p=0·81); 87 (7·2%) patients in the alert group and 70 (5·9%) patients in usual care group had received dialysis (odds ratio 1·25 [95% CI 0·90–1·74]; p=0·18); and 71 (5·9%) patients in the alert group and 61 (5·1%) patients in the usual care group had died (1·16 [0·81–1·68]; p=0·40).

Interpretation

An electronic alert system for acute kidney injury did not improve clinical outcomes among patients in hospital.

Funding

Penn Center for Healthcare Improvement and Patient Safety.

Introduction

Acute kidney injury affects up to 10% of patients admitted to hospital, and carries with it a great increased risk of mortality.1–8 In the past ten years, consensus definitions for acute kidney injury have become increasingly sensitive, with a goal of detecting earlier, milder disease.9–11 The most recent guideline, from the Kidney Disease Improving Global Outcomes (KDIGO) working group, defines acute kidney injury as an absolute increase of serum creatinine of 0·3 mg/dL (to convert to μmol/L, multiply by 88·4) within 48 h or a relative increase of 50% in 7 days.11 Some studies have suggested that even smaller changes in serum creatinine, as little as 0·1 mg/dL, are associated with substantial increases in the risk of death, dialysis, and other morbidities.5–7,12–16

Although individual trials of therapeutic interventions have been negative,17–20 consensus statements by expert panels of nephrologists have repeatedly recommended tailored early treatment including drug dose adjustment, nephrotoxin avoidance, and attention to fluid balance.10,11 Although the benefits of such interventions have not been rigorously tested in clinical trial settings, the recommendations are based on sound scientific principles and have the potential to improve clinical outcomes. Early implementation, however, is restricted by the fact that providers frequently fail to notice small changes, and sometimes even substantial changes, in serum creatinine.14,21–23 For example, we have previously reported that more than 25% of patients whose creatinine doubled during a stay in hospital had no chart documentation of acute kidney injury, and failure to document acute kidney injury was independently associated with increased mortality.24

Automated alerts have recently emerged as a major instrument to influence clinician behaviour. In the hospital setting, randomised trials have shown the efficacy of alerts to reduce drug interactions,25,26 increase the rate of venous thromboembolism prophylaxis,27 and improve the rates of various other preventive measures to positively influence outcomes.28 A recent synthesis of the results of several studies examining clinical decision support systems noted that 57% affect practitioner behaviour, whereas only 30% have shown a positive effect on patient outcomes.29 Because acute kidney injury is a complex syndrome that might benefit from personalised, early intervention, automated alerts have the potential to improve the clinical course of patients affected by this condition. We aimed to determine whether an automated electronic alert for acute kidney injury would reduce the severity of acute kidney injury and improve clinical outcomes in patients in hosptial.

Methods

Study design and participants

In this single-blind, parallel-group, randomised controlled trial, patients were recruited from the hospital of the University of Pennsylvania in Philadelphia, PA, USA, a tertiary care hospital serving the Philadelphia metropolitan area that has a wide referral base. Eligible participants were adults aged 18 years or older who were in hospital with acute kidney injury as defined by the KDIGO creatinine-based criteria.11 Using the electronic medical record system, we devised a computerised algorithm to track changes in patients’ serum creatinine values in real time for the duration of their stay in hospital. To be diagnosed with acute kidney injury, the current serum creatinine value must have been at least 0·3 mg/dL greater than the lowest value that occurred in the previous 48 h or 50% greater than the lowest value that occurred in the previous 7 days. Outpatient creatinine values that were obtained within the relevant time interval were included in the algorithm. Urine output criteria were not used to define acute kidney injury because our hospital does not mandate hourly urine output measurements on all patients, and because of the likelihood of inaccurate measurement in the substantial number of patients without urinary catheters. Exclusion criteria were initial hospital creatinine 4·0 mg/dL or greater, fewer than two creatinine values measured, inability to determine the covering provider, admission to hospice or the observation unit, previous randomisation, or admission International Classification of Diseases-9 code 58·6 (end-stage renal disease). No alerts were sent to patients who were not enrolled in this study.

Full trial methodology has been previously published.30 The protocol was reviewed and approved by the Institutional Review Board of the University of Pennsylvania, which granted a waiver to the informed consent process.

Randomisation and masking

Patients were randomly assigned (1:1) to either the acute kidney injury alert group or usual care group. A Data Monitoring Committee performed a safety analysis at 50% recruitment. No protocol changes were made during the course of the trial.

Included patients were assigned to one of four mutually exclusive strata based on whether they had been admitted to a medical or surgical service and whether or not they were in an intensive care unit (ICU) or not at the time they met the criteria for acute kidney injury. These strata were chosen based on our a-priori hypothesis that service type and location might modify the effect of the alert. Each stratum was assigned a computer generated, permuted block randomisation list with block sizes ranging from four to eight individuals.31 All randomisation was automated by the alert system, which sent alerts autonomously, thus maintaining allocation concealment. Whereas clinicians were aware of the allocation group (in the case of patients randomised to alerts), the study personnel and outcome assessors were kept masked to the allocation.

Procedures

In the alert group, the covering provider (one per patient; typically an intern, resident, or nurse practitioner) and unit pharmacist received a text page on his or her hospital-provided cell phone informing them of the presence of acute kidney injury (alert). The usual care group did not receive any alerts. Acute kidney injury alerts were sent via text-page by the hospital’s existing text-paging system, which is routinely used for clinical communication between providers. This system operates through two protocols, depending on the device being used by the recipient of the page. Wireless communication transfer protocol was used for most alerts (56%) and has the advantage of automatically confirming receipt of the page. The remainder of alerts (44%) were sent via simple mail transfer protocol. The alert text was standardised:

“[Initials], [Room Number], has been identified as having acute kidney injury (AKI) based upon the latest creatinine value. Please take appropriate diagnostic and therapeutic measures. THIS ALERT DOES NOT FIRE FOR ALL PATIENTS WITH AKI. For more information, please visit [internal study website].”

The study website, mentioned in the alert, contained study information and a link to the KDIGO acute kidney injury practice guidelines.11 Clinicians were informed at pre-trial department meetings, and with each page, that not all patients with acute kidney injury would generate an alert. The covering provider was determined electronically from the patient’s electronic medical record. The responsible pharmacists were identified based on the patient’s location, because the hospital at the University of Pennsylvania has a unit-based pharmacy system. To avoid so-called alert fatigue,32 the alert was sent to the covering provider and unit pharmacist once per patient. Alerts were batched to be sent hourly, ensuring no more than a one-hour delay between an acute kidney injury-defining creatinine result and provider notification. Pre-trial quality assurance activities, including random follow-up phone calls (n=25), confirmed that all pages were received by the correct providers within 1 h of the patient meeting acute kidney injury criteria.

Outcomes

Our primary outcome was a ranking of clinical outcomes, maximum relative change in creatinine, dialysis, and death, within 7 days of randomisation. Participants were ranked according to the maximum relative change in creatinine (difference from the randomisation creatinine, low to high), except for those who received dialysis or died during this interval. The highest rank was assigned to those who died, with the second highest rank assigned to those who survived but received dialysis. The rationale for this outcome was to ensure that patients who started dialysis or died while the serum creatinine concentration was still increasing were not inappropriately classified as having a better outcome. Patients discharged alive and without having received dialysis before 7 days were assumed to be alive and free of dialysis at 7 days.

We pre-specified several secondary outcomes including rates of dialysis, death, progression to higher stages of acute kidney injury, chart documentation of acute kidney injury, and several process measures (including rates of renal consult, contrast, and other nephrotoxin administration). A full list of the secondary outcomes evaluated in this trial was previously published.30

Statistical analysis

We used a non-parametric rank sum test for our primary analysis. We used the method described by Zhao and colleagues,33 as implemented in the programme PASS 12 (PASS 12, NCSS, Kaysville UT, USA), to calculate sample size by dividing possible outcomes into six categories and estimating the expected proportion falling into each category using distributions of outcomes observed retrospectively. The PASS programme provides estimates of power or sample size by simulation. This pre-alert phase of the trial, was conducted between June 1, 2013, and Sept 1, 2013, and examined 1080 patients. Compared with the active phase, the pre-alert phase had similar overall rates of dialysis but lower rates of death, and lower values of the composite outcome (appendix p 1).

Estimates were obtained over 5000 simulations. We estimated that with 1200 patients per group, we would have at least 90% power to detect a downward shift in the outcome rankings in the intervention group that we judged would represent a clinically significant improvement. This shift would include, for example, a reduction in number of deaths from 10% to 9%, in dialysis from 5% to 4%, and an increase in the proportion of patients with no further creatinine increase beyond the trigger creatinine from 50% to 56%. Power to detect other degrees of difference in the two groups is shown in appendix p 4.

Demographic, laboratory, and procedural data were obtained from the electronic health records as previously described.30 The presence and timing of renal consultation was determined from manual chart review by trained study personnel.

For the primary endpoint and for comparison of all continuous outcomes, the treatment groups were compared by Van Elteren rank testing, which allows comparison of the distribution of study outcomes across randomised groups and accounts for stratification.34 Differences in categorical outcomes were evaluated with the Mantel-Haenszel test of the common odds ratio (OR). All analyses were by intention to treat, and a two-sided p<0·05 was considered statistically significant. Modelling of the effectiveness of the alert across pre-specified subgroups and with time was done via logistic regression where treatment assignment was the outcome and with interaction terms included as modifiers of the main effect. Time interactions were modelled with time as a continuous covariate, but presented dichotomously for clarity. Where interactions were significant at p<0·05, stratified results are reported with the above non-parametric tests. Time to fluid bolus was assessed with Cox proportional hazards tests, censoring at death, discharge, or within 7 days of randomisation and including randomisation strata as covariates. One masked safety analysis was done after 50% recruitment, which examined only the outcome of inpatient mortality between the two groups. We pre-specified p≤0·003 as a threshold to stop the study.30 This single interim analysis (which showed no difference in mortality between the groups, p=0·79) could trivially bias towards a false-positive result. We did not adjust the reported p values in the primary analysis.

All analyses were done with Stata version 13.1 (College Station, TX, USA).

Role of the funding source

The funder of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. FPW had full access to all the data in the study, takes responsibility for the integrity of the data and the accuracy of the data analysis, and had final responsibility for the decision to submit for publication.

Results

Between Sept 17, 2013, and April 14, 2014, 23 664 patients were screened for the development of acute kidney injury.11 After application of our exclusion criteria, 2393 patients were randomised; of whom 1201 were assigned to the acute kidney injury alert group and 1192 were assigned to the usual care group (figure 1). There were 1044 (44%) patients in the medical ward stratum, 278 (12%) in the medical ICU stratum, 627 (26%) in the surgical ward stratum, and 444 (19%) in the surgical ICU stratum. 134 patients (6%) were enrolled on the basis of an inpatient serum creatinine value that was indicative of acute kidney injury when compared with an outpatient value that occurred within the relevant timeframe.

Figure 1. Trial profile.

Figure 1

Open in a new tab

ESRD ICD=end-stage renal disease International Classification of Diseases.

Baseline characteristics were similar across both study groups (table 1). The population was made up of 1325 (56%) men and 647 (27%) were black. The mean (SD) age was 60 years (16) years. 722 (30%) patients were in an ICU at the time of randomisation, and 1013 (42%) had been admitted to a surgical service.

Table 1.

Baseline characteristics of study participants

Alert (n=1201) Usual care (n=1192)
Age, years 60 (17) 61 (16)
Men 670 (56%) 655 (55%)
Black 324 (27%) 323 (27%)
Hispanic 33 (3%) 38 (3%)
Admitted to a surgical service 516 (43%) 497 (42%)
In intensive care unit at randomisation 365 (30%) 357 (30%)
Cerebrovascular disease 142 (12%) 126 (11%)
Congestive heart failure 386 (32%) 373 (31%)
Diabetes mellitus 352 (29%) 370 (31%)
Liver disease 164 (14%) 181 (15%)
Chronic kidney disease 323 (27%) 314 (26%)
Malignancy 335 (28%) 309 (26%)
Metastatic disease 103 (9%) 111 (9%)
Baseline creatinine, mg/dL 0·91 (0·60–1·37) 0·90 (0·56–1·36)
Enrolment creatinine, mg/dL 1·38 (1·01–1·90) 1·37 (0·95–1·88)
Enrolment creatinine ≥0·3 mg/dL above baseline 1012 (84%) 963 (81%)
Enrolment creatinine ≥50% above baseline 612 (51%) 627 (52%)
Time between admission and acute kidney injury, days 2·41 (1·18–5·71) 2·65 (1·15–5·75)
Open in a new tab

Data are mean (SD), n (%), or median (IQR). Comorbidities were defined by administrative International Classification of Diseases-9 codes.35 To convert creatinine to μmol/L, multiply by 88·4.

Overall, the primary outcome of the composite relative maximum change in creatinine, dialysis, and death at 7 days was no different between the groups (p=0·88; table 2). In both groups, the median relative increase in creatinine subsequent to the time of randomisation was close to zero, indicating that many patients in both groups never achieved a creatinine higher than the one initially meeting the criteria for acute kidney injury. At 7 days after randomisation, median maximum relative change in creatinine concentrations was 0·0% (IQR 0·0–18·4) in the alert group and 0·6% (0·0–17·5) in the usual care group (p=0·81). 87 (7·2%) patients in the alert group and 70 (5·9%) patients in usual care group had received dialysis (OR 1·25 [95% CI 0·90–1·74]; p=0·18); and 71 (5·9%) patients in the alert group and 61 (5·1%) patients in the usual care group had died (1·16 [0·81–1·68]; p=0·40). No differences between the treatment groups emerged after longer follow-up to 14 days and 30 days (table 2).

Table 2.

Primary outcomes at various timepoints

Alert (n=1201) Usual care (n=1192) p value Composite p value
7 days after randomisation
Increase in creatinine from randomisation, % 0·0% (0·0–18·4) 0·6% (0·0–17·5) 0·81 0·88
Dialysis 87 (7·2%) 70 (5·9%) 0·18
Death 71 (5·9%) 61 (5·1%) 0·40
14 days after randomisation
Increase in creatinine from randomisation, % 0·9% (0·0–20·6) 1·4% (0·0–20·2) 0·77 0·83
Dialysis 98 (8·2%) 79 (6·6%) 0·16
Death 93 (7·7%) 85 (7·1%) 0·58
30 days after randomisation
Increase in creatinine from randomisation, % 1·3% (0·0–21·9) 2·1% (0·0–22·1) 0·65 0·89
Dialysis 104 (8·7%) 88 (7·4%) 0·26
Death 106 (8·8%) 107 (9·0%) 0·85
Duration of hospital admission
Increase in creatinine from randomisation, % 1·3% (0·0–22·2) 2·1% (0·0–22·6) 0·64 0·96
Dialysis 105 (8·7%) 90 (7·6%) 0·30
Death 118 (9·8%) 112 (9·4%) 0·75
Open in a new tab

Data are median % increase (IQR) or n (%). The pre-specified primary outcome was a composite of maximum relative change in creatinine, dialysis, and death at 7 days. Patients discharged alive were presumed alive at all points of follow-up. Change in creatinine data are in the form of % increase (IQR) from randomisation to the maximum achieved creatinine in the relevant timescale. Composite p values are calculated by the Van Elteren test.

At 7 days, the combined rate of death and dialysis was 11·7% in the alert group and 9·9% in the usual care group (OR 1·22 [95% CI 0·93–1·60]; p=0·15). This rate was 13·7% versus 11·7% (OR 1·21 [95% CI 0·94–1·56]; p=0·14) at 14 days and 14·6% versus 13·7% (1·08 [0·85–1·37]; p=0·55) at 30 days. Maximum KDIGO acute kidney injury stages within 7 days of randomisation did not differ between the groups (p=0·83; appendix p 2).

At 7 days after randomisation, 222 (9·3%) of trial patients had received a nephrology consultation, 353 (14·8%) had received intravenous contrast, 147 (6·1%) had received an aminoglycoside, 155 (6·5%) had received an non-steroidal anti-inflammatory drug, and 512 (21·4%) had received an angiotensin-converting enzyme inhibitor or an angiotensin receptor blocker. No significant differences were noted between the alert group and the usual care group in any of these measures (table 3).

Table 3.

Secondary process outcomes

Alert (n=1201) Usual care (n=1192) p value
Renal consult within 7 days 120 (10%) 102 (9%) 0·23
Renal consult within 14 days 129 (11%) 112 (9%) 0·28
Renal consult inpatient 139 (12%) 125 (11%) 0·41
Time to consult 1·61 (0·36–4·07) 1·78 (0·74–4·41) 0·33
Chart documentation of AKI 545 (46%) 531 (45%) 0·68
Contrast within 7 days 179 (15%) 174 (15%) 0·84
Contrast within 14 days 219 (18%) 223 (19%) 0·92
Contrast during AKI 177 (15%) 176 (15%) 0·97
Fluid bolus within 7 days 426 (36%) 422 (35%) 0·75
Time to fluid bolus, h 9·5 (3·1–39·7) 11·1 (4·2–35·8) 0·38
Aminoglycoside within 7 days 64 (5%) 83 (7%) 0·09
Aminoglycoside within 14 days 78 (7%) 99 (8%) 0·08
Aminoglycoside during AKI 82 (7%) 91 (8%) 0·42
NSAID within 7 days 78 (7%) 77 (7%) 0·94
NSAID within 14 days 86 (7%) 92 (8%) 0·62
NSAID during AKI 74 (6%) 81 (7%) 0·55
ACE or ARB within 7 days 272 (23%) 240 (20%) 0·13
ACE or ARB within 14 days 287 (24%) 262 (22%) 0·27
ACE or ARB during AKI 238 (20%) 226 (19%) 0·60
Urinalysis within 48 h 280 (23%) 284 (24%) 0·74
Renal ultrasound within 48 h 92 (8%) 82 (7%) 0·47
Creatinine tests within 48 h 2 (2–3) 2 (2–3) 0·05*
Creatinine tests within 7 days 6 (3–9) 6 (3–9) 0·23
Length of stay, days 9·7 (5·6–16·1) 10·0 (6·0–17·8) 0·11
Time from randomisation to discharge, days 5·4 (2·5–11·4) 5·9 (2·5–12·3) 0·32
Open in a new tab

Data are n (%) or median (IQR). Administration during AKI connotes that the drug was given before the creatinine returned to within 10% of baseline. Chart documentation of AKI based on discharge International Classification of Diseases-9 codes. All times are from randomisation. Although not demonstrable from the distribution reported, creatinine tests were done less often in the alert group than in the usual care group. AKI=acute kidney injury. NSAID=non-steroidal anti-inflammatory drugs. ACE=angiotensin-converting enzyme inhibitor. ARB=angiotensin receptor blocker.

*

p=0·0501.

Providers who saw an alert might order more frequent subsequent creatinine testing, leading to ascertainment bias. There was weak evidence of more frequent creatinine testing in the usual care group compared with the alert group within 48 h of randomisation (p=0·05), but not within 7 days (table 3).

We hypothesised that the effectiveness of the alert might be strongly affected by the acuity of illness of the patient (ICU vs non-ICU) and the medical status versus surgical status of the patient. Compared with medical ward patients, the rank-based outcome was no different in patients on the medical ICU, surgical ward, or surgical ICU (pinteraction>0·2 for all comparisons).

Selected clinical outcomes were compared in the pre-specified subgroups at various timepoints (figure 2). Although the rate of death did not differ between any subgroups, we did find that the alert was significantly associated with an increased rate of renal consult (pinteraction=0·001) and dialysis (pinteraction=0·07) among surgical ward patients. Of 627 patients on the surgical ward, 36 (11·5%) in the alert group received a renal consult compared with 17 (5·4%) in the usual care group (OR 2·29 [95% CI 1·22–4·44]; p=0·006), and 19 (6·1%) in the alert group received dialysis compared with eight (2·5%) in the usual care group (2·49 [1·02–6·67]; p=0·03) during their stay in hospital.

Figure 2. Odds ratios of outcomes in acute kidney injury alert group compared with usual care group across the four study strata.

Figure 2

Open in a new tab

Favours alert means more events in the usual care group and favours usual care means more events in the alrt group.

Within the surgical ward stratum of the study, the alert did not seem to increase the rate of fluid boluses (32·3% in the alert group vs 28·9% in the usual care group; OR 1·18 [95% CI 0·84–1·68]; p=0·34) or the time to fluid bolus (p=0·92; table 3). In post-hoc analysis, we did not detect a difference in pre-dialysis serum concentrations of creatinine, potassium, bicarbonate, or blood urea nitrogen between the groups in any strata (appendix p 3).

Acute kidney injury might go unnoticed among patients with decreased baseline concentrations of serum creatinine. Among those with a baseline creatinine less than 1·0 mg/dL (n=1340), we noted no difference in primary outcome between the alert group and usual care group at 7 days (n=1340, p=0·60). Similarly, we did not find any notable effects of the alert when we examined the primary outcome among women (n=1051, p=0·96), Black patients (n=647, p=0·96), or patients aged 65 years or older (n=1015; p=0·41).

We hypothesised that the alert might become less effective as the study progressed, because receipt of alerts might have trained providers to have heightened awareness of acute kidney injury for patients in the usual care group as well. Therefore, we examined changes in the effect of the alert on the primary outcome as the study progressed. The effectiveness of the alert with regards to the primary endpoint did not change over the course of the study (pinteraction=0·20). However, as the study progressed, we noted a diminution of the effect of the alert on maximum relative change in creatinine (pinteraction=0·03) and a decrease in the effect of the alert increasing dialysis rates (pinteraction=0·02). For example, in the first 100 days of the study, 46 (7·6%) of 603 participants in the alert group received dialysis within 7 days versus 25 (4·2%) of 594 in the usual care group (OR 1·89 [95% CI 1·14–3·14]; p=0·01). In the subsequent 100 days, the rate of dialysis was 6·9% (41/598) in the alert group versus 7·5% (45/598) in the usual care group (OR 0·90 [95% CI 0·58–1·41]; p=0·66). Additionally, we did not record any effect of date of randomisation on the difference in death rates (pinteraction=0·82).

Discussion

In this study, we assessed the effectiveness of an automated, electronic alert system for acute kidney injury that used existing, ubiquitously-used paging technology within our tertiary care hospital. Although no other randomised trials have been reported, several studies have examined electronic alerting for use in various populations. A study in a large UK hospital showed the feasibility of sending electronic alerts to providers whose patients had acute kidney injury, but the absence of a control group restricts the ability to determine the effect of such an intervention.36 A study by Colpaert and colleagues37 used a before and after design to show that acute kidney injury alerts in patients in ICUs could improve some outcomes. Although not randomised, the study suggested that alerts for acute kidney injury (90% of which were diagnosed based on low urine output) might lead to fluid administration, which increased urine output at 24-h post-alert. This study is an excellent proof-of-concept, but the absence of long-term data or hard clinical endpoints is a limitation.

Despite the large number of patients with KDIGO creatinine-defined acute kidney injury in our study, we detected no difference in the primary composite endpoint of relative maximum increase in creatinine, dialysis, or death. Several possible explanations for this finding exist. First and foremost, the alert might not be a useful clinical intervention. This could be due to the possibility that the providers were already aware of acute kidney injury. Alternatively, if providers were not aware of acute kidney injury, it is possible that early identification of the disease does not lead to meaningful changes in management. Perhaps any changes in management do not translate to improved clinical outcomes. We chose a universally acknowledged, but very sensitive definition of acute kidney injury, which might have increased the rate of false positive cases in our cohort, attenuating the observed effects. Additionally, alert effectiveness is likely to be lower where the quality of usual care is better; different results might be seen in other settings. Finally, enrolment of all patients with acute kidney injury led to a substantially heterogeneous population. So it is conceivable that the interventions that might occur as a result of the alert (for example, fluid administration) might benefit patients with certain causes of acute kidney injury and harm others.

No trials have shown that a specific intervention improves clinical outcomes after acute kidney injury has developed. However, consensus guidelines provide several recommendations regarding the appropriate management of patients with acute kidney injury. These recommendations include addressing medication dosing, avoiding nephrotoxic exposures, and careful attention to fluid and electrolyte balance.11 Our analysis of the response to this alert suggests that there was no meaningful effect on these process measures, which might relate to how the alert was delivered. Alert effectiveness might be attenuated if not timed to the provider experience within the electronic health record. An alternative explanation is the minimally invasive nature of the alert—a one-time-only page. Studies to investigate more direct provider engagement, while more time-consuming and costly, could provide more benefit.38,39

The findings among the surgical ward strata of patients were notable. Although we did not detect a significant difference in overall mortality between the two groups in this stratum, we noted a trend towards higher inpatient mortality in the alert group than in the usual care group (4·5% vs 2·2%; p=0·12). Patients in the surgical ward in the alert group were more likely to receive a renal consultation and more likely to receive dialysis than patients in the usual care group, raising the possibility that consultant-recommended dialysis could have worsened clinical outcomes. Unfortunately, the few events in this group restrict our ability to make a robust assessment of mediating factors. We hypothesised that an increase in use of intravenous fluid might mediate the observed harmful effect, because several studies have shown that increased positive fluid balance during an admission to hospital is associated with adverse outcomes,40,41 but the use of fluid bolus did not differ between groups in this strata. Notably, these findings occurred in the setting of multiple subgroup analyses, and could therefore be the result of chance and thus should be considered hypothesis-generating only. Further research in this area, with careful assessment of provider response to alert, is necessary.

We noted that rates of dialysis in the alert group were increased relative to the usual care group in the first 100 days of the study, and these effects diminished with time. This trend could be consistent with so-called alert fatigue, whereby providers were less likely to heed an alert the longer they were exposed to it.42–44 Additionally, the early part of the study occurred in the first half of the academic year, when trainees new to the hospital could have been more amenable to acting on the paging message with changes in therapeutic decisions. Finally, this could be evidence of contamination of the control group, in which providers, educated by receiving alerts, are more likely to take action for control patients. Together with the findings in the surgical subgroup, these data suggest that this type of alert might increase certain interventions without significantly improving care.

These findings should inform the adoption of electronic alerting in the future. Specifically, future trials should examine novel diagnostic algorithms for acute kidney injury that might improve detection of individuals likely to progress to clinically meaningful endpoints. Also, studies that provide more direction regarding interventions and process measures could provide valuable intermediate data regarding alert effectiveness before hard outcomes are explored. Wherever feasible, such alerts should be assessed in a randomised setting to determine if unforeseen adverse consequences accompany the intervention. In the absence of demonstrable benefit, alert systems should not be imposed on a health-care system under the assumption that they are entirely benign.

The results of this study should be interpreted in view of several limitations. First, the study was done at a single academic medical centre where several layers of providers give care to patients; alert effectiveness might differ in other care settings. Second, we did not use urine output criteria for acute kidney injury, based on the fact that urine output was not reliably captured in real time in our electronic health records; however, several studies suggest that urine output criteria might not be as robustly associated with outcome as creatinine criteria for acute kidney injury.45,46

In conclusion, this randomised, controlled study did not show a meaningful benefit of an electronic alert system for acute kidney injury in patients in hospital. Signals of more intense health care use, such as a possible increased rate of dialysis in the surgical ward subgroup and during the first half of the trial, should temper enthusiasm for the adoption of electronic alerts for acute kidney injury in the absence of careful study of both their efficacy and potential adverse effects.

Research in context.

Evidence before this study

We searched PubMed, Embase, and ClinicalTrials.gov for articles published between Jan 1, 1900, and Jan 7, 2014, examining the effect of alerts for acute kidney injury. We used high-performance information search filters to identify studies of acute kidney injury. We further filtered these studies using the search terms “alert”, “monitoring”, and “point of care” and restricted results to human studies. We identified several observational studies documenting the institution of electronic alerts for acute kidney injury; nearly all recommended randomised trials be performed. One before-after study, noted more rapid improvement in RIFLE class during the alert period than in the pre-alert period.

Added value of this study

This is the first randomised trial to assess the effectiveness of automated alerting for acute kidney injury. In a large and diverse patient population, such alerts showed no obvious clinical benefit. It is of concern that one prespecified subgroup showed increased use of health-care resources (nephrology consult and dialysis) in the alert group of the study.

Implications of all the available evidence

Alerts for acute kidney injury might improve proxy measures of kidney function, but might not improve clinical outcomes. Health-care agencies should be aware of the potential for increased resource use in the absence of clinical benefit where alerts are instituted.

Acknowledgments

This work was funded in part by A K23 Mentored Career Development Award from the NIH (K23DK097201) and a Penn Center for Healthcare Improvement and Patient Safety Pilot Grant to FPW, a Mentored Career Development Grant from the NIH (K23HL114868), a Loan-repayment grant from the NIH (L30HL115790) to JT, a Fellowship Training Grant from the NIH (T32DK007006) to JL, and a midcareer investigator award from the NIH (K24DK090203) to CRP. We thank the Penn Data Store for their assistance in assembling some of the information used in this study.

Footnotes

Declaration of interests

FPW has a patent “A method and system for detecting and categorising disease” pending. MS reports a K23 Mentored Career Development Award from the National Institute of Health during the conduct of the study. HIF reports honorarium from Kyowa Kirin and GlaxoSmithKline outside of the submitted work. Other authors declare no competing interests.

Contributors

FPD did the literature search, designed the study, collected, analysed, and interpreted the data, drew the figures, and wrote the report. MS designed the study, interpreted the data, and wrote the report. JT interpreted the data and wrote the report. IA designed the study, collected and interpreted the data, and wrote the report. YB collected the data. SSE designed the study, interpreted the data, and wrote the report. HIF designed the study, interpreted the data, and wrote the report. HF designed the study, interpreted the data, and wrote the report. YG designed the study, collected the data, and wrote the report. JL designed the study, interpreted the data, and wrote the report. DN interpreted the data and and wrote the report. CRP interpreted the data and and wrote the report. PPR designed the study, interpreted the data, and wrote the report. RU provided informatics support and development and collected the data. BF designed the study, interpreted the data, and wrote the report.

References

  • 1.Andrikos E, Tseke P, Balafa O, et al. Epidemiology of acute renal failure in ICUs: a multi-center prospective study. Blood Purif. 2009;28:239–44. doi: 10.1159/000231986. [DOI] [PubMed] [Google Scholar]
  • 2.Bagshaw SM, George C, Bellomo R. Early acute kidney injury and sepsis: a multicentre evaluation. Crit Care. 2008;12:R47. doi: 10.1186/cc6863. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Bagshaw SM, George C, Dinu I, Bellomo R. A multi-centre evaluation of the RIFLE criteria for early acute kidney injury in critically ill patients. Nephrol Dial Transplant. 2008;23:1203–10. doi: 10.1093/ndt/gfm744. [DOI] [PubMed] [Google Scholar]
  • 4.Bagshaw SM, Uchino S, Bellomo R, et al. Septic acute kidney injury in critically ill patients: clinical characteristics and outcomes. Clin J Am Soc Nephrol. 2007;2:431–39. doi: 10.2215/CJN.03681106. [DOI] [PubMed] [Google Scholar]
  • 5.Joannidis M, Metnitz B, Bauer P, et al. Acute kidney injury in critically ill patients classified by AKIN versus RIFLE using the SAPS 3 database. Intensive Care Med. 2009;35:1692–702. doi: 10.1007/s00134-009-1530-4. [DOI] [PubMed] [Google Scholar]
  • 6.Kolli H, Rajagopalam S, Patel N, et al. Mild acute kidney injury is associated with increased mortality after cardiac surgery in patients with eGFR <60 mL/min/1·73 m(2) Ren Fail. 2010;32:1066–72. doi: 10.3109/0886022X.2010.510616. [DOI] [PubMed] [Google Scholar]
  • 7.Chertow GM, Burdick E, Honour M, Bonventre JV, Bates DW. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients. J Am Soc Nephrol. 2005;16:3365–70. doi: 10.1681/ASN.2004090740. [DOI] [PubMed] [Google Scholar]
  • 8.Linder A, Fjell C, Levin A, Walley KR, Russell JA, Boyd JH. Small acute increases in serum creatinine are associated with decreased long-term survival in the critically ill. Am J Respir Crit Care Med. 2014;189:1075–81. doi: 10.1164/rccm.201311-2097OC. [DOI] [PubMed] [Google Scholar]
  • 9.Bellomo R, Ronco C, Kellum JA, Mehta RL, Palevsky P. Acute renal failure—definition, outcome measures, animal models, fluid therapy and information technology needs: the Second International Consensus Conference of the Acute Dialysis Quality Initiative (ADQI) Group. Crit Care. 2004;8:R204–12. doi: 10.1186/cc2872. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Mehta RL, Kellum JA, Shah SV, et al. Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Crit Care. 2007;11:R31. doi: 10.1186/cc5713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int. 2012;2:1–138. [Google Scholar]
  • 12.Wilson FP, Yang W, Feldman HI. Predictors of death and dialysis in severe AKI: the UPHS-AKI cohort. Clin J Am Soc Nephrol. 2013;8:527–37. doi: 10.2215/CJN.06450612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Bihorac A, Delano MJ, Schold JD, et al. Incidence, clinical predictors, genomics, and outcome of acute kidney injury among trauma patients. Ann Surg. 2010;252:158–65. doi: 10.1097/SLA.0b013e3181deb6bc. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Bihorac A, Yavas S, Subbiah S, et al. Long-term risk of mortality and acute kidney injury during hospitalization after major surgery. Ann Surg. 2009;249:851–58. doi: 10.1097/SLA.0b013e3181a40a0b. [DOI] [PubMed] [Google Scholar]
  • 15.Garzotto F, Piccinni P, Cruz D, et al. RIFLE-based data collection/management system applied to a prospective cohort multicenter Italian study on the epidemiology of acute kidney injury in the intensive care unit. Blood Purif. 2011;31:159–71. doi: 10.1159/000322161. [DOI] [PubMed] [Google Scholar]
  • 16.Newsome BB, Warnock DG, McClellan WM, et al. Long-term risk of mortality and end-stage renal disease among the elderly after small increases in serum creatinine level during hospitalization for acute myocardial infarction. Arch Intern Med. 2008;168:609–16. doi: 10.1001/archinte.168.6.609. [DOI] [PubMed] [Google Scholar]
  • 17.Ejaz AA, Martin TD, Johnson RJ, et al. Prophylactic nesiritide does not prevent dialysis or all-cause mortality in patients undergoing high-risk cardiac surgery. J Thorac Cardiovasc Surg. 2009;138:959–64. doi: 10.1016/j.jtcvs.2009.05.014. [DOI] [PubMed] [Google Scholar]
  • 18.Tumlin JA, Finkel KW, Murray PT, Samuels J, Cotsonis G, Shaw AD. Fenoldopam mesylate in early acute tubular necrosis: a randomized, double-blind, placebo-controlled clinical trial. Am J Kidney Dis. 2005;46:26–34. doi: 10.1053/j.ajkd.2005.04.002. [DOI] [PubMed] [Google Scholar]
  • 19.Berger MM, Soguel L, Shenkin A, et al. Influence of early antioxidant supplements on clinical evolution and organ function in critically ill cardiac surgery, major trauma, and subarachnoid hemorrhage patients. Crit Care. 2008;12:R101. doi: 10.1186/cc6981. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Nigwekar SU, Navaneethan SD, Parikh CR, Hix JK. Atrial natriuretic peptide for preventing and treating acute kidney injury. Cochrane Database Syst Rev. 2009;4:CD006028. doi: 10.1002/14651858.CD006028.pub2. [DOI] [PubMed] [Google Scholar]
  • 21.Hsu CY, McCulloch CE, Fan D, Ordonez JD, Chertow GM, Go AS. Community-based incidence of acute renal failure. Kidney Int. 2007;72:208–12. doi: 10.1038/sj.ki.5002297. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Aitken E, Carruthers C, Gall L, Kerr L, Geddes C, Kingsmore D. Acute kidney injury: outcomes and quality of care. QJM. 2013;106:323–32. doi: 10.1093/qjmed/hcs237. [DOI] [PubMed] [Google Scholar]
  • 23.National Confidential Enquiry into Patient Outcome and Death. Adding insult to injury: a review of the care of patients who died in hospital with a primary diagnosis of acute kidney injury (actute renal failure) London: NCEPOD; 2009. [Google Scholar]
  • 24.Wilson FP, Bansal AD, Jasti SK, et al. The impact of documentation of severe acute kidney injury on mortality. Clin Nephrol. 2013;80:417–25. doi: 10.5414/CN108072. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Strom BL, Schinnar R, Aberra F, et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010;170:1578–83. doi: 10.1001/archinternmed.2010.324. [DOI] [PubMed] [Google Scholar]
  • 26.Strom BL, Schinnar R, Bilker W, Hennessy S, Leonard CE, Pifer E. Randomized clinical trial of a customized electronic alert requiring an affirmative response compared to a control group receiving a commercial passive CPOE alert: NSAID—warfarin co-prescribing as a test case. J Am Med Inform Assoc. 2010;17:411–15. doi: 10.1136/jamia.2009.000695. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Kucher N, Koo S, Quiroz R, et al. Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005;352:969–77. doi: 10.1056/NEJMoa041533. [DOI] [PubMed] [Google Scholar]
  • 28.Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001;345:965–70. doi: 10.1056/NEJMsa010181. [DOI] [PubMed] [Google Scholar]
  • 29.Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18:327–34. doi: 10.1136/amiajnl-2011-000094. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Wilson FP, Reese PP, Shashaty MG, et al. A trial of in-hospital, electronic alerts for acute kidney injury: design and rationale. Clin Trials. 2014;11:521–29. doi: 10.1177/1740774514542619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Matts JP, Lachin JM. Properties of permuted-block randomization in clinical trials. Control Clin Trials. 1988;9:327–44. doi: 10.1016/0197-2456(88)90047-5. [DOI] [PubMed] [Google Scholar]
  • 32.Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011;30:2310–17. doi: 10.1377/hlthaff.2010.1111. [DOI] [PubMed] [Google Scholar]
  • 33.Zhao X, Zhao Q, Sun J, Kim JS. Generalized log-rank tests for partly interval-censored failure time data. Biom J. 2008;50:375–85. doi: 10.1002/bimj.200710419. [DOI] [PubMed] [Google Scholar]
  • 34.Kahan BC, Morris TP. Improper analysis of trials randomised using stratified blocks or minimisation. Stat Med. 2012;31:328–40. doi: 10.1002/sim.4431. [DOI] [PubMed] [Google Scholar]
  • 35.Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43:1130–39. doi: 10.1097/01.mlr.0000182534.19832.83. [DOI] [PubMed] [Google Scholar]
  • 36.Porter CJ, Juurlink I, Bisset LH, Bavakunji R, Mehta RL, Devonald MA. A real-time electronic alert to improve detection of acute kidney injury in a large teaching hospital. Nephrol Dial Transplant. 2014;29:1888–93. doi: 10.1093/ndt/gfu082. [DOI] [PubMed] [Google Scholar]
  • 37.Colpaert K, Hoste EA, Steurbaut K, et al. Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit Care Med. 2012;40:1164–70. doi: 10.1097/CCM.0b013e3182387a6b. [DOI] [PubMed] [Google Scholar]
  • 38.Balasubramanian G, Al-Aly Z, Moiz A, et al. Early nephrologist involvement in hospital-acquired acute kidney injury: a pilot study. Am J Kidney Dis. 2011;57:228–34. doi: 10.1053/j.ajkd.2010.08.026. [DOI] [PubMed] [Google Scholar]
  • 39.Avorn J, Soumerai SB. Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based “detailing”. N Engl J Med. 1983;308:1457–63. doi: 10.1056/NEJM198306163082406. [DOI] [PubMed] [Google Scholar]
  • 40.Grams ME, Estrella MM, Coresh J, et al. Fluid balance, diuretic use, and mortality in acute kidney injury. Clin J Am Soc Nephrol. 2011;6:966–73. doi: 10.2215/CJN.08781010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Bagshaw SM, Brophy PD, Cruz D, Ronco C. Fluid balance as a biomarker: impact of fluid overload on outcome in critically ill patients with acute kidney injury. Crit Care. 2008;12:169. doi: 10.1186/cc6948. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Anderson HJ. Avoiding ‘alert fatigue’. Health Data Manag. 2009;17:42. [PubMed] [Google Scholar]
  • 43.Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J Am Med Inform Assoc. 2012;19:e145–48. doi: 10.1136/amiajnl-2011-000743. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Lee EK, Mejia AF, Senior T, Jose J. Improving patient safety through medical alert management: an automated decision tool to reduce alert fatigue. AMIA Annu Symp Proc. 2010;2010:417–21. [PMC free article] [PubMed] [Google Scholar]
  • 45.Ricci Z, Cruz D, Ronco C. The RIFLE criteria and mortality in acute kidney injury: a systematic review. Kidney Int. 2008;73:538–46. doi: 10.1038/sj.ki.5002743. [DOI] [PubMed] [Google Scholar]
  • 46.Hoste EA, Kellum JA. Acute kidney injury: epidemiology and diagnostic criteria. Curr Opin Crit Care. 2006;12:531–37. doi: 10.1097/MCC.0b013e3280102af7. [DOI] [PubMed] [Google Scholar]

RESOURCES