Utility of Electronic Medical Record Alerts to Prevent Drug Nephrotoxicity

Abstract

Nephrotoxin-induced AKI is an iatrogenic form of AKI that can be potentially avoided or ameliorated by prompt recognition and appropriate prescriber actions. Drug-targeted alerts, either for patients at risk of AKI or patients with existing AKI, may lead to more appropriate drug dosing and management and improved clinical outcomes. However, alerts of this type are complicated to create, have a high potential for error and off-target effects, and may be difficult to evaluate. Although many studies have shown that these alerts can reduce the rate of inappropriate prescribing, few studies have examined the utility of such alerts in terms of patient benefit. In this review, we examine the current state of the literature in this area, identify key technical challenges, and suggest methods of evaluation for drug-targeted AKI alerts.

Introduction

AKI is a frequent complication of hospitalizations and introduces a number of challenges with regard to medication administration and dosing (1–4). Depending on the definition of “nephrotoxin,” the prevalence of nephrotoxin exposure in hospitalized patients may exceed 75% (5,6), a significant statistic given that each exposure to a nephrotoxic agent increases a patient’s odds of developing AKI by 53% (7). Among patients who develop AKI, approximately 20% of cases are thought to be due to nephrotoxin exposure (8–10). Furthermore, despite international guidelines for the appropriate management of AKI, which focus on cessation and avoidance of nephrotoxins, physicians frequently fail to stop nephrotoxic medications or dose adjust kidney-cleared medications as kidney function worsens (11–13). In an effort to attenuate these issues, quality control and clinical research efforts have examined the utility of targeted alerts to modify provider behavior and improve patient outcomes. Herein, we discuss the challenges in developing, implementing, and evaluating these alerts.

Conceptual Framework

The efficacy of an AKI alert is dependent on multiple inter-related factors as schematized in the conceptual model in Figure 1. Alerts are least effective when a provider is already aware of the clinical situation being alerted (endogenous recognition) or when the alert is not actionable (e.g., there are no alternative therapies available). Conversely, alerts may be most effective when providers are unaware of AKI (e.g., in situations where the creatinine is rising slowly) or when actionable steps are immediately evident.

Figure 1.

A conceptual model for the efficacy of AKI alerts highlights that an alert's success is dependent on both patient and provider factors. Those represented by the green arrow are factors that would favor implementation of an alert system by providing scenarios where alerts may increase the ability of providers to recognize specific conditions and take immediate actions to improve patient prognosis. Those represented by the red arrow are factors that would discourage adoption of an alert system. If providers are already aware of a condition or if no actionable alternatives are available, alert fatigue may reduce efficacy of not only the alert in question but also, other clinically important alerts.

Several studies have shown the potential efficacy of alerts in the hospital setting (14–16); however, extensive reviews of the clinical decision support literature have consistently described specific elements that increase provider adherence and thus, the likelihood of alert success (17–20). These factors include the speed of the information system, timing of the alert (real time and at the point of care), minimal disruption of and integration into provider workflow, simplicity and clarity of the message, and provision of references and sufficient information within the alert. One study showed the positive effects of incorporating more user-friendly changes into a creatinine clearance alert system designed to reduce prescribing errors (21). The authors incorporated educational information, added links to additional laboratory information, and changed the timing of the alert to proximate the point of medical decision making, resulting in 43% fewer prescribing errors compared with the original alerts when tested by physicians in mock clinical scenarios.

Critical aspects to improve alert efficacy are continual monitoring of decision support system performance and collection of feedback from its users, which have shown that user acceptance, perception, and confidence in the support system are prominent causes of frequent alert over-ride. One study that surveyed clinician perceptions after use of an early warning and response system for severe sepsis reported a change in patient management in about 50% of all patients as a result of the alert, while only about one third of users described the alert as useful (22). Feedback from providers suggested that such low acceptance stemmed from possible low specificity of the alert, because they perceived most alerted patients to be stable at the time of the alert and were able to more quickly recognize illness as a result of the alert in only a few patients. Similarly, a study gathering clinician perceptions of an automated drug alert within a provider order entry system showed that, despite high potential for increased physician recognition of drug interactions, over one half of the surveyed clinicians perceived the poor signal-to-noise ratio as a significant barrier to their use (23). Shah et al. (24) attempted to address this issue by designing more selective alerts for ambulatory drug contraindications as a means to improve clinician acceptance. Computerized alerts were designed for selected drug contraindications that were deemed of highest clinical relevance. Alerts were then divided into disruptive alerts (requiring a provider action) for contraindications with the highest clinical severity and nondisruptive alerts (accounting for 71% of all alerts and displaying on top of the computer screen with minimal clinician interruption). Consequently, user acceptance of the more intrusive alert was 67%, an increase over previously published results in similar settings. Of course, this risks under alerting, and continual provider feedback is necessary to create an optimal balance of user acceptance and appropriate alert frequency.

Potential Harms of Alerting

Although alerts are commonly considered low-risk interventions, three potential harms are worth mentioning. First is the development of alert fatigue, a condition of decreased attention to alerts due to the proliferation or frequency of alerting (25–28). An alert, in the setting of nephrotoxin exposure, should increase “exogenous” recognition of the clinical scenario (recognition due to forces outside of the provider’s own mind). In contrast, a capable provider may be likely to have endogenous recognition of the importance of the clinical scenario in the absence of the alert. Alert fatigue is more likely when endogenous recognition and exogenous recognition are in conflict (i.e., an alert occurs for a condition of which a provider is already aware). Additionally, alert fatigue can occur due simply to the proliferation of alerts, even if they are informative (20,29,30). This is well documented in the intensive care unit, where frequent chimes, buzzers, and beeping fade quickly into background obscurity. Alert fatigue can be particularly insidious, because the proliferation of new alerts for different conditions can affect old alerts that were once proven to be successful. This argues for not only robust assessment of practical and clinical alert efficacy, but also, reassessment over time to ensure that alerts are still functioning as intended. Moreover, we should be comfortable discontinuing alerts, even if they are informative or have become “standard of care,” should they fail to continue to show a clinical benefit.

The second potential harm from alert systems is the risk of inattention to nonalerted patients. A highly effective alert reminding a physician that a certain drug should be redosed in a patient with AKI may lead that physician to believe that he or she will be alerted for all drugs that need to be redosed in a patient with AKI, leading to increased harms from those other agents. This may be difficult to measure, because often, studies may only look at the harms from the drug of interest rather than other agents to which a patient may be exposed.

Third, alerts may be categorized as “soft stop” alerts or “hard stop” alerts. A “soft stop” alert conveys a potential safety issue with existing alternatives but with no required action or acknowledgment from the provider. However, these hold a greater likelihood of being overlooked due to reasons, such as alert fatigue, as described above. (31,32) “Hard stop” alerts, which require a provider to get special approval to continue a drug, may be highly effective at decreasing the use of the drug, but they may elicit unintended adverse consequences and thus, may not be the best choice for all patients. In a randomized trial evaluating a “hard stop” rule for the coadministration of trimethoprim-sulfamethoxazole and warfarin, researchers found a dramatic 88% decrease in patients given both drugs. However, the study was stopped early because of four adverse events—two patients had a significant delay of therapy of trimethoprim-sulfamethoxazole, and two patients had a significant delay of therapy with warfarin when the alert essentially over-rode physician judgement regarding the importance of immediate treatment. This is especially applicable to patients with AKI, because nephrotoxic agents, such as angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers, and a variety of antibiotics, are frequently prescribed in elderly and critically ill patients. Although rare, significant risk exists for underdosing or complete avoidance of necessary medications, which may lead to potentially fatal therapeutic failure as shown by a case in which one patient with AKI received fatally subtherapeutic dosing of antibiotic for a pneumonia infection (9). Alerts may exacerbate this issue if providers place too much credence on the alert system rather than their own medical judgement. Careful consideration of a variety of factors, including patient risk factors and history, degree and type of injury, and their specific effects on volumes of distribution and other pharmacokinetic factors, and available alternative therapeutic strategies must be performed in the decision to continue such therapies during the course of kidney injury or resume them during the recovery. Because the speed and extent of recovery vary at the patient level, there is no hard boundary for nephrotoxin avoidance, and dosing becomes even more challenging. An individualized approach that includes a dynamic medication monitoring plan and frequent assessment throughout the entirety of disease progression is recommended and may be too complex to be captured for each patient by an overly broad and generalized alert (7).

Alert Intrusiveness

Alerting is a double-edged sword. There is a clear relationship between alert intrusiveness and efficacy, which we display in Figure 2. At the most extreme, “hard stop” alerts are highly effective but dramatically reduce provider autonomy and may introduce unintended consequences (such as when a truly necessary drug goes unused) as discussed above (33). At the other extreme, passive alerts delivered outside of the context of the electronic health record (EHR) may go ignored or unnoticed, such as was the case with our randomized trial of general alerts for AKI (34). The outlier on the intrusiveness-effectiveness line is “reflex” actions—scenarios where no alert is given but a clinical action is taken automatically. Commonly used for laboratory testing (where for example, a reflex differential may be applied to any white blood cell count over a given threshold), automated drug cessation or drug dosage adjustment in the face of changing kidney function has not been rigorously evaluated.

Figure 2.

Alert effectiveness is strongly related to the degree with which it intrudes on usual processes of care, creating a conflict in terms of clinical outcomes. EHR, electronic health record.

Limitations of Electronic Systems

Technical limitations can seriously limit the efficacy of alert systems, and they may explain why even “successful” alerts do not result in marked changes in provider behavior or patient outcomes. Integration of alerts at the point of care is likely to have a larger effect than “offline” alerting. This can be accomplished by trained personnel who personally interact with health care providers, although this approach is costly and time intensive. Direct integration within the EHR may be preferable, but many EHR systems are proprietary; a great deal of expertise is needed to create alerts within that environment.

The requirement that creatinine be in steady state to estimate a GFR is rarely met in hospitalized patients. To that end, eGFR or even absolute creatinine “thresholds” for drug dosing may overestimate dosages while kidney function is declining, and underestimate dosages when kidney function recovers. Several equations exist to estimate GFR in the setting of dynamic creatinine changes, but the use of these techniques to guide drug dosing has not been evaluated (35).

“False positive” alerts, in which a provider receives an alert that is incorrect or inappropriate, can severely hamper enthusiasm for the alert in general. For a nephrotoxic AKI alert, there are multiple potential points of failure. First, AKI itself may be misidentified on the basis of random variation in creatinine, changes in laboratory equipment standardization, or inclusion of individuals receiving dialysis (36). Second, nephrotoxic exposure may be misclassified by the alert if it unintentionally includes drugs that are not truly nephrotoxic, if it captures exposure to a drug that has already been discontinued, or if it fails to capture a drug due to a change in database coding for the agent (such as when a new manufacturer provides the agent to a hospital). Furthermore, alerts with specific dosing recommendations add another layer of complexity, which can lead to uninformative or incorrect alerts.

Finally, electronic alert systems may not be readily generalizable to different care settings, even in the relatively narrow space of inpatient nephrotoxic AKI alerts. Local practices with regards to drug dosing, pharmacist monitoring, and patient mix may have strong influence on alert effectiveness. This suggests that some “tailoring” of alert systems might be beneficial.

Figure 3 summarizes the key benefits, harms, and limitations of electronic alert systems discussed throughout the text.

Figure 3.

Benefits, disadvantages, and limitations of an electronic medical record (EMR) alert system. A broader discussion of each is in the text.

Drug Alerts for Hospitalized Patients with CKD

Because of the more static nature of CKD, most studies evaluating the efficacy of drug dosage alerts have occurred in this setting.

An early study evaluating computerized clinical decision support for drug dosing in CKD analyzed 7490 hospitalized patients at a single tertiary care hospital (12). Before its introduction, 46% of prescriptions for kidney-cleared or nephrotoxic agents written had an inappropriate dose, and 65% had an inappropriate frequency. After its introduction, only 33% were deemed to have an inappropriate dose, and 41% had an inappropriate frequency (P<0.001 for both comparisons). Although this study shows that electronic clinical decision support can move the needle in terms of appropriate drug dosing, the high percentage of inappropriate doses even after its implementation suggests that it is far from a panacea.

In 2005, a group from Mt. Sinai reported on the development of a customized alert system that would detect inappropriate dosing of kidney-cleared medications (37). The system generated alerts each morning on the basis of the previous day’s medication and kidney function data. Before implementation, 23% of medications administered were dosed “in excess” compared with 17% under the alert system. This system was notable for the fact that it was augmented via human interaction; trained pharmacists would contact care teams directly with dosing advice.

A small single-center study of 233 patients showed that “contraindicated” alerts can be highly efficacious for the discontinuation of potentially harmful agents (38). For each formulary drug, a “threshold” creatinine clearance was defined, below which an alert would fire to indicate that the drug was contraindicated. Before implementation, 89% of patients with an order for a contraindicated drug would receive at least one dose. After implementation, the rate of receipt of the contraindicated agents decreased to 47%, almost entirely due to immediate cancellation of the order in the face of the alert. This study, like many clinical decision support studies, did not evaluate the clinical effect of these discontinuations.

Drug Alerts in Patients at Risk of AKI

It is common practice to avoid nephrotoxic agents in patients at particularly high risk of AKI, but few studies have systematically examined whether operationalizing and explicit support of that practice would improve patient outcomes.

Goldstein et al. (39) showed that an enhanced surveillance system targeting nephrotoxic agents could both modify a provider’s behavior and reduce rates of AKI. At a large tertiary children’s hospital, his team identified, in near real time, patients receiving an aminoglycoside or a combination of three other nephrotoxic medications. Patients identified in this manner (n=1749) were targeted for more frequent creatinine measurement (because hospitalized children often do not undergo daily laboratory monitoring). In this study, the rate of nephrotoxic medication exposure decreased by 38%, and the rate of AKI decreased by 64%, providing strong support for the hypothesis that reducing nephrotoxin exposure can reduce AKI rates.

In adult populations, theissue of increased creatinine screening is moot given a near-universal practice in the United States of at least daily measurement. Whether risk-based targeting of nephrotoxic agents before AKI would be an effective strategy to reduce AKI incidence is an intriguing one that has yet to be rigorously evaluated. Under such a framework, at-risk individuals (due to comorbidities or laboratory variables associated with future AKI) could be identified and targeted with efforts to reduce their nephrotoxin exposure (40).

Drug Alerts in the Setting of AKI

With its dynamic changes in drug pharmacokinetics, AKI presents both the most challenging and the most promising use case for drug-targeted alerts. A study that examined the effect of alert intrusiveness is particularly instructive in this case. In a quality improvement initiative, McCoy et al. (41) analyzed 1598 adult patients with AKI as defined by a 0.5-mg/dl increase in creatinine over 48 hours who had received one of 122 nephrotoxic or kidney-cleared medications. A passive alert within the medical record, indicating a rising creatinine and instructing the provider to discontinue or change the dose of key medications, had no effect on provider behavior. However, a more intrusive alert, requiring the provider to change the dose of or discontinue the agent or confirm that “this is the correct dose,” led to a substantial increase in dose modification (from 35% to 53%; P<0.001). Again, it remains unclear whether these changes lead to beneficial clinical effect, but the study clearly shows that alert “intrusiveness” is a variable that needs to be strongly considered in alert design.

Although alerts and clinical decision support may be most effective when integrated into the daily workflow, a pre-/poststudy of 1001 patients in a geriatric teaching hospital showed that a clinical decision support system independent of the EHR could dramatically improve dosing of key kidney-cleared medications, including enoxaparin, gentamicin, and vancomycin, when accompanied by comprehensive academic detailing (42). Notably, during episodes of acute kidney impairment, these drugs were held in 38% of patients preintervention and 62% of patients postintervention (P=0.01). Longer-term clinical outcomes were not assessed.

Table 1 provides a summary of the relevant literature, highlighting the use of electronic nephrotoxin alert systems in the setting of both CKD and AKI.

Table 1.

Summary of selected studies of drug alerts in patients with CKD and patients with AKI (43,44)

Study	Title	Study Participants	Duration	Control	Intervention	Measure of Efficacy	Key Results
Hospitalized   patients with CKD
Chertow et al. (12)	Guided medication dosing for inpatients with renal insufficiency	7490 Patients in a single tertiary care hospital	Four consecutive 2-mo intervals	Usual computerized order entry system	Real time computerized decision support system for prescribing drugs in patients with renal insufficiency coupled with computerized order entry system	Rates of appropriate prescription (dose, frequency), length of stay, hospital and pharmacy costs, changes in renal function	A computerized decision support system increased the number of appropriate prescriptions versus the computerized order entry system alone by both dose (67% in the intervention group versus 54% in the control group; P<0.001) and frequency (59% intervention; 35% control)
Nash et al. (37)	Reducing excessive medication administration in hospitalized adults with renal dysfunction	Adult inpatients with impaired kidney function at a 1171-bed academic medical center	Baseline data collection: 7 wk; alert (intervention) data collection: 10 wk	Usual care	Computerized alert system for pharmacists to identify hospitalized patients who had medications requiring dose adjustments in the setting of renal insufficiency coupled with pharmacist feedback	Percentage of medications dosed in excess in the setting of renal insufficiency that were subsequently dose adjusted	During usual care, 23.2% of medications administered to renally impaired adult inpatients were dosed in excess compared with 16.8% after alert implementation (P<0.05)
Galanter et al. (38)	A trial of automated decision supp0ort alerts for contraindicated medications using computerized physician order entry	233 Patients at a single tertiary care hospital	14 mo after alert implementation	Historical cohort established for the fourth month period before alert implementation	Automated alert system built into the electronic medical record system that triggered when an order was made for a drug with a “threshold” creatinine clearance that was greater than the patient's most recently estimated creatinine clearance	Proportions of patients receiving at least one dose of a contraindicated medication	In the historical cohort, 87% of patients received at least one dose of a contraindicated drug versus 47% after alert implementation (P<0.001); in the alert group, 41% of instances in which an alert was given resulted in immediate cancellation of the order
Bhardwaja et al. (44)	Improving prescribing safety in patients with renal insufficiency in the ambulatory setting: The Drug Renal Alert Pharmacy (DRAP) program	6125 Adult patients in an integrated health care system with estimated creatinine clearance of 5 ml/min or lower and not receiving dialysis	15 mo	Usual care	A computerized tool used to alert pharmacists at the time of dispensing to errors in drug selection and dosing in patients with renal insufficiency	Proportion of mediation errors (target drugs that should be avoided or were inappropriately dosed)	The proportion of medication errors in the alert group (33%) was significantly lower than that for the control group (49%; P<0.001)
Terrell et al. (43)	Computerized decision support for medication dosing in renal insufficiency: A randomized, controlled trial	42 Physicians in an academic emergency department randomized to either intervention or control group	2 yr	Usual care	A decision support system that provided dosing recommendations for targeted medication in adult patients with renal insufficiency being discharged and displayed when the patient’s estimated creatinine clearance was below	Proportion of targeted medications that were excessively dosed among all prescriptions for the targeted population of patients	Physicians in the control group had a larger occurrence of excessively dosing of medications (74% of prescriptions) compared with the control group (43%)
Hospitalized patients  at risk of AKI
Goldstein et al. (39)	A sustained quality improvement program reduces nephrotoxic medication-associated acute kidney injury	1749 Noncritically ill hospitalized children in a quaternary pediatric inpatient hospital receiving intravenous aminoglycoside or more than three nephrotoxins	3 yr, 7 mo	Prospective study with no control	Automated system to identify patients exposed to nephrotoxins in near real time and recommend more frequent creatinine measurement	Nephrotoxic medication exposure, AKI rates	After implementation, the rate of exposure to nephrotoxic medications decreased by 38%, and the rate of AKI decreased by 64%
Hospitalized patients  with AKI
McCoy et al. (41)	A computerized provider order entry intervention for medication safety during acute kidney injury: A quality improvement report	1598 Adult inpatients in an academic tertiary care facility with a minimum 0.5-mg/dl increase in serum creatinine over 48 h after an order of one of 122 nephrotoxins or medications excreted by the kidneys	1 yr, 8 mo	Usual care before alert implementation (717 patients)	(1) A passive popup alert that displayed for patients with a 0.5-mg/dl increase in serum creatinine and prescribed a targeted medication; (2) an interruptive alert appearing when providers tried to exit from an ordering session without adjusting medication as suggested by the passive alert as required a provider action	Discontinuation or modification of a target medication within 24 h of alert; time to discontinuation or modification	In response to the interruptive alert, medication discontinuation/modification improved from 35.2% preintervention to 52.6% postintervention (P<0.001); rates of this modification were significantly faster than those preintervention (P<0.001)
Roberts et al. (42)	Clinical decision support implemented with academic detailing improves prescribing of key renally cleared drugs in the hospital setting	300-Bed teaching hospital	5 mo	Usual care 6 mo before alert implementation	An automated system that calculated and updated renal function and doses of key drugs adjusted for renal function; academic detailing incorporated a 15-min session with clinicians on how to navigate the program	Rate of dosing conformity and management for key renally cleared drugs in hospitalized patients	Dosing conformity improved for enoxaparin (from 68% to 86%; P=0.03), gentamicin (from 63% to 87%; P=0.01), and vancomycin (from 47% to 77%; P=0.07); during episodes of acute renal impairment, renally cleared medications were held in 62% of cases in patients during the intervention period versus 38% of cases in patients in the preintervention period

Methods of Evaluation

Given the risk of alert fatigue, all alerts should be evaluated to ensure effectiveness. The minimum bar for effectiveness is a practical measure—the rate of nephrotoxic medication discontinuation, for example, or the rate of appropriate dosing for kidney-cleared medications. Although this standard may be sufficient to show that the alert is acting as intended, it does not capture the off-target effects of alerts as described above, and thus, a more stringent metric for alert efficacy should be evaluated. Hard clinical outcomes, like the rate of dialysis or death, may be impractical to assess due to the low prevalence of these events and the likelihood of secular trends in patient characteristics influencing the results (in the absence of a randomized trial). However, AKI metrics, including peak achieved creatinine and duration of AKI, may offer reasonable surrogates for these outcomes (45).

Beyond simply reassuring the designers of alert systems that their work has been put to good use, evidence of clinical effectiveness may increase the faith that health care providers put in the alert system, leading to broader adoption (46). As such, feedback to those receiving alerts regarding performance over time may be particularly important, because multiple studies have shown that alert efficacy may wane as the novelty wears off. In terms of measuring alert effectiveness, change in drug dosing or cessation of the nephrotoxic agent is a critical practical measure. However, more broadly, the clinical efficacy of a drug-targeted AKI alert should be assessed in terms of its ability to show improved clinical outcomes, such as a reduced rate of AKI progression, dialysis, or (perhaps) death.

Summary and Conclusions

Electronic alerts for AKI in the setting of nephrotoxin exposure hold promise to dramatically change the rates of nephrotoxin exposure and modestly improve clinical outcomes in patients with AKI. However, the potential benefits of these alerts need to be weighed against the very real concern of increasing alert fatigue and the need to ensure that systems are robust with an extremely low false positive rate. As shown in the examples above, the intrusiveness, timing, and mode of alerting as well as the type of information relayed must all be considered and optimally balanced to create the most beneficial alert while mitigating associated risks. Randomized trials generating long-term data of outcomes in hospitalized patients with nephrotoxin exposure are warranted to better understand the utility and risks associated with the adoption of electronic alerting.

Disclosures

None.