“Stealth” Alerts to Improve Warfarin Monitoring When Initiating Interacting Medications

ABSTRACT

BACKGROUND

As electronic health records (EHRs) become widely adopted, alerts and reminders can improve medication safety, but excessive alerts may irritate or overwhelm clinicians, thereby reducing their effectiveness. We developed a novel “stealth” alert in an EHR to improve anticoagulation monitoring for patients prescribed a medication that could interact with warfarin. Instead of alerting the prescribing provider, the system notified a multidisciplinary anticoagulation management service, so that the prescribing clinicians never saw the alerts. We aimed to determine whether these “stealth” alerts increased the frequency of anticoagulation monitoring following the co-prescription of warfarin and a potentially interacting medication.

METHODS

We conducted a pre-post intervention study, analyzed using an interrupted time-series, within a large, multispecialty group practice that uses a common EHR. The study included a 12-month period preceding the intervention, a 2-month period during intervention implementation, and a 6-month post-intervention period. The primary outcome measure was the proportion of patients completing anticoagulation monitoring within 5 days of a new co-prescribing event.

RESULTS

Prior to implementation of the stealth alert, 34 % of patients completed anticoagulation monitoring within 5 days after the prescription of a medication with a potential warfarin interaction. After implementation of the alert, 39 % completed testing within 5 days (odds ratio 1.24, 95 % confidence interval 1.12–1.37).

CONCLUSIONS

Stealth alerts increased the proportion of patients who underwent anticoagulation monitoring following the prescription of a medication that could potentially interact with warfarin. This team-based approach to clinical-decision support directs alerts away from prescribing clinicians and toward individuals who can directly implement them.

INTRODUCTION

As electronic health records (EHRs) become widely adopted, there is increasing interest in computerized clinical-decision support. For example, alerts delivered to clinicians at the time of co-prescribing have been shown to reduce the co-prescription of warfarin with potentially harmful partner agents.¹ However, clinicians may ignore this type of clinical-decision support due to “alert fatigue” from a myriad of other less significant drug alerts.^2–4 Also, in situations when the co-prescription is unavoidable, existing clinical-decision support may not instruct the clinician to initiate frequent monitoring. Novel IT-based strategies are needed to streamline care, facilitate a more team-based approach, and off-load busy primary care physicians.

Warfarin is widely used to treat patients undergoing anticoagulation. While warfarin is effective in providing anticoagulation, it is difficult to manage due to its narrow therapeutic window and interaction with various medications and foods. Warfarin leads all other drugs in causing emergency room visits due to adverse effects.⁵ The coadministration of warfarin with many other medications can lead to fluctuation in the anticoagulant effect of warfarin.⁶ Therefore, co-administration of warfarin with these drugs either should be avoided or, when unavoidable, should be accompanied by more frequent monitoring of the international normalized ratio (INR).⁶

Our large, multispecialty medical group uses a centralized Anticoagulation Management Service (AMS) to manage the anticoagulation care of approximately 4,200 patients. Our EHR includes a standard module that provides drug-drug interaction alerts to the ordering clinician at the time of prescription. Although the existing alerts advise the clinician of potential warfarin-drug interactions, optimal follow-up depends on the clinician taking note of the alert, deciding on the time for INR follow-up, informing the patient, and notifying the AMS of the encounter and plan. Each of these steps represents an opportunity for disruption of optimal care. Therefore, we developed an alert to notify the AMS automatically whenever a clinician orders a medication with a potential warfarin interaction.

We performed this study to assess the effect of these additional EHR alerts on rates of INR monitoring, following the initial co-administration of a potentially interacting medication in the presence of warfarin anticoagulation. We refer to these alerts as “stealth” alerts, since they are automatically routed to the AMS while remaining invisible to the prescribing clinician, who already has seen the initial drug-drug interaction alert. This approach intends to promote team-based management and avoids burdening the prescriber with additional alerts. We hypothesized that the presence of these alerts would increase the frequency of monitoring the INR level within the first five days after the co-prescription of the interacting drugs, thereby minimizing adverse effects from inadvertent anticoagulation changes and thus increasing anticoagulation safety.

METHODS

Setting and Participants

The study was conducted within Harvard Vanguard Medical Associates, a large, integrated, multispecialty group practice with approximately 630 physicians caring for 495,000 adult and pediatric patients in 17 offices across eastern Massachusetts.⁷ All physicians in the practice use the Epic® EHR,⁸ through which all prescribed medications are electronically ordered. All patients receiving warfarin, numbering approximately 4,200 at any given time, are eligible for enrollment in the central AMS. The AMS is staffed by 13 nurses and two clinical pharmacists who manage warfarin dosing and monitor INR results, along with four physicians with anticoagulation expertise, who are available for consultation. Although the AMS manages anticoagulation of a few pediatric patients, this study was limited to patients aged 18 years and older.

Study Design

We conducted a pre-post intervention study. There was no contemporaneous control group. Analysis used an interrupted time-series, where the interruption was the implementation of the alerts. Alerts were implemented over a period of approximately two months across all 17 practice sites. The study was approved by the Harvard Pilgrim Health Care Human Studies Committee.

Intervention

In January and February 2009, we implemented stealth alerts, a new electronic system of clinical-decision support for warfarin and co-administration of potentially interacting drugs. This alert was sent to the AMS, rather than the prescribing clinician. The alert was initiated when a medication known to alter the anticoagulation effects of warfarin was first prescribed, with the expectation that the patient may require more frequent monitoring. Table 1 shows the drugs and drug classes that triggered the alerts. The alerts were not triggered by acetaminophen-containing prescription drugs because acetaminophen is more commonly taken over the counter, or by nonsteroidal anti-inflammatory drugs (NSAIDs), aspirin and clopidogrel, whose interaction with warfarin is mediated predominantly through mechanisms other than alteration of warfarin’s anticoagulant effect. We also excluded anti-neoplastic agents, which would already be closely monitored by the prescribing hematologist/oncologist, and other medications prescribed specifically for anticoagulation, such as low-molecular weight heparins. Newer oral anticoagulants such as dabigatran were not available at the time of the study.

Table 1.

Drugs and Drug Classes Triggering Stealth Alerts*

Allopurinol
Amiodarone and dronedarone
Anti-androgens (e.g., flutamide)
Antibacterial agents (including cephalosporins and other classes)
Antifungal agents (e.g., ketoconazole)
Barbiturates (e.g., Phenobarbital)
Fibric acid derivatives (e.g., fenofibrate)
Glucocorticoids (e.g., prednisone)
HMG-CoA reductase inhibitors (statins)
Phenytoin and chemically related anti-epileptic agents (e.g., fosphenytoin)
Protease inhibitors (e.g., ritonavir)
Tamoxifen

*The prescription of any of these medications or classes of medications in a patient receiving warfarin triggered the stealth alert

The stealth alert functions in the following manner: When a clinician electronically prescribes a potentially interacting medication to an AMS-enrolled patient already receiving warfarin, an electronic alert appears in the in-basket of the local AMS manager. This alert advises the manager to review the patient’s record and, if judged clinically necessary, to contact the patient to perform additional laboratory monitoring. The prescribing clinician does not see these alerts. A screen shot of the alert is shown in Figure 1.

Figure 1.

The screen shot (from a "test" patient in the "test" EHR) shows the stealth alert generated for patients in the post-intervention period. Whenever a patient enrolled in the anticoagulation management service (AMS) received a prescription for a medication that could potentially interact with warfarin (see Table 1), this stealth alert was generated in the InBasket (left side of the screen) of the AMS nurse, who would then contact the patient to arrange appropriate follow-up of the international normalized ratio (INR).

Study Period

The 20-month study period (1/1/2008 – 8/31/2009) included the 12-month period preceding the intervention, the 2-month period during which the intervention was being implemented in the clinical sites, and the 6-month post-intervention period.

Outcome Measures

The primary outcome measure, specified a priori, was the proportion of patients completing INR monitoring within 5 days of a new co-prescribing event, measured in each of the 12 one-month periods prior to the intervention and in each of the 6 one-month periods following the intervention. Five-day INR monitoring was chosen because it is within this time period that INR fluctuations due to drug interactions most commonly appear.

The secondary outcome measures were the proportion of patients completing recommended INR monitoring within 3 days, 4 days, and 7 days of a new co-prescribing event, measured in each of the 12 one-month periods prior to intervention and in each of the 6 one-month periods following intervention. For a time-to-event analysis, we also measured days to INR testing following the co-prescribing event.

Data Sources

Data from the EHR were used for all analyses. Patients were identified as enrolled in the AMS during the study period if they had an AMS identifier in the problem list field. For each AMS patient, every prescription (with prescription date, medication name, prescribing clinician, prescribing department, and flag indicating whether it was a new or continuing prescription) and all INR test results during the study period were collected. In addition, the following patient-level variables were collected: date of birth, sex, race/ethnicity (if available), primary care physician, and date enrolled in AMS.

Statistical Analysis

We compared characteristics of patients in the pre-intervention and post-intervention periods. Since a patient may appear in both periods, we used bivariate generalized estimating equations to adjust for the correlation within each individual when comparing the characteristics before and after the intervention.

For the analysis of the primary outcome, we used a logistic version of generalized linear mixed models (GLMM) to estimate the effect of stealth alerts on the outcome of INR testing within 5 days. GLMM can account for correlations within individuals before and after the stealth alerts. We controlled for age and sex in the model.

We also carried out a time-to-event (survival) analysis, with the outcome being time from co-prescribing event until the first INR laboratory test. The date of co-prescribing was the index date (origin). Time until the first INR was calculated as the difference between the date of INR test and the date of co-prescribing. Differences of more than 30 days were discarded, because testing every 30 days was the typical frequency recommended for routine monitoring of patients receiving warfarin during the study period. We developed a Cox proportional hazards model to estimate the adjusted hazard ratio and checked the proportional hazard assumption.^9,10 Because the proportional hazard assumption was not satisfied, we needed to incorporate time by intervention group and sex by intervention group interaction terms in the model. The model accounted for the fact that some individuals were included in both pre-intervention and post-intervention periods.¹¹

RESULTS

Patient Characteristics

Patients in the pre-intervention and post-intervention groups were similar in terms of sex and race/ethnicity, but varied slightly in terms of age and clinical indication for anticoagulation (Table 2). A total of 1,553 patients in the pre-intervention period and 1,709 patients post-intervention experienced a co-prescribing event.

Table 2.

Patient Characteristics

	All patients enrolled in AMS during study period (n = 5871)	Patients with co-prescribing, pre-intervention (n = 1553)	Patients with co-prescribing, post-intervention (n = 1709)	P Value*
Sex				0.46
Female	46.4 %	47.5 %	46.8 %
Male	53.6 %	52.5 %	53.2 %
Age				0.01
18-49 years	8.6 %	5.5 %	6.5 %
50-59 years	12.1 %	10.9 %	10.5 %
60-69 years	20.7 %	18.5 %	21.5 %
70-79 years	26.0 %	30.5 %	27.7 %
80+ years	32.4 %	34.5 %	33.8 %
Race/Ethnicity				0.16
White	80.2 %	84.8 %	83.6 %
Black	8.4 %	7.9 %	7.5 %
Other	6.6 %	5.7 %	6.7 %
Missing	4.8 %	1.5 %	2.2 %
Indication for Anticoagulation†
Atrial Fibrillation	48.5 %	52.5 %	56.2 %	0.02
DVT	22.6 %	12.6 %	17.3 %	<0.001
PE	20.3 %	9.8 %	15.2 %	<0.001
Atrial Flutter	12.3 %	3.2 %	14.3 %	<0.001
DVT/PE Prophylaxis	6.6 %	1.5 %	3.0 %	0.001
Stroke	3.5 %	3.9 %	4.0 %	0.90
Prosthetic valve	3.5 %	4.5 %	4.5 %	0.85

^*P values reflect the comparison of patients with co-prescribing pre-intervention to patients with co-prescribing post-intervention

^†Indications for anticoagulation: Totals may exceed 100 % because patients may have more than one indication for anticoagulation

DVT = deep vein thrombosis; PE = pulmonary embolism

Main Outcome Measure

Prior to implementation of the stealth alert, 34 % of AMS patients completed INR monitoring within 5 days after the prescription of a medication with a potential warfarin interaction. Post implementation, 39 % completed testing within 5 days (odds ratio 1.24, 95 % confidence interval 1.12–1.37).

Figure 2 illustrates the effect of the intervention on the main outcome measure. Comparing the pre-intervention and post-intervention periods, there was a statistically significant change in the level of the trend line. There was no change in the slope of the line. A change in level—e.g., a jump or drop in the outcome after the intervention—constitutes an abrupt intervention effect. A change in trend is defined by an increase or decrease in the slope of the segment after the intervention as compared with the segment preceding the intervention. A change in trend represents a gradual change in the value of the outcome during the segment. An intervention can result in a level change, a slope change, neither, or both.¹² Our intervention led to a level change. We found no evidence of effect modification by sex or by age.

Figure 2.

The interrupted time-series analysis shows the monthly percentage of patients undergoing international normalized ratio (INR) testing within 5 days of co-prescribing event before and after the implementation of the stealth alerts. The figure shows a significant increase in the level, but not the slope, of the trend line. This Finding corresponds to a statistically significant increase in the monitoring rate following the implementation of the stealth alerts.

Secondary Outcome Measures

Table 3 shows the pre-intervention and post-intervention rates of INR testing for each of the secondary outcome measures (i.e., INR testing within 3 days, 4 days and 7 days following the prescription of an agent potentially interacting with warfarin). For each outcome measure, the intervention effect was statistically significant, with an effect size of similar magnitude to the primary outcome measure.

Table 3.

Effect of the Intervention on Primary and Secondary Outcome Measures

Measure	Pre-Intervention (n = 1553)	Post-Intervention (n = 1709)	Adjusted Odds Ratio	95 % CI
Primary Outcome
INR within 5 days	34 %	39 %	1.24	1.12 - 1.37
Secondary Outcomes
INR within 3 days	19 %	24 %	1.31	1.14 - 1.51
INR within 4 days	27 %	33 %	1.25	1.11 - 1.41
INR within 7 days	45 %	51 %	1.27	1.15 - 1.41

CI = Confidence Interval; INR = international normalized ratio

Time-to-Event Analysis

The unadjusted mean time to INR testing following co-prescribing event was 13.2 days before the intervention and 12.1 days after the intervention. In analyses incorporating interaction terms to meet the assumptions of the survival model, the adjusted hazard ratio was 1.24 (95 % confidence interval 1.14–1.36).

DISCUSSION

We developed a novel clinical-decision support tool to prevent adverse drug events associated with the administration of warfarin. We found that stealth alerts—alerts triggered by a clinician’s prescription but delivered to the anticoagulation management service, rather than to the prescribing clinician—increased the proportion of patients who underwent INR monitoring within 5 days following the prescription of a medication that could potentially interact with warfarin. The effect of this intervention was robust, with evidence of effectiveness in increasing monitoring rates as early as 3 days following the alert and as long as 7 days following the alert, and a result that was consistent in time-to-event analysis. Although this study was not designed with sufficient power to detect a reduction in adverse drug events, more vigilant monitoring of INR following the co-administration of potentially interacting medications, as demonstrated in this study, would likely prevent both hemorrhagic and thrombotic complications of warfarin therapy.

Clinical-decision support aids have been shown to reduce co-prescription of warfarin with potentially harmful interacting medications.¹ When these co-prescriptions are required, targeted alerts to prescribers can increase the margin of safety.⁴ Alerts and reminders may improve therapeutic monitoring of medications. Many of these medication safety alerts have been built into EHR’s to enhance patient safety.^13,14 However, few prior studies of computerized clinical-decision support have focused on improving warfarin monitoring.^15,16 To our knowledge, no prior studies have evaluated alerts targeted to accelerate monitoring in the specific clinical context of the co-prescribing of warfarin and potentially interacting drugs.

Four novel dimensions of this intervention deserve mention. First, we were concerned that busy primary care clinicians would have “‘alert fatigue,”^3,17 and might not respond to the new co-prescription safety alerts, having already just seen the standard drug-drug interaction alert in each case. Others have proposed more parsimonious alerting (e.g., fewer triggers in a given system¹⁸) or tailoring alerts to the specific characteristics of individual patients.^4,19 Our strategy was to direct these alerts to the AMS managers, rather than to the ordering clinicians. Kesselheim and colleagues recently suggested a similar approach of directing alerts to individuals who have responsibility for implementing them.² The stealth alerts in this study fostered sharing of responsibility between the primary care team and the disease management team, which can focus on the safety issues pertinent to its area of expertise.²⁰ Integrating specialized care for patients with chronic conditions is a hallmark of the patient-centered medical home.²¹

Second, while co‐prescribing alerts might cause the clinician to modify the prescription, they infrequently provide guidance regarding laboratory monitoring or follow up.^15,16,22 The stealth alert guided the AMS manager to arrange appropriate follow‐up with the patient, which was operationalized as an accelerated INR test if determined to be clinically indicated.

Third, this intervention could be easily replicated in other health care systems using the Epic® EHR; currently more than 250,000 physicians in the US use Epic®, and by 2013, more than 127 million patients’ records are expected to reside on an Epic® EHR.²³ Moreover, the “stealth” approach is not specific to Epic®, and could be implemented in any EHR meeting the standards of the Certification Commission for Health Information Technology.²⁴

Fourth, this alerting system functions irrespective of the pharmacy where the patient fills the prescription, because the alert is triggered by the clinician’s medication order, not by the dispensing of the medication. Although this feature does create some false positive alerts (i.e., medication ordered but not filled by the patient), the communication by the AMS with the patient can easily resolve these alerts.

The magnitude of the intervention’s effect in this study was considerably smaller than that seen in other studies of alerts to improve monitoring. For example, Feldstein and colleagues found that reminders in the EHR sent to the prescribing clinicians more than doubled the rate of monitoring.²² In that study, the patients were not enrolled in a formal program for anticoagulation monitoring; in such programs, as in the present study, patients would be expected to have more vigilant routine monitoring, making it challenging for any intervention to demonstrate a large effect.

The modest size of the intervention effect—a 5 % increase, from 34 % monitored before intervention to 39 % monitored after intervention—should not diminish the potential value of this approach. Many of the co-prescriptions that triggered the alert, such as a 3-day course of a short-acting antibiotic, would have been considered low-risk by the AMS and, therefore, would not have prompted more urgent monitoring. Furthermore, we recognize that in our EHR, many of the alerts occurred in the context of what appeared to be a new prescription for a potentially interacting agent, but instead was a re-writing of an ongoing steady regimen of a medication that would not require accelerated monitoring. Given these considerations, one would not necessarily expect 100 % monitoring within 5 days of co-prescribing. In this context, an absolute 5 % increase may appear more meaningful.

In addition, other factors may have converged to limit the effectiveness of the intervention. Although an alert may have guided the AMS staff to instruct the patient to complete urgent laboratory monitoring, in some cases patients may not have complied with this recommendation. Unfortunately, our data do not afford us the opportunity to examine the individual conversations between AMS and patients to characterize the nature of these interactions.

Several other study limitations should be considered. First, because we did not have a concurrent comparison group, there is a possibility that changes temporally associated with the intervention were attributable to secular factors other than the intervention itself, such as a general increase in ordering INR tests for all patients, regardless of timing. To address this issue, we used an interrupted time-series analysis to demonstrate a stable monthly outcome rate in the pre-intervention period.

Second, as mentioned above, although the stealth alert was designed to occur only in the context of a new prescription for a medication potentially interacting with warfarin, we learned that the alert was also occurring in some cases of a medication refill, e.g., patients receiving a standing dose of phenytoin or simvastatin. This occurred when the prescribing clinician created a new order in the EHR, rather than selecting to “reorder” a standing medication. These “false positives” could have diluted the effect of the intervention, as patients on established medications may not need the same monitoring intensity as patients on new medications. Additionally, the excess alerts could have had the effect of desensitizing the AMS staff, thereby reducing their effectiveness.

Third, conversely, some clinicians may have ordered a new medication using the “re-order” or “refill” function of the EHR. For example, in a patient treated in the remote past with trimethoprim-sulfamethoxazole now presenting with a new infection, the clinician might simply refill the previously prescribed medication from the medication history list, rather than writing a completely new prescription. In this clinical situation, an alert would be appropriate, but none would occur, since the system did not recognize this prescription as “new.” Using a system that would permit certain drugs (such as antibiotics) to initiate alerts at the time of refills, while excluding alerts on drugs more likely to be refills (such as statins), would prevent this problem.

Fourth, on occasion, clinicians prescribed potentially interacting medications that were not filled or taken by the patients. Although the prescription would generate an alert to the AMS manager, the AMS manager could make the determination, after communicating with the patient, that accelerated INR testing was unnecessary. In such cases, lack of a follow-up INR would not imply failure of efficacy of the alert. We suspect that the rate of unfilled prescriptions was small and relatively constant throughout the entire study period, thereby minimizing its potential to diminish the measured effect of the alert.

Finally, the study did not measure outcomes such as intracranial bleeds or strokes, or the time in therapeutic range, which would have been a reasonable surrogate measure for efficacy of the alert. However, we note that time in therapeutic range has consistently hovered at over 80 % for our AMS, with no differences discerned before or after the implementation of the alert.

The results of this study support the conclusion that stealth alerts result in higher rates of anticoagulation monitoring following the initial prescription of medications that potentially interact with warfarin in patients enrolled in an AMS. Increased INR monitoring in these situations likely results in improved patient safety, since the co-prescription of these medications may cause INR values to drift higher or lower than the anticoagulation goals and increase the risk of complications. Earlier monitoring in these situations fosters more frequent warfarin adjustments, and therefore likely maintenance of anticoagulation goal ranges, in addition to prevention of potential adverse effects of over-anticoagulation and under-anticoagulation. Future studies should examine the effect of alerts to accelerate laboratory monitoring on rates of patient outcomes, such as adverse drug events. Because stealth alerts have the potential to reduce the burden of excessive clinical-decision support, this approach should be explored for reducing alert fatigue and improving the quality of care for other chronic disease management systems.