Evolution of an Electronic Health Record-Based Alert to Optimize Venous Thromboembolism Prophylaxis

Abstract

Background

Venous thromboembolism (VTE) prophylaxis in hospitalized patients must balance risks of bleeding and thrombosis. Clinical changes such as bleeding or renal injury can also trigger changes or delays in thromboprophylaxis. Electronic health record alerts (EHRAs) can allow for targeted notification to providers to improve venous thromboembolism prophylaxis and improve patient outcomes at the risk of alert fatigue if not carefully designed and implemented.

Objective

This study aimed to develop and refine an EHRA that minimizes nuisance alerts while facilitating appropriate ordering of VTE prophylaxis for medical patients.

Methods

A multidisciplinary group at a single large safety-net academic medical center developed an EHRA to identify patients at increased thrombosis risk, but without orders for VTE prophylaxis. This was refined over four phases: development and validation, initial monitoring and exclusion criteria adjustment, COVID-19-related modifications, and delayed surveillance and modification. Data analysis evaluated criteria including alert frequency, alert action/utilization, and alert duration.

Results

The EHRA fired an average of 33.3 times per day across all phases of the study. Phase 1 of EHRA implementation showed significantly increased alerts per patient (6.4 to 43.3 alerts per day, p  < 0.01) as well as the percentage of patients with >5 alerts (2.8 to 60.0%, p  < 0.01). Modifications in phase 2 and phase 3 increased alert rates without any significant effect on subsequent action taken by a provider. Phase 4 modifications led to a significant reduction in alert frequency (44.1 to 14.9 alerts per day, p  < 0.01) coupled with a notable increase in provider action (0.24 to 7.73%, p  < 0.01).

Conclusion

This multidisciplinary, provider-centered, intervention improved alert appearance, and information needed to guide providers increased provider engagement 32-fold, with a 3-fold decrease in alert frequency. Despite improvements, ongoing monitoring and maintenance of this alert is important.

Background and Significance

Up to 11% of high-risk hospitalizations are complicated by venous thromboembolism (VTE), which improves to 2.2% with appropriate VTE prophylaxis. 1 Inpatient prevention of VTE requires thoughtful implementation to balance risks of bleeding and thrombosis and is important for patient care as well as reportable metrics. 2 Transitions between phases of patient care, and clinical changes such as bleeding or renal injury can trigger changes or delays in thromboprophylaxis. Although human alerts interventions via pharmacists and support staff, nurse education, provider education, and even patient education are useful and necessary, they are resource-intensive, difficult to sustain, and prone to failure. 3 4 In addition to educational efforts, electronic health record alerts (EHRAs) can allow for targeted notification to providers to improve VTE prophylaxis, improving patient outcomes. 5 6 7

Although previous interventions demonstrate improvements in appropriate administration and patient outcomes, these positive outcomes should be balanced with the increase in alert frequency and potential for alert fatigue. 8 9 Furthermore, poorly timed alerts can significantly interrupt provider tasks, increasing likelihood of ignoring. 9 10 Alert fatigue can be mitigated by thoughtful alert redesign. 11 Several evidence-based strategies for clinical decision support (CDS) implementation to minimize alert fatigue include the 5 Rights of Clinical Decision Support 12 or 10 commandments of CDS. 13

Objectives

At a large safety-net academic medical center specializing in stroke and trauma care, many patients have comorbidities or injuries that warrant regular reassessment of thrombosis and bleeding risk. In 2019, a multidisciplinary group developed an EHRA to identify patients at increased thrombosis risk without orders for VTE prophylaxis. This case report describes the development and iterative improvement of this EHRA over 5 years, to define and decrease nuisance alerts for medical patient populations.

Methods

A single-center retrospective case study of an EHRA to notify providers of patients with missing VTE prophylaxis was developed and iterated over four phases ( Fig. 1 ). This was implemented at a 953-bed, urban, safety-net, level 1 trauma and advanced comprehensive stroke center in Atlanta, Georgia. This study was approved by the Emory Institutional Review Board.

Fig. 1.

Phases of development. EHRA, electronic health record alert.

Population

At this institution, patients are stratified on admission to low, moderate, or high risk of VTE using an internally validated risk prediction tool embedded within the electronic health record (EHR). All patients admitted using the general admission order set and identified by the admitting provider as moderate or high risk of VTE were included for analysis.

Phase 1: Development and Validation

After assembling a multidisciplinary committee of pharmacists, internists, subspecialists, and informaticists, we developed an EHRA to identify patients stratified as moderate or high risk for VTE during hospitalization who did not have active orders for pharmacologic or mechanical prophylaxis as well as several safety and location criteria ( Table 1 ). Pharmacist and physician input was used to define the patient population and common clinical parameters as well as usability testing and accuracy testing in initial design by the informaticist.

Table 1. Initial trigger and exclusion criteria.

Trigger criteria

Exclusion criteria

• Admission stratification to moderate or high risk of venous thromboembolism (VTE) • Admitted to using the general medicine admission order set • No active anticoagulant order (heparin, enoxaparin, argatroban, dabigatran, edoxaban, bivalirudin, warfarin, dalteparin, fondaparinux, or apixaban) • No active order for mechanical prophylaxis in the previous 12 hours • Fires for clinicians capable of ordering medications/procedures upon entering the orders section of the chart

• Patients located in operating room, hemodialysis, anesthesia, perioperative, procedural area, radiology department • Time 7p.m.-7a.m. • Laboratory parameters within the previous 48 hours:  o International normalized ratio (INR) greater than 2.0  o Hemoglobin concentration less than 7.0 g/dL  o Platelets <50,000

For patients meeting the trigger criteria for the EHRA, the EHRA fired to notify treatment team providers (users capable of ordering medications/procedures) upon opening the orders tab of the electronic medical record (EMR). The initial alert was linked to order management or offered acknowledgment reasons which suspended the alert for 24 hours ( Fig. 2 ). After a background pilot of the EHRA from July through September 2019, the EHRA was implemented in October 2019.

Fig. 2.

Initial electronic health record alert (EHRA) for venous thromboembolism (VTE) prophylaxis.

Phase 2: Initial Monitoring and Modification

A multidisciplinary group reviewed alert frequency and appropriateness 2 months after activating the alert. We identified a high burden of alerts among patients receiving twice daily aspirin as prophylaxis for VTE following orthopedic surgery (because patients were admitted using a general medicine order set). As this is an appropriate, evidence-based therapy, we modified exclusion criteria for the EHRA by adding aspirin with twice daily frequency. We also identified ordered-and-held anticoagulants inappropriately triggering the alert, and adjusted the alert to account for these.

Phase 3: COVID-19- Related Modifications

In response to the 2019 COVID pandemic, the hospital adjusted the protocol for VTE prophylaxis for patients admitted with COVID-19, who were admitted using a new process for which the alert was not built. After several incidental cases, we identified the EHRA would not trigger for patients admitted using the new COVID-19 VTE prophylaxis protocol and adjusted the inclusion criteria to include the new COVID-19 process.

Phase 4: Delayed Surveillance and Modification

After usability testing with clinicians, we re-evaluated the EHRA and identified opportunities for improvement. Without changing the inclusion or exclusion criteria, we reformatted the alert to present relevant information including laboratory values, narrowed providers receiving the alert to the primary treatment team (self-identified by the provider upon entering the patient chart), and implemented new functionality allowing for direct ordering from within the alert ( Fig. 3 ).

Fig. 3.

Updated electronic health record alert (EHRA) after phase 4 (data presented in the figure are imaginary).

Process Measure

We developed a dashboard within the EHR to identify patients at moderate to high risk for VTE. The dashboard included information on active VTE prophylaxis orders, unit, and treatment team. We used this tool to identify the proportion of patients eligible for VTE chemoprophylaxis admitted to a general medicine teams who had no active orders for VTE prophylaxis.

Data Retrieval and Analysis

Using EPIC workbench reporting, regarding alert frequency, we identified 30-day time intervals before and after implementation of each phase (excluding the day of implementation). Alert duration was determined by the time of first minus last alerts. Patients with only one alert were assigned a duration of 1 hour. Statistical analyses were performed in Microsoft Excel. Comparative analysis was performed using two-tailed t-tests assuming unequal variance with a = 0.05. Nominal values were compared with a chi-squared test.

Results

During the eight time periods in question (30 days before and after the four phases), the EHRA alerted 7,841 times for 653 individual patients and 2,013 providers.

Alert Frequency

The EHRA fired an average of 33.3 times per day across the study. Before deployment, the EHRA occurred at an average background rate of 6.4 alerts per day, increasing to 43.3 alerts after adjustments and deployment. Phase 2 changes did not significantly affect alert frequency (34.5 vs. 41.0 alerts per day, p  = 0.19). Phase 3 changes with regards to missing patients increased alert frequency from 28.6 to 53.4 alerts per day ( p  < 0.01). Phase 4 changes significantly decreased alert frequency from 44.1 to 14.9 alerts per day ( p  < 0.01) ( Table 2 ).

Table 2. Descriptive and comparative statistics.

	Phase 1			Phase 2			Phase 3			Phase 4
	Pre	Post	p -Value	Pre	Post	p -Value	Pre	Post	p -Value	Pre	Post	p -Value
Number of alerts, n	192	1,299		1,034	1,230		801	1,603		1,278	404
Number of individual patients with alerts, n	109	110		61	61		62	102		81	67
Number of providers alerted, n	N/A	343		284	303		277	363		339	104
Alerts with action, n (%)	N/A	13 (1.0%)	N/A	8 (0.77%)	8 (0.65%)	0.73	7 (0.88%)	15 (0.94%)	0.88	3 (0.24%)	34 (7.73%)	<0.01
Alerts per day, average (SD)	6.4 (2.1)	43.3 (24.1)	<0.01	34.5 (15.3)	41.0 (21.9)	0.19	28.6 (23.8)	53.4 (2.7)	<0.01	44.1 (34.3)	14.9 (13.6)	<0.01
Alerts per alerted patient, average (SD)	1.8 (1.8)	11.8 (18.1)	<0.01	17.0 (20.8)	20 (45.4)	0.62	12.9 (16.8)	15.7 (30.1)	0.45	15.8 (33.7)	6.6 (10.7)	0.02
Patients with >5 alerts, %	2.8%	60.0%	<0.01	73.8%	62.3%	0.17	61.3%	53.9%	0.36	53.1%	37.3%	0.06
Alerts per provider, average (SD)	N/A	3.79 (5.22)	N/A	3.64 (6.53)	4.06 (6.23)	0.43	2.89 (3.26)	4.42 (5.68)	<0.01	3.7 (4.12)	4.23 (4.79)	0.38
Duration of alerts per alerted encounter, days Average (SD)	1.23 (3.18)	1.09 (2.93)	0.73	1.92 (3.25)	1.37 (2.68)	0.31	1.04 (1.99)	0.95 (1.74)	0.77	1.32 (2.93)	0.65 (1.84)	0.09
Patients without VTE chemoprophylaxis orders, %	11.8%	10.5%	0.29	10.5%	10.7%	0.91	10.5%	11.9%	0.30	14.6%	13.8%	0.57

Abbreviations: N/A, not applicable; SD, standard deviation; VTE, venous thromboembolism.

Alert Action/Utilization

In the overall study, 1.14% of alerts resulted in an action by the provider. In the first 30 days after phase 1, the alert had an action rate of 1.0% (13 actions per 1,299 alerts). The percentage of alerts with action remained steady before and after phases 2 and 3 (0.77, 0.65, 0.88, and 0.94%, respectively). Phase 4 modifications increased the alert action rate 32-fold, from 0.24 to 7.73% ( p  < 0.01).

It is possible that providers addressed the conditions triggering the alert outside of the EHRA. To determine this, we measured the duration of alerts (days) per alerted encounter. For encounters which required an alert, the alert fired for 1.17 days on average, with the lowest average duration after phase 4 implementation ( Table 2 ). There were no statistically significant differences in pre/post phases for alert duration; however, after phase 4, the alert duration was numerically lowest (0.65 days on average).

In order to evaluate impact on overall VTE prophylaxis prescribing, we measured the proportion of patients on internal medicine teams at moderate or high risk for VTE with chemoprophylaxis orders. Overall, 11.8% of eligible patients received no chemoprophylaxis, ranging from 10.5 to 14.6% depending on the study interval. There were no statistically significant pre–post differences for any phase ( Table 2 ).

Discussion

Although CDS in the form of EHRAs can be very helpful and reliable in managing VTE prophylaxis and other important system issues, this must be thoughtfully balanced with alert frequency and fatigue. Using the 5 Rights of Clinical Decision Support 12 or similar guidelines can narrow and guide design to maximize benefit and minimize harm.

Despite intentional multidisciplinary development, and re-evaluation twice in the subsequent year, with slight modifications, this EHRA maintained a 0.65 to 1% rate of action. Despite the initial work, the alert action rate dropped to 0.23% in the subsequent 3 years, with providers becoming significantly more likely to ignore the alert—likely a manifestation of alert fatigue. The phase 4 revision focused on several of the 5 Rights of Clinical Decision Support principles, with intentional modifications to test and improve usability and allow for direct action to facilitate best practice.

1. Right Information

   Through end-user usability testing, and changes in the EHR functionality, we were able to include recent laboratory, and pharmacologic information at the time of the alert, so users could address the problem without exiting the alert.

2. Right Person

   We identified alerts to providers not responsible for ordering such as consultants, and limited alerts to only the primary service. Furthermore, the update streamlined opt-out criteria for the user. This modification accounts for the decrease in number of providers alerted without a decrease in the average number of alerts per provider.

3. Right Format

   Rather than a link to an external window, new EHR functionality allowed for embedded order panels within the EHRA, which helped to direct providers to best-practice therapeutic options.

Unlike the iterative updates in phases 1 to 3, the phase 4 changes significantly decreased alert frequency, simultaneously increasing alert actions. As alerts only occur when trigger criteria are met, decreasing the average alerts per patient, and there are strong trends toward a decrease in patients requiring five or more alerts and duration of alerts all suggesting improved efficiency and fewer nuisance alerts. Moreover, there was a strong trend toward a decreased duration of alerts from 1.35 to 0.65 days ( p  = 0.09), suggesting earlier action by alerted providers. Ultimately despite a 32-fold improvement, only 7.7% of alerts led to orders or provider action, suggesting opportunities for further improvement, specifically mitigating the ongoing and compounded effect of alert fatigue. Intentional, long-term monitoring of other customized alerts can identify further opportunities for usability increases.

The majority of patients (88.2%) received VTE chemoprophylaxis during the study. Despite the 3-fold decrease in both alert frequency and number of providers alerted, there was no statistically significantly decrease with the phase 4 clinical changes.

Limitations

Several limitations should be considered for this retrospective study. First, over the course of the study, the health system expanded with several new units, new providers, and changes to the EMR. This would expect a numerical increase in the number of alerts over time.

This process was developed during the COVID-19 pandemic which caused significant time-specific changes to patient mix, allowing for more patients to be eligible for alerts during phase 3 of the study. This limits the comparison between this and other phases.

The 30-day time interval for measuring alert frequency truncated certain patient encounters, falsely decreasing some durations of alert, which should have affected each study interval equally.

Finally, the scope of this study is limited to performance measures of the CDS tool as it relates to nuisance alerts, and the process measure of VTE chemoprophylaxis prescription. No data are presented regarding clinical outcome measures such as appropriate use of VTE prophylaxis, rates of VTE, or bleeding. VTE at 90 days occurs in as many as 11% of the highest risk hospitalizations, and bleeding is as rare as 1.6%. 1 This alert impacted 60 to 100 patients per study interval, at the most preventing 1 to 2 VTE per study interval, and likely 0 bleeds. More robust study of clinical outcomes such as VTE or bleeding would require much larger populations or much longer study intervals.

Conclusion

Despite thoughtful initial design, this well-intentioned CDS tool became a nuisance alert with minimal provider engagement. The EHRA changes in phase 4 applied structured “5 Rights of Clinical Decision Support” framework and incorporated end-user feedback. Remarkably, these changes led to a 3-fold decrease in alert frequency, while increasing utilization and action 32-fold. During this time, overall VTE prophylaxis rates remained unchanged at 88%, with a numerical, but not statistically significant, improvement. These results underscore the critical importance of ongoing, thoughtful refinement in CDS interventions.

Our findings emphasize that successful CDS requires surveillance and adaptation to changing clinical practices, technological capabilities, and user behaviors. Thoughtful CDS design is critical to ultimately supporting rather than hindering healthcare providers in delivering optimal patient care.

Clinical Relevance Statement

As many health systems are engaged with electronic health records, thoughtful stewardship of electronic health record alerts and adherence to standards will help decrease nuisance alerts and improve provider engagement.

Multiple-Choice Questions

1. Which of the following interventions improved alert frequency most?

   1. Adding aspirin as appropriate venous thromboembolism prophylaxis

   2. Changing the inclusion criteria to include COVID-19 patients

   3. Making the alert easier to ignore

   4. A multidisciplinary review of the 5 Rights of Clinical Decision Support, including incorporating end-user feedback

   Correct Answer: The correct answer is option d. A multidisciplinary review of the 5 Rights of Clinical Decision Support, including incorporating end-user feedback showed 32-fold improvement in alert actionability and a 3-fold decrease in alert frequency.

2. Electronic health record alerts warrant evaluation for alert fatigue. Which of the following is a potential metric for evaluation of alert fatigue?

   1. Alerts per patient

   2. Alerts per day

   3. Alerts per provider

   4. All of the above

   Correct Answer: The correct answer is option d. Evaluating the number of alerts per patient, alerts per day, and alerts per provider are all metrics to gain insight into the potential for alert fatigue. In addition, looking at the number of providers alerted and time of alerts per patient may also provide insight into the utilization of an electronic health record alert.