The Effect of an Electronic “Hard-stop” Alert on HIV Testing Rates in the Emergency Department

Abstract

Use of electronic alerts in clinical practice has had mixed effects on providers’ prescribing practices. New York City has one of the highest rates of HIV in the United States. Recent New York State legislation requires healthcare providers to offer an HIV test to patients aged 13–64 years during a clinical encounter. Adhering to this requirement is particularly challenging in emergency department (ED) settings, which are frequently overcrowded and under-resourced. The purpose of this study was to evaluate the effect of an electronic “hard-stop” alert on HIV testing rates in the ED. Approximately four months of data were reviewed before and after the implementation of the alert. We found that use of the electronic alert significantly increased documentation of offering an HIV test (O.R. = 267.27, p<0.001) and resulted in a significant increase in HIV testing. Findings from this study add to the current knowledge about the use of electronic alerts for improving disease screening.

Introduction

Numerous studies have evaluated the use of electronic alerts in changing clinicians’ practice in order to improve and standardize patient care. Substantial research has centered on the use of these alerts for computerized provider order entry (CPOE) systems and for medication alerts. Mixed outcomes have been reported from this research. [1] Some studies have demonstrated that using preprogrammed alerts as part of a CPOE system is ineffective in changing prescribing behaviors. [2–5] Other studies have found that electronic alerts as part of an inpatient CPOE system can be extremely effective in changing prescribing behavior. [6–8]

Little research has explored the effectiveness of electronic alerts to improve screening practices, especially in the Emergency Department (ED) setting. The ED is a major component of the United States (US) healthcare system, and considerably different from other hospital care areas. An essential element to the US health care safety net, the ED has an open door policy, providing care 24 hours a day, 7 days a week, 365 days a year, regardless of an individual’s ability to pay, the severity of his or her illness, and irrespective of the presence of adequate ED staff and resources. [9] In recent years, the role of the ED has continued to expand in the US. From 1999 to 2009, it is estimated that there was a 32% increase in the number of annual ED visits from 102.8 to 136.1 million, [10] which has been met with an insufficient supply of resources, resulting in a growing crisis of overcrowding. [11] ED overcrowding is associated with prolonged wait times and higher rates of patients who are not seen, [12, 13] and has resulted in patients receiving care in nontraditional treatment areas, such as waiting rooms, conference rooms and hallways,[14] and has impinged on the provision of safe and efficient care. [15]

At the same time, the ED has been shown to be a very effective and important healthcare setting for testing patients for HIV since the patients who use the ED as their primary or sole source of medical care may be those most affected by the HIV/AIDS epidemic, including racial and ethnic minorities and the socioeconomically disadvantaged.[16, 17] Despite increasingly intensive public health initiatives, it is estimated that 250,000 individuals in the US still remain unaware of their diagnoses, and between 50,000 and 60,000 new infections occur annually. [16, 18] The US National HIV/AIDS Strategy has established a goal of increasing the awareness of HIV status in the US population from 79% to 90% by 2015. Meeting this goal is especially critical in New York City, the setting of our study, which has the largest HIV/AIDS epidemic in the nation, accounting for 17.9% of the estimated number of persons living with HIV in the US. [19] Alarmingly, the New York City epidemic continues to expand with a 40% increase in the number of persons living with HIV since 2000, and an incidence of new infections that is three times the US average. [20, 21]

In response to this public health crisis, there have been a number of policy changes, recommendations, and outreach efforts to increase the number of persons who know their HIV status. In 2006, the CDC published revised recommendations for performing rapid HIV testing in healthcare settings, including EDs. [22] These recommendations called for performing non-targeted rapid HIV screening where prevalence rates are estimated to be ≥0.1%. In response to these recommendations, in 2010, New York State enacted legislation to increase HIV testing and to promote linkage of HIV-positive persons to care. A key provision of the legislation includes that HIV testing must be offered to all persons between the ages of 13 and 64 seeking hospital or primary care services, including visits to the ED. [23] While this legislation has the potential to curb the HIV epidemic, it is especially challenging to implement in the ED since this setting is under-resourced, over-utilized and overcrowded. This legislation was an unfunded mandate and so no resources were provided by New York State to support HIV screening in the ED.

Informatics solutions, such as electronic alerts, are a low cost approach for ensuring adherence to this legislation. Use of electronic order sets and “hard-stop” alerts has the potential to ensure that all providers offer an HIV test to every patient treated in the ED. The purpose of this study was to assess the effectiveness of an electronic alert on HIV testing rates in the ED.

Materials and Methods

Study Setting

Our study setting included the adult ED at a large tertiary care center in New York City, with an annual ED volume of over 83,000 encounters (pediatric and adult visits). The ED is a level-one trauma center and provides specialized cardiac, stroke and burn care. [24] This hospital site has 850 inpatient beds and is part of the NewYork-Presbyterian Healthcare System that includes 5 medical centers with over 2,409 total inpatient beds. [25]

Methods

We analyzed time-stamped data files, also known as system usage logs collected from the electronic health record used in the Adult Emergency Department (Sunrise Emergency Care, Allscripts Corporation, Chicago, IL). Our data set did not include any identifying patient information but included data on patient gender, age, race, ethnicity and severity index (ESI). The ESI is categorized on 5 levels: 1-resuscitation, 2- emergent, 3- urgent, 4- less urgent, 5- non-urgent.

We included patients seen in the adult ED. We excluded patients over 64 years of age since they were not included in the legislation. In addition, we excluded all cases of patients who left the ED before being seen by a prescribing provider.

We analyzed data during a 4.5-month period. Prior to the implementation of an electronic order set (Figure 1), HIV testing was not done in this ED. On July 7, 2011, the electronic order set (“HIV Testing Order Set”) went live. We defined the time period from July 7 until September 14, 2011 as the pre-intervention period. During this time, providers were instructed to use the HIV testing order set. The Adult ED policy as of July 7^th was that every ED patient under the age of 65 was required to be offered an HIV test by their treating physician.

Figure 1.

Electronic Order Set for HIV testing

On September 15, 2011, a hard-stop alert was implemented (Figure 2) and providers were not able to discharge patients from the ED unless they documented HIV testing information. The electronic hard-stop alert appeared when a provider attempted to write an electronic ED discharge order and either 1) an HIV test had not been ordered, or 2) no documentation existed recording why the test was not ordered (i.e., the patient declined, the patient was known to be HIV-positive). We defined the time period from September 15–November 21, 2011 as the post-intervention period. The hard-stop alert only triggered when a provider discharged a patient home from the ED (it did not fire when patients were admitted to the hospital.)

Figure 2.

Electronic “hard-stop” alert

Data Analysis

Data were managed and analyzed using SPSS System for Windows, release 20.0 (IBM, Chicago, IL). Descriptive statistics were used to characterize the number of HIV tests ordered before and after the electronic hard-stop alert. Compliance with the legislation was determined by any documentation in the HIV order set including documenting that an HIV test was not offered.

In addition, we conducted a logistic regression to determine if an electronic hard-stop alert had a significant impact on the completion of the HIV Testing Order Set. We characterized each encounter as belonging to one of the following study groups: 1) pre-intervention, no electronic hard-stop alert, 2) post-intervention, no electronic “hard-stop” alert (these were patients admitted into hospital from the ED), and 3) post-intervention, electronic hard-stop alert (these were patients treated and released from ED). In our regression model, we included gender, ESI and age as covariates. We did not include race/ethnicity in the model due to a large amount of missing data.

Results

During the pre-intervention period, there were a total of 8,994 ED visits by patients between 21 and 64 years of age. 7,048 of these patients were treated in the ED and released. Of these patients, 4.2% were offered an HIV test and declined, 1.9% were tested for HIV, and 0.3% of patients had documentation explaining why the provider did not order the test. For the remaining 93.6% of patients, the provider did not document anything in the HIV Testing Order Set. The remaining patients were admitted to the hospital or walked out. Among the ED patients who were later admitted, 93.6% had no documentation in the HIV Testing Order Set. Compliance included documenting that an HIV test was not offered since the providers oftentimes documented that they did not offer a test because a patient was too sick to decide or that a patient had recently been tested for HIV. Overall compliance with the HIV testing law pre-implementation of the hard-stop alert was 6.3% across all patients (Table 1).

Table 1.

HIV Testing Order Rates – N (%)

Location	Pre-Intervention	Post-Intervention
ED Treated-and-Released Patients	7048	6706
HIV Test Ordered	136 (1.9)	583 (8.7)
HIV Test Not Offered	18 (0.3)	374 (5.6)
HIV Test Offered and Declined	298 (4.2)	5348 (79.8)
No Documentation	6596 (93.6)	401 (6.0)
Compliance	452 (6.4)	6305 (94.0)
Admitted Patients from the ED	1946	2006
HIV Test Ordered	63 (3.2)	223 (11.1)
HIV Test Not Offered	8 (0.4)	120 (6.0)
HIV Test Offered and Declined	45 (2.3)	549 (27.4)
No Documentation	1830 (94.0)	1114 (55.5)
Compliance	116 (6.0)	892(44.5)

During the post-intervention period, from September 15 – November 21, 2011, 8,712 patients 21–64 years of age were seen in the adult ED. At this time, the hard-stop alert was active and providers were not able to discharge patients from the ED unless they completed the HIV Testing Order Set. Providers could not override the alert and needed to complete the Order Set. The impact of the electronic “hard-stop” alert is presented in Table 1. During the post-intervention phase, 6,706 patients were discharged from the ED. Of these patients, there was a 94.0% compliance rate with the HIV testing law. The remaining 401 patients who are represented in the “No Documentation” category consisted of patients who left against medical advice, walked out before being discharged or were deceased.

During the post-intervention phase, there were 2,006 patients that entered the ED and were later admitted to the hospital. For these patients, providers did not encounter a “hard-stop” alert though they were able to use the HIV Testing Order Set. This provided a post-test comparison group of clinical encounters where clinicians were not exposed to the hard-stop alert intervention. In the post-intervention group without a hard-stop alert there was a 44.5% compliance rate with the HIV testing law.

Predictors of Completing the HIV Order Set

Logistic regression showed that providers were much more likely to complete the HIV order set if they were in the post intervention group and had a hard-stop alert than if they were in the pre-intervention group and only had an electronic order set (OR = 267.27, p < .001) (95% CI, 229.60. 311.13). Providers were also more likely to complete the HIV order set if they were in the posttest intervention group and only had an electronic order set, without a “hard-stop” alert, than if they were in the pre-intervention group and only had an electronic order set. (OR = 11.72, p < .001) (95% CI, 9.96, 13.79).

ESI was not a significant covariate in our model (p>.05). However gender and age were significant covariates. Females were more likely to be offered an HIV test than males (OR= 1.87, p<.001) (95% CI, 1.65, 2.12); and younger adults were more likely to be offered a test than older adults (OR= 0.99, p<.01) (95% CI, 0.99, 1.00).

Discussion

Electronic alerts have long been studied, especially in CPOE systems. In some studies, CPOE system interventions have reduced medication error rates; [26, 27] however, electronic alerts are frequently ignored or overridden. Clinicians tend to override the alerts because they are perceived to be inappropriate and a nuisance [28, 29, 30] Given that most CPOE alerts are overridden and little research has been conducted on hard-stop alerts, findings from this study add to the current knowledge about the use of this intervention for changing providers’ practice. Past literature has reported that clinicians find it inappropriate to design hard-stop alerts that prevent them from completing a prescription order entry, since they report that decision support should not replace clinicians’ responsibility for their patients.[5]

One of the benefits of electronic health records is that they can offer electronic reminders to clinicians when patients need screening tests or treatment. In one study, an electronic reminder system that notified physicians when patients need colorectal cancer tests failed to significantly increase screening rates. [31] In contrast to this study, our findings demonstrate that the effect of an electronic hard-stop alert was very effective at increasing documentation of offering an HIV test, as well as increasing the number of HIV tests performed.

We defined our patient encounters as belonging to three groups. In the first group, the pre-intervention group without the hard-stop alert, the overall compliance rate was 6.3% across all patients. Our post-intervention groups were divided into two separate groups, with or without the hard-stop alert. In group two, the post-intervention group without the hard-stop alert, the compliance rate was 44.5% which was a significant increase over group one. In group three, the post-intervention group with the hard-stop alert, the compliance rate was 94.0% which was a significant and marked increase over the other two groups.

Given that the intervention involved a “hard-stop” alert, providers were required, except in limited circumstances, to offer an HIV test before discharging a patient. It may be argued that the increase in testing rates supports the use of an electronic hard-stop alert for improving HIV screening in the ED. Even when the hard-stop did not appear (i.e., for patients admitted to the hospital from the ED), the rate of offering an HIV test and the rate of HIV testing increased. This suggests that knowledge of, or expectation of an electronic “hard-stop” alert impacted clinicians’ screening practices. The offer for HIV testing often occurs well before the decision to admit or discharge, thus the effect was seen for this group as well.

Age was a significant predictor of testing, with younger patients more likely to be offered a test than older patients. This is not surprising since the HIV epidemic disproportionately affects younger individuals. [32] Gender was a significant predictor for offering an HIV test. Since offering an HIV test was mandatory for all patients, it is not readily apparent why females would more likely be offered an HIV test. One possible explanation is that providers were more accustomed to offering women an HIV test since offering an HIV test to pregnant women has been mandated in many states since the 1990s. [33] Another possibility is that women may be more likely to have other blood tests than men and so providers were more likely to offer women the test since they were already ordering blood work for them. Further research in this area is warranted.

Given the universal mandatory requirement associated with this legislation, a “hard-stop” alert may be one of the few solutions that guarantee compliance. However, further research should explore whether targeted screening would allow for better identification of more HIV-positive patients without imposing unnecessarily on clinicians’ time. Extending the study to analyze how the compliance changes further into the post-intervention time period would be worthwhile to understand if providers remain in compliance months after the “hard-stop” alert has been put into effect. There are also disadvantageous effects of “hard-stop” alerts which can become apparent when using this strategy with more than one policy recommendation.

Despite the success of the hard-stop alert in ensuring compliance with the New York State legislation, it is worth considering the cost. In New York, considerable resources have been dedicated to the implementation of this legislation as well as the ongoing resource of clinician time in completing the necessary documentation. Additional research is needed to understand whether a mandatory offer of HIV testing is cost-effective in the ED, which can inform future policy decisions. Future research is also warranted to assess the effect of the electronic order set and the “hard-stop” alert in detecting HIV-positive patients. Finally, in our analysis, we did not control for time of day, season, staffing and patient volume, all of which may affect the rate of HIV testing.

Limitations of this study include its single setting, which limits the generalizability of the findings. In addition, the electronic data which we analyzed was self-reported by the providers. As such, providers documented that they offered an HIV test to their patients and the patients declined. In some cases, it’s possible that a provider may not have offered the test, but documented having done so in order to avoid the hard-stop alert and discharge a patient. Alternatively, many providers may have offered the HIV test in either pre-intervention period or for admitted patients in the post-intervention period but forgot to document the offer because there was no hard-stop reminder.

Conclusion

Findings from this study demonstrate the usefulness and effectiveness of electronic hard-stop alerts for implementing mandatory screening tests in the ED. The use of a hard-stop alert significantly improved compliance with a state-wide HIV testing law and resulted in a significant increase in the number of HIV tests performed. Interestingly, even when providers did not receive a “hard-stop” alert but had exposure to the intervention, they were more likely to offer their patients an HIV test. This suggests that exposure to “hard-stop” alerts can change practice. Further study is warranted over a longer time period as well as assessment of the use of electronic hard-stop alerts in additional ED settings.