Abstract
Current clinical data repositories primarily extract data from multiple administrative and electronic medical record (EMR) data resources (e.g., hospital and physician billing records) containing specific patient-level data including demographics, medications, laboratory results, diagnoses, and procedure codes. It overlooks the importance of EMR system-level data (e.g., medication alerts that are routinely used by physicians, nurses, and pharmacists for decision support) for the surveillance of EMR decision support tools. These medication alerts are a significant source of information for providers, to minimize avoidable adverse drug events. This study describes the integration of medication alert data into an i2b2-based clinical data repository to support the investigation of clinical events occurring around patients with anticoagulation treatment that triggered drug-drug interaction alerts. The integration of medication alerts allows us to repurpose the clinical and translational research infrastructure to conduct retrospective effectiveness surveillance of clinical decision support tools.
Introduction
Clinical care data are increasingly captured electronically and effort in making these data available for scientific research has received widespread support1-3. The NIH has funded the Informatics for Integrating Biology and the Bedside (i2b2) National Center for Biomedical Computing to provide an open-source clinical data informatics framework to facilitate clinical data reuse and integration4. Since its first release in 2007, over 240 scholarly articles have been published using data derived from i2b2-based repositories5.
At the University of Kansas Medical Center (KUMC), we established an i2b2 based Healthcare Enterprise Repository for Ontological Narration (HERON) in 2010 that integrated information “siloed” in disparate information systems with available data types including patient demographics, medication, laboratory results, and diagnoses data. While most clinical integrated data repositories are created and used to support clinical trial recruitment, observational analysis of healthcare utilization and cost, and post-marketing surveillance of medications, we believe that integrating data about the use of electronic medical record (EMR) systems and the decision support provided will enable repurposing of the clinical and translational research infrastructure for conducting effectiveness surveillance of clinical decision support tools analogous to post-marketing medication safety surveillance.
Several reviews have found that computerized clinical decision support tools can improve healthcare providers’ performance6-8 and substantially reduce medication error rates9. However, few studies have observed any significant benefits on patient outcomes8, perhaps due to small sample size or short length of time for any clinically important effects to be revealed. Decision support tools may include passive and active referential information as well as reminders, alerts, and guidelines, among others. Since medications are commonly used for therapy but can also cause harm such as adverse drug reactions, various medication-related decision support tools have been implemented to improve medication safety and efficiency, which include drug-allergy checking, dose guidance, and drug-drug interaction checking.
In this study, we propose to enrich our i2b2-based HERON repository with medication alert data to facilitate retrospective effectiveness studies of EMR decision support tools such as drug-drug interaction alerts. Thus, cohorts can be defined based on the patient providers’ alert exposure and analyzed to gain understanding of how alert response impacts patient outcomes as well as descriptive statistics on the makeup of the patients whose provider received the interventions (e.g., most common alerts and diagnoses).
The national Patient-Centered Clinical Research Network (PCORnet) work is heightening integration of data resources in a standard manner. KUMC has a long tradition of incorporating rich data into i2b2-based clinical repositories. We previously published our work on incorporating observations from nursing flowsheets10, the University of Kansas Hospital Tumor Registry data dating back to the 1950s (https://informatics.kumc.edu/work/wiki/TumorRegistry), the University HealthSystem Consortium (UHC) data regarding inpatient hospitalizations11, REDCap patient registries12, cardiovascular test results (https://informatics.kumc.edu/work/blog/heron-marmaton-update), and microbiology laboratory results including the organisms and their antimicrobial resistance13. We believe that adding EMR usage data will be equally important over time, due to the increasing focus on healthcare outcomes. For this pilot study, we start with a case study on the most classic form of alerting in EMR, drug-drug interaction alerts, specifically focused on common alerts for combination of anticoagulant and other medications that heighten the risk of bleeding.
Methods
Enriching i2b2-based Data Repository with Medication Alerts
KUMC’s affiliated clinical organizations adopted the Epic Systems Corporation (“Epic”) as its EMR in 2007 with initial deployment within the hospital and has expanded over time to encompass the emergency department, outpatient cancer center treatment, and is currently being deployed across all ambulatory clinics. Data from the Epic EMR were obtained by exporting the Clarity database. The Clarity module transforms data from Epic’s operational database (Intersystems Cache®) into a relational form (Oracle® 10g Release 2) for reporting. Clarity stores patient and system configuration information in over 7,000 tables with over 60,000 columns. The extracted data is transformed into an i2b2 compatible star schema, de-identified, and loaded on a separate database server to be accessed by the i2b2 application. These Extract, Transform, and Load (ETL) processes are written in Structured Query Language (SQL) statements and the Python programming language. The resulting data used by investigators and analyzed in this study is deemed non-human subjects research by the KUMC institutional review board but data use requires approval by the HERON Data Request Oversight Committee composed of representatives from KUMC and participating clinical organizations.
As a result, 22,927,205 alert observations for 474,035 patients were loaded from Clarity into our i2b2-based HERON data repository. As shown below (Figure 1), we used hierarchical relationship to organize the alerts in the i2b2 framework with alert status (i.e., actions taken) as modifiers. Specifically, the drug-drug interaction alerts are organized in i2b2 as: Alerts → Med Interaction Checking → Drug-Drug → Alert descriptions in alphabetic order.
Figure 1:
Medication interaction alert data represented in i2b2.
Case Study: Effectiveness of Anticoagulant Drug-Drug Interaction Alerts
As a case study, we will focus on a common form of alerting, i.e., drug-drug interaction alerts involving combination of anticoagulant and other medications that heighten the risk of bleeding. This combination of drug- drug interaction, alert, laboratory test, and bleeding outcome provides a specific timeline in clinical practice that can be followed and evaluated rather than a population-based inference through number of alerts and number of outcomes. We will use the enriched i2b2 clinical data repository to obtain data and analyze the clinical events (e.g., INR lab ordering and bleeding events) occurring around hospitalized patients who had anticoagulant drug-drug interaction alerts fired during their stay.
Warfarin is a commonly prescribed oral anticoagulant with a narrow window for safe and effective use. It requires regular laboratory testing to ensure the warfarin dose is appropriate, and a number of foods and medications can have a significant impact on warfarin effectiveness. The addition of new medications to the regimen of a patient who is also using warfarin can require a proactive adjustment of the warfarin dose to maintain effectiveness, or the patient may need additional monitoring to limit the risk of potentially fatal bleeding events. For example, one interacting medication which may require a warfarin dose adjustment is allopurinol, used to treat gout. Titration of allopurinol may be required in order to relieve the symptoms of gout, but concurrent use of allopurinol with warfarin can increase the risk of bleeding.
A patient who is stable on warfarin may be admitted to the hospital for an acute gout attack which requires allopurinol. The attending physician orders allopurinol, and is presented with a drug-drug interaction alert informing her that allopurinol interacts with warfarin and increases the risk of bleeding. The physician has the option of continuing both medications, or discontinuing one. Because the patient is in the hospital, and the gout attack is acute, the physician may feel the risk of bleeding is relatively small and both medications may be continued. While the patient is in the hospital and medication therapy is changing, a physician presented with the drug-drug interaction alert may accept the risk of bleeding while the patient is taking both medications, but will attempt to mitigate that risk by ordering International Normalized Ratio (INR) tests to ensure the patient’s warfarin dose is in the appropriate range to minimize the risk of bleeding.
Drug-drug interaction alerts can produce a number of actions, either from the system or the providers. The system may be designed to filter alerts to reduce alert fatigue for physicians and nurses and in such cases, a filtered alert may only present to pharmacists, even though it was originally triggered by the physician’s actions. The pharmacist, or if the alert is presented to the physician, will be given a number of action options. They may only view the alert, but close it without choosing an action, override the alert indicating they have seen it but will not act upon the information, cancel the alert, remove one of the interacting medications, or hold a medication for a period of time.
Patient Cohort:
We constructed a cohort of 3,506 adult patients (18 < age < 64) who were admitted to KUMC hospital from Q4, 2008 to Q4, 2015 with a length of stay ≥ 2 days and have at least one anticoagulant drug-drug interaction alert fired during their stay. For the purpose of this pilot case to evaluate integrated data, patients over 65 years of age are excluded to reduce confounders as they are more likely to have comorbid conditions and polypharmacy which can increase the risk of a patient having bleeding event. Then for each hospitalization encounter, we extracted data on INR lab ordering for the study patients to assess if INR tests were conducted as a result of the drug-drug interaction alert and any diagnostic codes suggesting bleeding complication. Bleeding events in this study are identified in HERON through the use of ICD-9 and E-codes for bleeds that can be caused or exacerbated by medications (Table 1).
Table 1:
Bleeding related ICD-9 codes used in analysis
ICD-9 | Description | ICD-9 | Description |
---|---|---|---|
285.1 | Acute posthemorrhagic anemia | 535.61 | Duodenitis, with hemorrhage |
286.5 | Hemorrhagic disorder due to intrinsic circulating anticoagulants, antibodies, or inhibitors | 562.02 | Diverticulosis of small intestine with hemorrhage |
286.7 | Acquired coagulation factor deficiency | 562.03 | Diverticulitis of small intestine with hemorrhage |
286.9 | Other and unspecified coagulation defects | 562.12 | Diverticulosis of colon with hemorrhage |
430 | Subarachnoid hemorrhage | 562.13 | Diverticulitis of colon with hemorrhage |
431 | Intracerebral hemorrhage | 568.81 | Hemoperitoneum (nontraumatic) |
432 | Other and unspecified intracranial hemorrhage | 569.85 | Angiodysplasia of intestine with hemorrhage |
456.0 | Esophageal varices with bleeding | 578.9 | Hemorrhage of gastrointestinal tract, unspecified |
456.20 | Esophageal varices in diseases classified elsewhere, with bleeding | 578.0 | Hematemesis |
459.0 | Hemorrhage, unspecified | 596.7 | Hemorrhage into bladder wall |
530.21 | Ulcer of esophagus with bleeding | 626.2 | Excessive or frequent menstruation |
531.0 | Acute gastric ulcer with hemorrhage | 782.7 | Spontaneous ecchymoses |
531.2 | Acute gastric ulcer with hemorrhage and perforation | 784.7 | Epistaxis |
531.4 | Chronic or unspecified gastric ulcer with hemorrhage | 784.8 | Hemorrhage from throat |
531.6 | Chronic or unspecified gastric ulcer with hemorrhage and perforation | 785.5 | Shock without mention of trauma |
532 | Duodenal ulcer | 786.3 | Hemoptysis |
533.2 | Acute peptic ulcer of unspecified site with hemorrhage and perforation | 920 | Contusion of face, scalp, and neck except eye(s) |
533.4 | Chronic or unspecified peptic ulcer of unspecified site with hemorrhage | 921 | Contusion of eye and adnexa |
533.6 | Chronic or unspecified peptic ulcer of unspecified site with hemorrhage and perforation | 922 | Contusion of trunk |
534.0 | Acute gastrojejunal ulcer with hemorrhage | 923 | Contusion of upper limb |
534.2 | Acute gastrojejunal ulcer with hemorrhage and perforation | 924 | Contusion of lower limb and of other and unspecified sites |
534.4 | Chronic or unspecified gastrojejunal ulcer with hemorrhage | 964.2 | Poisoning by anticoagulants |
534.6 | Chronic or unspecified gastrojejunal ulcer with hemorrhage and perforation | 998.1 | Hemorrhage or hematoma complicating a procedure |
535.11 | Atrophic gastritis, with hemorrhage | 578.1 | Blood in stool |
535.21 | Gastric mucosal hypertrophy, with hemorrhage | 790.92 | Abnormal coagulation profile |
535.31 | Alcoholic gastritis, with hemorrhage | E858.2 | Accidental poisoning by agents primarily affecting blood constituents |
535.41 | Other specified gastritis, with hemorrhage | E934.2 | Anticoagulants causing adverse effects in therapeutic use |
535.51 | Unspecified gastritis and gastroduodenitis, with hemorrhage |
Results
Descriptive Statistics of the Population
Table 2 shows the basic demographics of the study patient cohort (3,506 patients with 6,183 hospital encounters). The study cohort contains more female patients than male patients who satisfied the inclusion criteria described above. Nearly half of the patients in the cohort were over 55 years old.
Table 2:
Patient demographics
Demographic | # of Patients = N (%) | |
---|---|---|
Gender | Male | 1506 (43.95%) |
Female | 2000 (57.05%) | |
Race | White | 2593 (73.96%) |
Black | 585 (16.69%) | |
Asian | 14 (0.40%) | |
American Indian | 13 (0.37%) | |
Pacific Islander | 1 (0.02%) | |
Other | 300 (8.56%) |
From the study population, there are 317 distinct admission diagnoses categorized using the All Patient Refined DRG classification system. Table 3 lists the top 10 admission diagnoses or APR DRGs of the study patients, which were tabulated at the encounter level.
Table 3:
Top 10 All Patient Refined DRG diagnoses assigned for the hospital encounters with anticoagulant drug- drug (DDI) interaction alerts triggered
ALL PATIENT REFINED DRG | UNIQUE ENCOUNTERS WITH ANTICOAGULANT DDI ALERTS FIRED |
---|---|
SEPTICEMIA & DISSEM INFECT | 412 |
CARD ARRHYTHMIA & CONDUCTION | 331 |
HEART FAILURE | 244 |
CARDIAC VALVE PROC W/0 CATH | 186 |
PULMONARY EMBOLISM | 167 |
REHABILITATION | 161 |
PERCUT CARDIOVASC W/O AMI | 152 |
KNEE JOINT REPLACEMENT | 151 |
CARD CATHET EXC ISCHEMIA | 132 |
PERIPHERAL & OTH VASC DIS | 122 |
Descriptive Statistics of the Anticoagulant Interaction Alerts
In this study, there are 37 different drug-drug interaction alerts involving warfarin or other anticoagulants. Table 4 summarizes the top 10 warfarin/anticoagulants related interaction alerts fired during the study cohort’s hospital stay. Multiple alerts can be triggered during a single stay.
Table 4:
Top 10 anticoagulant interaction alerts triggered for the study inpatients
ALERT DESCRIPTION | UNIQUE ALERTS |
---|---|
ANTICOAGULANTS / SALICYLATES | 6293 |
ANTICOAGULANTS / CORTICOSTEROIDS | 6091 |
ANTICOAGULANTS / SELECTED PENICILLINS | 2706 |
ANTICOAGULANTS / THYROID | 2559 |
ANTICOAGULANTS / NSAIDS | 1613 |
ANTICOAGULANTS / CEPHALOSPORINS, INJECTABLE | 1290 |
ANTICOAGULANTS / ALLOPURINOL | 818 |
ANTICOAGULANTS / CEFAZOLIN; CEFOXITIN; CEFTRIAXONE | 772 |
ANTICOAGULANTS / VITAMIN K | 658 |
ANTICOAGULANTS / METRONIDAZOLE; TINIDAZOLE | 627 |
Among the 3,506 study patients, there were 6,183 hospitalization encounters with length of stay ≥ 2 days and a total of 26,270 unique warfarin/anticoagulants interaction alerts generated. For the 26,270 interaction alerts, we analyzed the recorded actions taken for the alerts, namely override, remove, filter, cancel, view, or hold (Table 5). In addition, as shown in Table 5, we examined the number of patients who had INR test result immediately within 1 day of the alert trigger vs. no INR result, and the number of patients who had INR result within 1 day of the alert but had bleeding events within 30 days of their hospital discharge.
Table 5:
Statistics on clinical events following alerts according to different actions taken for the alert
Overridden | Removed | Filtered | Canceled | Viewed | Held | |
---|---|---|---|---|---|---|
Alerts fired during hospital stay | 10933 | 190 | 21830 | 2282 | 1609 | 15 |
Alert → INR (≤ 1 day) | 660 | 5 | 1533 | 157 | 120 | 0 |
Alert → NO INR (≤ 1 day) | 10167 | 185 | 20110 | 2111 | 1485 | 15 |
Alert → INR (≤ 1 day) → Bleeding (≤ 30 day) | 9 | 0 | 38 | 2 | 1 | 0 |
Furthermore, using the i2b2 query analysis tool, we were able to visualize patients’ clinical events on a timeline as shown in Figure 2 below. For example, the second patient (red box) in Figure 2 is a 56 year old white female, who was admitted to the hospital around September, 2014 for a major small & large bowel procedure. During her stay, the patient had three interaction alerts and one INR test result Various interaction alerts fired around the time of the alerts.
Figure 2:
Study patient timeline of clinical events. Note: the i2b2 query analysis tool shows all occurrences of clinical events included in the query on a timeline; thus, alerts fired in all encounters (not only the ones fired during hospitalization) are also shown here.
Discussion
This study was a pilot to enrich an i2b2-based clinical data repository with medication alert data to enable clinical effectiveness studies of EMR decision support tools such as drug-drug interaction alerts. Through the expansion, cohorts can be defined based on the patient providers’ alert exposure and analyzed to understand provider actions which occur in response to drug-drug interaction alerts.
Drug-drug interactions are common in the inpatient setting. Medication alerts are an option for preventing medication-related adverse events by raising prescriber awareness, but hospitals often struggle with the alerts being ignored or overridden 9, 14, 15. Studies have found override rates for high-severity drug-drug interactions can be as high as 88%16, 17. Previous work on alerts and overrides were generally limited to order analysis or prescriber questionnaires/focus groups, which have limitations in both context and recall15. Integrating alert actions with other clinical information within i2b2-based clinical data repositories may provide additional insight into provider decision-making to facilitate observational study and design of more effective medication-related alerts.
Oral anticoagulants are a class of medications which is commonly associated with drug-drug interaction alerts due to the number of interacting medications and the potentially serious adverse events associated with the drug interactions18. In our study, we identified approximately four medication alerts for each visit related to anticoagulants. Anticoagulants commonly interact with antibiotics, which were further reflected in the single most common DRG code for septicemia and infection. Additionally, oral anticoagulants are typically used to manage cardiac conditions and blood clots, which were reflected in the most frequent DRG codes associated with the drug- drug interactions, including cardiac arrhythmias, heart failure, and pulmonary embolisms. Furthermore, these conditions are frequently associated with age. One study found the average age of warfarin users was 7119 and this was demonstrated in the age distribution of the patients identified in our study, where nearly half the patients in the cohort (18-64 years old) were over 55.
Despite the potentially fatal adverse drug events which can occur when medications interact with anticoagulants, only minority of patients (660 out of 3,506 = 18.8%) who had a drug-drug interaction alert presented during their hospital stay had an INR test result within a day of the alert, which may be partially due to the delay in obtaining INR test results from the order time. Among our cohort of patients, most patients (3,454 out of 3,506 = 98.5%) had an INR test result during their stay, regardless of when a drug-drug alert appeared. This may suggest that INR tests are likely part of usual care and as a result, are not ordered in response to drug-drug interactions, but are ordered regularly in an attempt to proactively monitor for potential adverse drug events regardless of the concurrent medications being used. Drug-drug interaction alerts are provided to physicians and other healthcare professionals to aid clinical decision making, but in the case of oral anticoagulants, the necessary monitoring may be occurring routinely during the course of regular care, and as a result, presentation of drug-drug interaction alerts for these medications may not aid clinical decision making, and may contribute to alert fatigue without improving patient safety.
Limitations: There are several limitations to this study. First, many of the identified drug-drug interaction alerts are for classes of medications rather than individual agents. In the case of anticoagulants, this type of class-level interaction alert may be presented for medication combinations which do not have a true interaction and do not alter the patient’s risk of a bleeding event. Second, we only looked at adults who were under 65 years of age, and do not capture prescribing actions which may occur for older adult patients that could alter the findings. Patients 65 years of age and older have increased likelihood of being on multiple medications or have multiple conditions that would increase bleeding risk. Therefore, in future work, we plan to consider the older population independently from the younger population in analysis. Third, the data collected do not capture information about which medication was being prescribed to produce the alert. A patient may be starting warfarin at the end of a course of antibiotics, which clinically may not pose a risk to a patient because one medication is ending. Fourth, it is not clear how many alerts were filtered so they were only presented to pharmacists, or how many providers may have viewed a specific alert for a specific patient and what the overall action of that group effort was.
Future work: With the medication alert data integrated into an i2b2-based clinical data repository, we will be able to explore many questions regarding the effectiveness of the EMR decision support tools. For instance, one question may be comparing filtered alerts vs. non-filtered ones, and for those filtered alerts, whether pharmacists provide any valuable insight. We can also analyze how effective the alerts are in reducing bleeding events or changing the INR ordering pattern. Last but not least, one can quantify the incidence of bleeding for the various drug interaction responses.
Conclusion
This study describes a pilot to integrate medication alert data and provider actions that occur in response into an i2b2-based clinical data repository to allow repurpose of the clinical and translational research infrastructure to conduct surveillance of EMR effectiveness. We started with the most classic form of alerting in EMR, drug-drug interaction alerts, specifically focused on common alerts for combination of warfarin or anticoagulant and other medications that heighten the risk of bleeding. We showed that the enriched i2b2 framework can facilitate the investigation of clinical events occurring around patients with anticoagulation treatment that triggered the interaction alerts. It has the potential to be a powerful tool in supporting various retrospective effectiveness studies of EMR and the embedded decision support tools.
Acknowledgement
The HERON data repository described in the paper is supported by institutional funding from the University of Kansas Medical Center and CTSA grant #UL1TR000001 from NCATS/NIH. The work is partially supported by PCORI CDRN-1306-04631. Special thanks to the HERON development team for the ETL work.
References
- 1.Prokosch HU, Ganslandt T. Perspectives for medical informatics. Reusing the electronic medical record for clinical research. Methods of information in medicine. 2009;48(1):38–44. [PubMed] [Google Scholar]
- 2.Ohmann C, Kuchinke W. Future developments of medical informatics from the viewpoint of networked clinical research. Interoperability and integration. Methods of information in medicine. 2009;48(1):45–54. [PubMed] [Google Scholar]
- 3.Kush R, Alschuler L, Ruggeri R, et al. Implementing Single Source: the STARBRITE proof-of-concept study. J Am Med Inform Assoc. 2007 Sep-Oct;14(5):662–73. doi: 10.1197/jamia.M2157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Murphy SN, Weber G, Mendis M, et al. Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2) J Am Med Inform Assoc. 2010 Mar-Apr;17(2):124–30. doi: 10.1136/jamia.2009.000893. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Informatics for Integrating Biology to the Bedside. [cited 2016 September]; Available from: https://www.i2b2.org/about/index.html.
- 6.Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998 Oct 21;280(15):1339–46. doi: 10.1001/jama.280.15.1339. [DOI] [PubMed] [Google Scholar]
- 7.Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research; Annals of internal medicine; 1994. Jan 15, pp. 135–42. [DOI] [PubMed] [Google Scholar]
- 8.Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011 May 1;18(3):327–34. doi: 10.1136/amiajnl-2011-000094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review; Arch Intern Med.; 2003. Jun 23, pp. 1409–16. [DOI] [PubMed] [Google Scholar]
- 10.Waitman LR, Warren JJ, Manos EL, Connolly DW. Expressing observations from electronic medical record flowsheets in an i2b2 based clinical data repository to support research and quality improvement; AMIA Annu Symp Proc.; 2011. pp. 1454–63. [PMC free article] [PubMed] [Google Scholar]
- 11.Nair M, Connolly DW, Whittkopp C, Waitman LR. Combining Quality and Accountability Study Metrics from University Health System Consortium in an i2b2 based Clinical Data Repository with EMR data for Patient Centered Outcome Research (PCOR); AMIA Clinical Research Informatics Joint Summit; San Francisco, CA. 2013. [Google Scholar]
- 12.Adagarla B, Connolly DW, Nair M, Sharma P, Waitman LR. Integrating REDCap Patient Registries within an i2b2 Integrated Data Repository; AMIA Clinical Research Informatics Joint Summit; San Francisco, CA. 2013. [Google Scholar]
- 13.Chandaka S, Horvat RT, Atrouni WE, Antoun M, Hoag MC, Waitman LR. Representing Microbiology Laboratory Results, Including Susceptibility and Sensitivity Findings with i2b2 Compatible Ontologies; AMIA Clinical Research Informatics Joint Summit; San Francisco, CA. 2015. [Google Scholar]
- 14.Heringa M, Floor-Schreudering A, Tromp PC, de Smet PA, Bouvy ML. Nature and frequency of drug therapy alerts generated by clinical decision support in community pharmacy; Pharmacoepidemiol Drug Saf.; 2016. Jan, pp. 82–9. [DOI] [PubMed] [Google Scholar]
- 15.van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006 Mar-Apr;13(2):138–47. doi: 10.1197/jamia.M1809. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Payne TH, Nichol WP, Hoey P, Savarino J. Characteristics and override rates of order checks in a practitioner order entry system; Proceedings / AMIA Annual Symposium AMIA Symposium.; 2002. pp. 602–6. [PMC free article] [PubMed] [Google Scholar]
- 17.Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care; Arch Intern Med.; 2003. Nov 24, pp. 2625–31. [DOI] [PubMed] [Google Scholar]
- 18.Pirmohamed M. Warfarin: almost 60 years old and still causing problems; Br J Clin Pharmacol.; 2006. Nov, pp. 509–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Fang MC, Go AS, Hylek EM, et al. Age and the risk of warfarin-associated hemorrhage: the anticoagulation and risk factors in atrial fibrillation study. Journal of the American Geriatrics Society. 2006 Aug;54(8):1231–6. doi: 10.1111/j.1532-5415.2006.00828.x. [DOI] [PMC free article] [PubMed] [Google Scholar]