Machine Learning to Predict, Detect, and Intervene Older Adults Vulnerable for Adverse Drug Events in the Emergency Department

Abstract

Adverse drug events (ADEs) are common and have serious consequences in older adults. ED visits are opportunities to identify and alter the course of such vulnerable patients. Current practice, however, is limited by inaccurate reporting of medication list, time-consuming medication reconciliation, and poor ADE assessment. This manuscript describes a novel approach to predict, detect, and intervene vulnerable older adults at risk of ADE using machine learning. Toxicologists’ expertise in ADE is essential to creating the machine learning algorithm. Leveraging the existing electronic health records to better capture older adults at risk of ADE in the ED may improve their care.

Introduction

An adverse drug event (ADE) is an injury resulting from medical intervention related to a drug, which includes medication errors (i.e., inappropriate use of drugs), adverse drug reactions (i.e., harm caused by drugs at normal doses), allergic reactions, and overdoses [1]. In 2008, 1.9 million hospitalizations were recorded due to ADEs. The rate of hospitalization due to ADEs increased 52% from 2004 to 2008 [2]. In patients 65 years old or older, the rate of hospitalization due to ADE is four times higher compared to younger patients [3]. Older adults experience serious consequences from ADE. One in five older adults may fall due to ADE, which results in a fracture, functional decline, and death [4, 5]. Up to 39% of delirium could be caused by ADE, which results in increased hospital length of stay, mortality, and decreased functional status [6, 7]. In addition, 88% of these hospitalizations are considered to be preventable [3, 8]. In the course of morbidity experienced from ADE, individuals are commonly evaluated in the emergency department (ED). These ED visits are significant in older patients—they may signal an inflection point in their functional status and worsening progression of diseases [9].

Although pharmacists have led the initiative to reduce medication errors [10], toxicologists have clinical experience of detecting and treating ADEs. ED visits are opportunities to identify and alter the course of such vulnerable patients. ED visits due to ADE is estimated to occur in four per 1000 individuals in 2013 to 2014, and the rate of these ED visits is increasing from 26 to 35% among older adults between 2005 and 2013 [11]. The Geriatric Emergency Department Guidelines recommend that all geriatric patients have medication reconciliations and identify patients at high risk (polypharmacy > 5 medications and presence of high-risk medications using “Beers criteria”) to review appropriate use of such medications [12]. Current practice, however, is limited by inaccurate reporting of medication list, time-consuming medication reconciliation, and poor ADE assessment. A novel approach to accurately identify and intervene vulnerable older adults at risk of ADE is desperately needed.

One potential mechanism to help identify vulnerable patients due to ADE is machine learning. Machine learning is an approach of artificial intelligence that “trains” mathematical algorithms to reveal potential patterns and relationships (e.g., the presence of ADE and other comorbid conditions) that may not be previously anticipated using existing data. Unlike traditional statistical method with parametric or non-parametric assumptions, machine learning can iteratively “learn” from patterns seen in the data alone [13]. Machine learning approaches use big data stored in the electronic health records (EHRs) to provide a data-driven approach to quickly and reliably identify patients who are at high risk for a clinical outcome (e.g., death), as well as to predict future likelihood of such outcomes [14, 15]. In this manuscript, we will explore the use of machine learning to identify older adults at high risk of ADE in the ED as well as to predict the future ADE.

Advantages of Machine Learning as an Approach to Predict Future Adverse Drug Events

Unlike traditional statistical methods used in medicine based on deductive reasoning (i.e., theory-based hypothesis testing), machine learning relies on inductive reasoning (i.e., recognizing patterns within data to infer associations). Machine learning approaches can account for far greater number of pre-specified variables (e.g., single heart rate reading) embedded within the EHRs compared to a limited number (e.g., age, comorbid conditions) that can be included in traditional statistical analysis approaches. [16, 17] Most widely used machine learning approaches are supervised learning and unsupervised learning. In supervised learning, the mathematical algorithm is trained using known, labeled examples (e.g., age > 90 years) to learn by comparing its actual outcome (e.g., mortality) with correct outputs (e.g., age > 90 years, high predicted mortality) to find errors. In unsupervised learning, no historical labels are used, and the algorithm must figure out what is being shown. The inductive reasoning of machine learning hinges on the availability of robust data (such as that generated from every patient encounter and stored in the EHR) and analytical ability to harness the meanings in available data. Although these techniques have been commonly applied to biological and computer research, the frontier for adapting them to medical research has recently opened.

Traditional statistical methods are insufficient to predict patients with future ADEs [18, 19]. Such methods are limited due to selection bias for choosing the variable of interest (i.e., deductive reasoning), parametric and non-parametric statistical assumptions, and a limited number of variables included in the statistical model. Machine learning can overcome these barriers and predict clinical outcomes more accurately than the traditional statistical methods or experienced physician coders, such as mortality [20, 21], drug-drug interactions [22], and clinical impressions documented by the clinicians (e.g., advance care planning conversations) [23].

Roles of the Toxicologist in Machine Learning and Adverse Drug Events in the ED

Medical toxicologists have growing experience with machine learning [24–27] and stewardship in ADE detection/management [28, 29]. Harnessing the potential of machine learning in the ED may allow better detection, prediction, and possible intervention to address the growing number of ADE-related ED visits for vulnerable older adults. While validated decision criteria like the Beers and Screening Tool of Older Persons’ Potentially Inappropriate Prescriptions (STOPP) have been demonstrated to enhance the detection of ADE, they remain complex and lengthy, making their application in the ED difficult [30]. Machine learning can enhance our understanding of ADE, using the Beers or STOPP criteria as a training point to create an algorithm that identifies patients before the onset of ADE ED visits. Identification of these patients may allow medical toxicologists to concentrate their efforts towards a high-risk population to prevent ADE [31]. Machine learning may also identify effective interventions that prevent ADE while boosting medication adherence through long-term follow-up in identified individuals [32]. An improved understanding of how to leverage both unstructured, text (e.g., clinician progress notes) and structured, non-text (e.g., vital signs, laboratory test results) data within EHR will help by extracting as much information available to detect and predict ADE. Toxicologists may apply their expertise in medication side effects and interactions to create this algorithm for non-toxicologists to use in the ED setting.

Potential Concepts and Preliminary Data to Use Machine Learning for Vulnerable Older Adults in the ED

At present, the potential to leverage the EHR data to detect, predict, and intervene ADE in the ED is understudied. We propose the following line of investigations to fill this knowledge gap for older adults in the ED.

Prediction models with high accuracy can be developed using the structured, non-text data (e.g., vitals, laboratory test results) and established methods like random forest. Random forest is a supervised machine learning approach based on traditional decision tree algorithm. Like decision tree algorithm, the data is categorized into binary splits (e.g., age > 90? yes/no) until certain pre-set criteria is met. Each decision point has an attribute (i.e., the reason to be related to the outcome of interest) and a label (e.g., age > 90? yes/no for mortality prediction). The available data is categorized one attribute after another and assigned a final probability for the outcome of interest. In random forest approach, the data is split randomly and decision tree algorithm is performed iteratively to generate a higher accuracy for the outcome prediction [33]. Random forest approach has been used to predict 6-month mortality for older adults with congestive heart failure (a nationally representative 5% sample of all Medicare fee-for-service beneficiaries, N = 2,000,000). Using traditional predictors (e.g., age, sex, comorbid conditions) augmented by markers of disease progression (e.g., number of medical encounters), this model correctly classified patients who died from patients who lived 82.6% of the time (AUC = 0.826). This was substantially higher than traditional model building (AUC = 0.563) [34]. This study demonstrates the feasibility of predicting ADEs using random forest approach. This type of approach would leverage the entire EHR and yield a much higher accuracy of the prediction model compared to the traditional statistical models.

In addition to using the structured, non-text data in EHR, unstructured, text data (e.g., physician notes) could also be leveraged to better capture likely ADE embedded. Natural language processing is another machine learning approach that allows the computers to analyze the unstructured data (e.g., free-text clinician notes) and learns to interpret human language. It breaks down language into shorter, elemental pieces and tries to learn the relationships between pieces of text [35]. Natural language processing approach has been used to detect patient-reported symptoms from free-text EHR notes. Among clinical notes of 2695 breast cancer patients between 1996 and 2015, 103,564 sentences were analyzed to detect patient-reported symptoms like pain, fatigue, and nausea and compared against a gold standard of physician manual review. The final model achieved precisions of 0.82, 0.86, and 0.99 for an active symptom, the absence of a symptom, and no symptoms at all, respectively under 2 min (> 18,000 times faster than physician coders) [23, 36]. These studies demonstrate the feasibility of detecting difficult to detect symptoms of ADEs using natural language processing.

Upon successful development and validation of above algorithms, implementation into the existing ED workflow could be explored to better understand the acceptability and feasibility of such algorithms to alter the clinical course of vulnerable older adults.

Pitfalls and Future Use of Machine Learning to Predict, Detect, and Improve Adverse Drug Events

Despite the promising ability of machine learning to leverage the data within EHR, multiple limitations exist. Prediction and detection of ADE are only as accurate as the integrity of data entered in the EHR. Intentional or unintentional omission or misclassification by the person who entered the data (e.g., harried physician choosing appropriate diagnostic codes) can greatly alter the integrity, yet such incidences are not readily detectable [37]. Further, the data being used is the product of human decisions (e.g., physician’s decision to obtain diagnostic tests), rather than the biologic phenomenon itself (e.g., empiric mechanism of a drug side effect producing the phenotype of ADE) [38]. A large number of variables are included in the machine learning models which may fit too closely to the data (i.e., overfitting). Such over-fitted model may fail to fit in a different dataset and limit its generalizability. When a machine learning model is created, implementation of this model into the EHR in real-time clinical practice will be an additional barrier to its usefulness. The machine learning algorithms may not surpass the human judgment; rather, they could be a powerful tool to circumvent when correctly applied by trained clinicians. Finally, machine learning algorithms are designed to discover meaning within data (i.e., associations) and may not be interpreted as causality (i.e., predictors may not cause ADEs).

In the future, integration of other sources of data that may predict ADE may enhance the predictive strength of a machine learning algorithm. For example, investigations that integrate wearable biosensor data which has previously identified opioid use may be combined with EHR prescribing data to identify the onset of ADE in this population, prompting intervention by a medical toxicologist [39, 40].

Conclusion

Machine learning approaches can be an innovative method to predict, detect, and intervene adverse drug events in the ED. Toxicologists’ expertise in adverse drug events is essential to creating the machine learning algorithm. Leveraging the existing electronic health records to better capture older adults at risk of adverse drug events in the ED may improve their care.

Sources of Funding

Dr. Boyer is supported by the National Institutes of Health 1K24DA037109. Dr. Ouchi is supported by the Grants for Early Medical and Surgical Subspecialists’ Transition to Aging Research award from the National Institute on Aging (1R03AG056449), the Emergency Medicine Foundation (EMF), and the Society of Academic Emergency Medicine (SAEM).

Compliance with Ethical Standards

Conflict of Interest

The authors declare there are no additional conflicts of interest.