A Review of Challenges and Opportunities in Machine Learning for Health

Abstract

Modern electronic health records (EHRs) provide data to answer clinically meaningful questions. The growing data in EHRs makes healthcare ripe for the use of machine learning. However, learning in a clinical setting presents unique challenges that complicate the use of common machine learning methodologies. For example, diseases in EHRs are poorly labeled, conditions can encompass multiple underlying endotypes, and healthy individuals are underrepresented. This article serves as a primer to illuminate these challenges and highlights opportunities for members of the machine learning community to contribute to healthcare.

Introduction

Health problems impact human lives. During medical care, health providers collect clinical data about each particular patient, and leverage knowledge from the general population, to determine how to treat that patient. Data thus plays a fundamental role in addressing health problems, and improved information is crucial to improving patient care.

Health problems impact human lives. During medical care, health providers collect clinical data about each particular patient, and leverage knowledge from the general population, to determine how to treat that patient. Data thus plays a fundamental role in addressing health problems, and improved information is crucial to improving patient care.

Using data, machine learning has driven advances in many domains including computer vision, natural language processing (NLP), and automatic speech recognition (ASR) to deliver powerful systems (e.g., driverless cars, voice activated personal assistants, automated translation). Machine learning’s ability to extract information from data, paired with the centrality of data in healthcare, makes research in machine learning for healthcare crucial.

Interest in machine learning for healthcare has grown immensely, including work in diagnosing diabetic retinopathy¹, detecting lymph node metastases from breast pathology², autism subtyping by clustering comorbidities³, and large- scale phenotyping from observational data⁴. Despite these advances, the direct application of machine learning to healthcare remains fraught with pitfalls. Many of these challenges stem from the nominal goal in healthcare to make personalized predictions using data generated and managed via the medical system, where data collection’s primary purpose is to support care, rather than facilitate subsequent analysis.

Existing reviews of machine learning in the medical space have focused narrowly on biomedical applications⁵, deep learning tasks well suited for healthcare⁶, the need for transparency⁷, and use of big data in precision medicine⁸. Here, we emphasize the broad opportunities present in machine learning for healthcare and the careful considerations that must be made. We focus on the electronic health record (EHR), which documents the process of healthcare delivery and operational needs such as tracking care and revenue cycle management (i.e., billing and payments). While we choose to focus on the inpatient setting as the majority of machine learning projects currently focus on this data-rich environment, we note that clinical data is heterogeneous, and comes in a variety of forms that can be relevant to understanding patient health⁹.

In this work, we cover the unique technical challenges that should be considered in machine learning systems for healthcare tasks, especially as performance between trained models and human experts narrows¹⁰. Failure to carefully consider these challenges can hinder the validity and utility of machine learning for healthcare. We present a hierarchy of clinical opportunities, organized into the following general categories: automating clinical tasks, providing clinical support, and expanding clinical capacities. We conclude by outlining the opportunities for research in machine learning that have particular relevance in healthcare: accommodating shifts in data sources and mechanisms, ensuring models are interpretable, and identifying good representations.

Unique Technical Challenges in Healthcare Tasks

In tackling healthcare tasks, there are factors that should be considered carefully in the design and evaluation of machine learning projects: causality, missingness, and outcome definition. These considerations are important across both modeling frameworks (e.g., supervised vs. unsupervised), and learning targets (e.g., classification vs. regression).

Understanding Causality is Key

Many of the most important and exciting problems in healthcare require algorithms that can answer causal, “what if?” questions about what will happen if a doctor administers a treatment¹¹. These questions are beyond the reach of classical machine learning algorithms because they require a formal model of interventions. To address this class of problems, we need to reason about and learn from data through the lens of causal models (e.g., see prior work^12,13). Learning from data to answer causal questions is most challenging when the data are collected observationally; that is, it may have been influenced by the actions of an agent whose policy for choosing actions is not known.

In healthcare, learning is done almost exclusively using observational data, which poses a number of challenges to building models that can answer causal questions. For instance, Simpson’s paradox describes the observation that the relationship between two variables can change directions if more information is included in the model¹². To better understand this issue, consider prior work in which researchers found that asthmatic patients who were admitted to the hospital for pneumonia were more aggressively treated for the infection, lowering the subpopulation mortality rate¹⁴. A model that predicts death from asthma will learn that asthma is protective. If, however, an additional variable to account for the level of care is included, the model may instead find that having asthma increases the risk of death. This example demonstrates that causal models are not only useful to evaluate treatments, but can also help to build reliable predictive models that do not make harmful predictions using relationships caused by treatment policies in the training data¹¹. To account for these challenges, strong assumptions must be made that cannot be statistically checked or validated; i.e., gathering more data will not help¹⁵. The shortcomings of classical statistical learning when answering causal questions are discussed in greater detail in prior work¹⁶.

Models in Health Must Consider Missingness

Even if all important variables are included in a healthcare dataset, it is likely that many observations will be missing. Truly complete data is often impractical due to cost and volume. Learning from incomplete, or missing, data has received little attention in the machine learning community (although exceptions exist^17-19), but is an actively studied topic in statistics²⁰. Because healthcare is a dynamic process where vitals are measured and labs are ordered over time by doctors in response to previous observations there are strong dependencies between what variables are measured and their values, which must be carefully accounted for to avoid biased results (e.g. see prior work²¹ Appendix B).

There are three widely accepted classifications of missing data mechanisms; i.e., the measurement mechanism determining whether a value is recorded or not²². The first, missing completely at random (MCAR), posits a fixed probability of missingness. In this case, dropping incomplete observations—known as complete case analysis—is commonly used (albeit naively), and will lead to unbiased results. Second, the data may be missing at random (MAR), where the probability of missingness is random conditional on the observed variables. In this case, common methods include re-weighting data with methods like inverse probability of censoring weighting or using multiple imputations to in-fill^23,24. Finally, data may be missing not at random (MNAR), where the probability of missingness depends on the missing variable itself, or other missing and unobserved variables.

Sources of missingness must be carefully understood. Sources of missingness should be carefully examined before deploying a learning algorithm. For example, lab measurements are typically ordered as part of a diagnostic work-up, meaning that the presence of a datapoint conveys information about the patient’s state. Consider a hospital where clinical staff measures patient lactate level. If a power outage led to a set of lactate levels being lost, the data are MCAR. If nurses are less likely to measure lactate levels in patients with traumatic injury, and we record whether patients were admitted with trauma, the data are MAR. However, if nurses are less likely to measure lactate levels when believed to be already, then the lactate measures themselves are MNAR, and the measurement of the signal itself is meaningful. The key feature of missing data is that there may be information conveyed by the absence of an observation, and ignoring this dependence may lead to models that make incorrect, and even harmful, predictions.

Include missingness in the model. Including missingness indicators provides the most information for making predictions²⁵. However, learning models without an appropriate model of missingness leads to issues such as inaccurate assessment of feature importance and models that are brittle to changes in measurement practices. For example, troponin-T is commonly measured only when a myocardial infarction is considered probable. A model learned by treating troponin-T as MCAR would likely overpredict the rate of myocardial infarction on data where troponin-T was more regularly measured. A model trained with MAR troponin values would be more robust to this.

Missingness can reflect human biases. We note that data may also be missing because of differences in access, practice, or recording that reflects societal biases^26-27. Models trained on such data may in turn exhibit unfair performance for some populations if a machine learning practitioner is unaware of this underlying variation; thus, checking models across groups is important²⁸.

Make Careful Choices in Defining Outcomes

Obtaining reliable outcomes for learning is an important step in defining tasks. Outcomes are often used to create the gold-standard labels needed for supervised prediction tasks, but are crucial in other settings as well, e.g., to ensure well-defined cohorts in a clustering task. There are three key factors to consider with outcome definitions: creating reliable outcomes, understanding the relevance of an outcome clinically, and the subtlety of label leakage.

Create reliable outcomes from heterogeneous source data. Multiple data sources should be considered when creating labels because EHRs often lack precise structured labels. Or, in some cases, structured labels may unreliable²⁹. For example, a diagnostic clinical code for pneumonia could mean a patient was screened for pneumonia rather than that they actually had pneumonia. Machine learning methods to pool different data types and obtain a more reliable label is known as phenotyping30, and is an important subfield of machine learning in healthcare^31,32. Recent work has emphasized the need to integrate the rich information available in clinical notes, and building natural language processing pipelines to extract information from unstructured clinical text accurately is an active subject of research³³.

Understand the outcome in context of a healthcare system. Medical definitions are working models using current scientific understanding of a disease. As understanding evolves, so does the definition. The implication for machine learning is that good predictive performance on labels based on such definitions is only as good as the underlying criteria. For example, acute kidney injury (AKI) is an important critical illness with two recent definitions: RIFLE³⁴ and KDIGO³⁵. Similarly, it is tempting to use the actual behavior of clinicians as labels, but it is important to remember that they may not be correct. For example, work that targets prediction of clinical actions must carefully consider whether the treatments are good labels, and whether “incorrect predictions” are in fact forecasting treatments that would have been given by other clinicians rather than treatments that would optimally treat the patient^36-38.

Beware of label leakage. The information collected in an individual’s hospital encounter is tightly coupled across time, and this can result in information about the targeted task outcome leaking back into features. While exploiting such relationships between features and targets is a goal of learning, information leakage can render a prediction meaningless. Consider predicting mortality of hospital patients using all available data up until their time of death. Such a task could lead to a pathological prediction rule—”if the ventilator is turned off in the preceding hour, predict death.” This commonly happens when patients and their families decide to withdraw care at a terminal stage of illness. A machine learning algorithm trained naively on this signal would have high predictive performance by nearly any metric, yet absolutely no clinical utility. High capacity neural models examining the entire EHR may also be subject to label leakage³⁹; for example, a final diagnostic label could leverage labels that appear within clinicians’ initial hypotheses, confirming clinical consistency in hypotheses, rather than predicting a diagnosis based on relevant data.

Addressing a Hierarchy of Healthcare Opportunities

There are many high-impact opportunities in healthcare. Before fitting models, goals should be clearly identified and validated as worth solving. Here, we frame potential healthcare opportunities into three high-level categories: automating clinical tasks, providing clinical support, and expanding clinical capacities. The details of how a technical solution is deployed can change its intent, and it is therefore crucial to engage clinical stakeholders early on.

Clinical Task Automation: Automating clinical tasks during diagnosis and treatment

There are many tasks currently performed by clinicians that present low-hanging fruit for machine learning researchers. Clinical task automation encompasses a class of work that clinicians currently do. These tasks are well- defined (i.e., known input and output spaces), and thus require the least amount of domain adaptation and investment. The evaluation of task replacement is also straightforward—performance can be measured against existing standards. We emphasize that algorithms should not replace clinical staff, but rather be used to optimize the clinical workflow. Clinical roles will likely evolve as these techniques improve, empowering staff to spend more time with patients⁴⁰.

Automating medical image evaluation. Medical imaging is a natural opportunity for machine learning because clinicians undergo intensive training to map from a fixed input space (e.g. the images) to an output (e.g. the diagnosis). There have been several recent successes in applying deep learning to medical imaging tasks. For example, physician- level parity in the detection of diabetic retinopathy¹, distinguishing between malignant and non-malignant skin lesions using in dermatoscopic images⁴¹, detecting lymph node metastases from breast pathology slides², and detecting hip fractures in x-ray images⁴².

Automating routine processes. Similarly, automating routine clinical processes stands to reduce the burden placed on clinical staff⁴³. For example, prioritizing triage order in the emergency department is often left to staff⁴⁴, but could be done algorithmically. Likewise, summarizing the contents of patients’ medical records⁴⁵ is a time-consuming, but valuable, task. For example, when hospital staff are unsure about patient’s disease status, they may call for an infectious disease consultation in which a specialist meticulously reviews all the available patient data, and manually summarizes disparate sources into a series of recommended tests and treatments⁴⁴. Researchers can leverage work from natural language generation working towards generating summaries from structured her data, saving much needed time⁴⁷.

Clinical Support and Augmentation: Optimizing clinical decision and practice support

Another set of opportunities focus on supporting and augmenting care. Rather than replacing a well-defined task, support requires understanding clinical pain points and working with clinical staff to understand appropriate input data, output targets and evaluation functions. Opportunities for support focus on work that often suffers due to real- world constraints on time and resources, often leading to information loss and errors (e.g., mistaken patient identification, flawed interpretations, or incorrect recall⁴⁸). In this setting, it is appropriate to evaluate how models improve downstream outcomes in tandem with clinical input rather than head-to-head comparisons to clinical staff⁴⁹.

Standardizing clinical processes. Variations in clinical training and experience lead to ranges of treatment choices that may not be optimal for targeting the underlying maladies of a patient’s state. For example, clinical staff may be unsure which medication sets or doses are most appropriate for a patient⁵⁰. To support such needs, past work has examined recommending both standardized order sets to help care staff quickly assess what medications they may have missed⁵¹ and default dosages to avoid dangerous dosing⁵². We note that automating a static, existing, protocol is significantly easier than full decision support for complex clinical guidelines that may change over time, and there is potentially much value in both in-hospital and ambulatory decision support⁵³, and at-home support⁵⁴.

Integrating fragmented records. Finite resources can also lead to a lack of communication and coordination, affecting patient care. For example, it can take years to identify domestic abuse survivors because any single clinical visit in isolation may be consistent with other causes (e.g. an admission for bruising is consistent with a spontaneous fall). Only a thorough review of a patient’s record will demonstrate the pattern of repeated admissions and other indicators (e.g., partners’ alcohol abuse⁵⁵). While possible without support systems in principle, machine learning can be a powerful tool, e.g., identifying domestic abuse up to 30 months in advance of the healthcare system⁵⁶.

Expanding Clinical Capacities: New horizons in screening, diagnosis and treatment.

As healthcare records become increasingly digitized, clinicians are faced with an ever-increasing amount of novel data for patients and populations. The result is an opportunity to give the healthcare system a new set of capacities to deliver healthcare in better and smarter ways. Importantly, creating new capacities requires the most involvement with clinical collaborators, and impact should be measured both in innovation and clinical value.

Expanding the coverage of evidence. While healthcare is an inherently data-driven field, most clinicians operate with limited evidence guiding their decisions. Randomized trials estimate average treatment effects for a trial population, but numerous day-to-day decisions are not based on high-quality randomized control trials (RCTs)⁵⁷. For example, the majority of commonly used ICU treatments are not rigorously empirically validated⁵⁸; some analysts have estimated that only 10-20%⁵⁹ of treatments are backed by an RCT. Even in settings where an RCT exists, the trial population tends to be a narrow, focused subgroup defined by the trial’s inclusion criteria⁶⁰, but that cohort may not be representative of the heterogeneous population to which trial results are then applied⁶¹. Finally, RCT results cannot reflect the complexity of treatment variation because, in practice, patient care plans are highly individualized. Prior work found that approximately 10% of diabetes and depression patients and almost 25% of hypertension patients had a unique treatment pathway (i.e., zero nearest neighbors) in a cohort of 250 million patients⁶². One way forward would be to leverage this naturally occurring heterogeneity to design natural experiments⁶³ that approximate the results of an RCT using fewer resources, thereby allowing a much larger set of clinical questions to be investigated.

Moving towards continuous behavioral monitoring. Phenotyping is an important goal in healthcare³⁰, and wearable data provides an ongoing way for devices to collect continuous non-invasive data and provide meaningful classifications or alerts when a patient needs attention. Recent work has focused on predicting such varied outcomes as heart attack from routinely collected data⁶⁴. There are further settings where non-invasive monitoring may be the only practical way to provide detection. For example, automatic fall detection for geriatric patients, or enforcing hand washing compliance in a clinical setting⁶⁵. Importantly, prior challenges identified in label leakage, soft labels, confounding and missingness must be considered carefully. In phenotyping chronic conditions, patients are often already being treated, and so early detection may sometimes amount to identifying an existing treatment.

Precision medicine for early individualized treatment. Precision medicine seeks to individualize the treatment of each patient; this is particularly important for syndromes—conditions defined by a collection of symptoms whose causes are unknown⁶⁶. For instance, acute kidney injury (AKI) is defined by a collection of symptoms characterizing kidney failure, not an underlying cause⁶⁷. Two individuals may have developed AKI for different reasons because there are many reasons that kidneys can fail. More measurements of the two individuals could reveal the difference in cause, which may in turn suggest alternative treatment strategies⁶⁸. By personalizing over time, one can learn individual-specific treatment effects that address the cause of the syndrome^11,69,70 in a particular individual. This relates to the ideas from “N=1” crossover studies in experimental design⁷². Personalized treatment is enabled by growing repositories of longitudinal data, where long-term progressions of an individual’s health are available^73-76.

Opportunities for New Research in Machine Learning

Addressing the hierarchy of opportunities in healthcare creates numerous opportunities for innovation. Importantly, clinical staff and machine learning researchers often have complementary skills, and many high-impact problems can only be tackled by collaborative efforts. We note several promising directions of research, specifically highlighting those that address issues of data non-stationarity, model interpretability, and discovering appropriate representations.

Accommodating Data and Practice Non-stationarity in Learning and Deployment

In most existing work, models are trained on the largest dataset possible and assumed to be fit for deployment, i.e., models do not keep learning. This is problematic in clinical settings, because patient populations and recommended treatment procedures will change over time, resulting in degraded predictive performance as the statistical properties of the target change. For example, clinicians previously assumed that estrogen was cardioprotective in menopausal women⁷⁷ and hormone therapy was routinely prescribed as a preventative measure until large trials reported either no benefit or an increase in adverse cardiac events⁷⁸. In developing new models for healthcare, models must be made robust to these changes, or acknowledge their mis-calibration for the new population⁷⁹.

Internal Validity - Shift over time. In a notable example of concept drift, Google Flu Trends persistently overestimated flu due to shifts in search behaviors⁸⁰. The initial model was a great success, leveraging Google search data to predict flu incidence; however, without update the model began to overestimate flu incidence in subsequent years as user search behaviors had shifted. While the drift was unintentional, the example motivates the need for models that continuously update.

External Validity - Shift over sources. There is also no reason to believe a priori that a model learned from one hospital will generalize to a new one. Many factors impact generalizability, including local hospital practices, different patient populations, available equipment, and even the specific kind of EHR each uses—transitions from one EHR to another create non-obvious feature mapping problems^81,82. This issue will remain until infrastructure to easily test across multiple sites becomes prevalent. The absence of such standardization creates opportunities with respect to data normalization and the development of models that are robust to differences in data collection at different sites^83-84.

Creating models robust to feedback loops. Models that learn from existing clinical practice are susceptible to amplifying the biases endemic to modern healthcare. While not yet observed in healthcare, such feedback loops have been noted in the deployment of predictive policing⁸⁵. Such biases reflected in deployed predictions can propagate into future training data, effectively creating a feedback loop that causes further bias⁸⁶. Work in algorithmic fairness should be considered as it motivates the need for systems that are sufficiently aware that they can alert us to such unwanted behavior. Additionally, models trained on EHR will learn from existing procedures, rather than represent formally best-practice protocols, which may be problematic given low manual compliance in much clinical practice⁸⁷.

Creating interpretable models and recommendations

In a clinical setting, black box methods present new challenges. Traditionally, quantitative training has not been emphasized in a physician’s extensive medical training⁸⁸ and most physicians do not have a robust understanding of rubrics such as positive predictive value⁸⁹. Models cannot be deployed “in the wild” at a low cost, and clinical staff must justify deviations in treatment to satisfy both clinical and legal requirements.

Defining what interpretibility means. There are many possible ways to interpretability, e.g., through feature space minimization, model regularization, or a preference for particular classes of models that have well-known post-hoc analysis methods⁹⁰. For example, providing a posterior distribution over possible decision lists⁹¹. Such lists can provide a natural way for clinicians to think about the relative risks of their patient’s condition.

Moving from interpretation to justification. In other domains, many forms of interpretability rely on human expertise, e.g., a model may highlight a single sentence from a user review (“The coffee is wonderful.”) as the rationale for a review prediction. Clinicians are unlikely to have a similar contextual framework, and it is unlikely to be obvious what a particular pattern of lab measurements that maximally activates a model means, biologically or clinically⁴⁵. We argue that models should instead provide “justifiability”; beyond explaining a specific prediction, models should strive towards justifying the predictive path itself. For example, recent work has proposed that locally-interpretable results can be presented for each individual prediction⁹². Another possibility is learning influence functions to trace a model’s prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction⁹³. Outside of reassuring end users, justifying a machine learning algorithm’s decision is also important for security concerns, as medicine may be uniquely vulnerable to “adversarial attacks”^94,95.

Adding interaction to machine learning and evaluation. Machine learning work in healthcare has an opportunity to create systems that interact and collaborate with human experts. Clinical staff provide more than their expertise; they also act as caregivers, and empathy is recognized as an important element of clinical practice^96,97. Building collaborative systems can leverage the complementary strengths of physicians and learning systems. There are many threads of research in the machine learning literature that can serve as a foundation for such systems. In active learning, for example, the goal is to leverage an oracle in order to learn using fewer samples⁹⁸. Apprenticeship learning is another related set of ideas⁹⁹. The study of collaborative systems, where the human and machine work together, is still in its early stages. Examples of such systems include content creation algorithms that alternate back and forth between human and machine proposals¹⁰⁸, and intelligent, data-driven operations for drawing software¹⁰¹.

Identifying Representations in a Large, Multi-source, Network

Representation learning has prompted great advances in machine learning; for example, the lower dimensional, qualitatively meaningful representations of imaging datasets learned by convolutional neural networks. Healthcare data lacks such obviously natural structures, and investigations into appropriate representations should include multi- source integration, and learning domain appropriate representations.

Integrating predictions from multi-source high-dimensional data. Individual patient data has ballooned to include many sources and modalities, making integration more challenging in systems currently struggling with overload^88,102. Using high-dimensional data to make well-calibrated predictions of established risks is a way that researchers can contribute. For example, inferring drug-resistance status in tuberculosis from whole-genome sequencing data¹⁰³, predicting cardiovascular risk from EHR data^64,104, readmission and mortality risk from longitudinal EHR data^39,105, and predicting cardiovascular risk from retinal images¹⁰⁶. New memory models are needed for sequence prediction tasks, because such records are not evenly spaced, can cover very long durations, and early events can affect patient state many years later; early work in this space has targeted learning individual variable decays in EHR data with missingness¹⁰⁷.

Learning meaningful representations for the domain. Learning meaningful state representations that provide good predictive performance on diverse tasks, and account for conditional relationships of interest, is an important area. Dealing with representations explicitly may be advantageous because they can conveniently express general priors that are not specific to a single predictive task¹⁰⁸, and this is particularly important for zero-shot learning¹⁰⁹ in unseen categories. There are several potential opportunities to address for representations for a clinical setting. First, a single patient input (e.g., physiological data) can correspond to many possible correct outputs (e.g., diagnostic codes), and this must be possible in the representations we explore. Additionally, there is likely value in incorporating structure and domain knowledge into representation learning. There has also been some initial exploration into learning representations of other data types that simultaneously capture hierarchy and similarity^110-112.

Conclusion

In this work, we give a practical guide that researchers can engage with as they begin work in healthcare. We emphasize that there are many opportunities for machine learning researchers to collaborate with clinical staff, and encourage researchers to engage with clinical experts early on as they identify and tackle important problems. Such efforts may lead to models that are clinically useful and operationally feasible¹¹³.