Applications of machine learning in acute care research

Abstract

Artificial intelligence has been successfully applied to numerous health care and non‐health care‐related applications and its use in emergency medicine has been expanding. Among its advantages are its speed in decision making and the opportunity for rapid, actionable deduction from unstructured data with that increases with access to larger volumes of data. Artificial intelligence algorithms are currently being applied to enable faster prognosis and diagnosis of diseases and to improve patient outcomes.^1,2 Despite the successful application of artificial intelligence, it is still fraught with limitations and “unknowns” pertaining to the fact that a model's accuracy is dependent on the amount of information available for training the model, and the understanding of the complexity presented by current artificial intelligence and machine learning algorithms is often limited in many individuals outside of those involved in the field. This paper reviews the applications of artificial intelligence and machine learning to acute care research and highlights commonly used machine learning techniques, limitations, and potential future applications.

1. INTRODUCTION

The medical community is currently experiencing an upsurge in the applications of machine learning and artificial intelligence research, due in part to the ease of deployment of related algorithms to cognitive, cloud‐based computing platforms, made possible by major technological firms. These techniques have been used to evaluate issues in a range of medical disciplines including mental health, oncology, and cardiovascular care. ¹ , ² Artificial intelligence is a computer science area that emphasizes the creation of intelligent machines that have human‐like working and reaction. Machine learning, a subset of artificial intelligence, is a data analysis method that makes analytical model building automated, based on the theory that systems can identify patterns, learn from data, and make decisions with minimal to no human intervention. Machine learning algorithms can be used to identify patterns in unstructured data through supervised or unsupervised learning, using them as the basis for reaching reliable conclusions. Several machine learning techniques are being used with increasing frequency in emergency medicine, with specific techniques being well suited to acute care research. These tools have the potential to evaluate large datasets with exceptional precision, often in short periods of time (ie, minutes). Although these techniques show promise, there are limitations to their application that are important for acute care research. The purpose of this article is to (1) identify commonly used machine learning techniques that may apply to acute care research, and (2) outline the steps an investigator may take when planning a study using machine learning techniques to address potential limitations and mitigate bias.

2. OVERVIEW OF MACHINE LEARNING TECHNIQUES

2.1. How do machine learning techniques work?

Machine learning models use various techniques to recognize patterns in data. Machine learning techniques can be thought of similar to how providers use pattern recognition to identify critical conditions on an ECG such as ventricular fibrillation or ST‐segment elevation myocardial infarctions. ³ Machine learning models “learn” patterns through incremental and iterative exposure to problem‐solving, and various techniques can be used to “teach” machine learning models how to understand systems for which solutions are sought. Machine learning techniques are broadly grouped into 4 categories; “supervised” (learning by example in which inputs and outputs are already defined), “semi‐supervised” (learning based on predefined parameters), “unsupervised” (independent identification of patterns), and “reinforcement learning” (independent learning within the parameters of defined boundary conditions) (Figure 1). ⁴

These techniques can be applied to different datasets depending on required outcomes. For example, supervised learning may be used when known factors are being “taught” to the machine learning algorithm. An example of this may involve training an algorithm to identify predictors of cardiac arrest based on ECG data. Here, investigators identify dysrhythmias such as ventricular fibrillation and apply supervised learning techniques to the data analysis. Semi‐supervised learning techniques can be applied to the same ECG data. Investigators may input broad parameters such as a heart rate and QRS width into the machine learning model but not specifically define ventricular fibrillation. In an unsupervised approach, the machine learning technique can identify patterns on its own that may be associated with the outcome of interest, potentially identifying previously unknown ECG variables associated with cardiac arrest. Finally, reinforcement learning allows for a combination of semi‐supervised (specific parameters) and unsupervised (independent analysis) techniques.

Machine learning algorithms (especially when using reinforcement learning) can apply complex and extensive models to data analysis and attempt to replicate the workings of the human brain's neural network for inferencing. The framework of these machine learning techniques is not often linear, instead, it is a network of computations that iteratively train the models, with increasing accuracy as the models are exposed to more data. ⁴

2.2. Why use machine learning?

The efficacy of various traditional statistical analysis tools (eg, logistic regression) in drawing useful conclusions from datasets vary, depending on unique cases and the needs of the investigators. Several key differences between machine learning and traditional statistical regression models are detailed in Table 1. Machine learning algorithms further enrich the existing analytic toolset by allowing for nuanced analyses of larger data sets and identification of patterns that may not be easily observed (eg, in a large dataset) or that might be challenging with conventional statistical analyses (eg, comparing time‐dependent variables at multiple, variable intervals). In addition, machine learning models continue to “learn” with the addition of new data, thus continually improving the model's accuracy. ⁴

TABLE 1.

Comparison between regression models and machine learning techniques

Characteristics	Traditional statistical regression models (linear or logistic)	ML Techniques
		ANN	SVM	NB
Purpose	Inference about the relationship between variables	Maximize prediction accuracy
Transparency	High transparency with model building	Variable depending on the ML technique. Interpretation of the internal working can be difficult and limited (ie, “black box”)
Prediction accuracy	Variable	High	High	High
Training and testing	Single dataset used for model development	Training and testing sets required and additional training improves the prediction accuracy
Human dependency	High	Low	Low	Low

ML, machine learning; ANN, artificial neural network; SVM, support vector machine; NB, Naïve Bayes classifier

3. COMMONLY USED MACHINE LEARNING TECHNIQUES

3.1. Artificial neural network

Artificial neural network is a widely used machine learning technique that is available on different computing frameworks. It intends to mimic the neural network and decision making skills of the human brain. ⁵ Structurally, artificial neural network can be conceptualized using 3 layers; an input neuron layer, a hidden neuron layer, and an output neuron layer (Figure 2). The neurons are connected in a way that the outputs of each neuron in the preceding layer serve as input for the succeeding layer of neurons. Artificial neural network systems use a set of well‐defined input parameters (a dataset) to reach an output. Generally, the dataset is divided into unequal parts. The large portion is used for “training” (a subset of available data that the machine learning model depends on for data characterization) the artificial neural network algorithm and the smaller portion for “testing” (evaluation and comparison of chosen model with other ones to determine which is the best fit to the trained model) or “cross‐validation” (used for tuning a model's hyperparameters). ⁶ , ⁷ , ⁸ In machine learning, a variable whose value is set before the learning process begins (such as a d‐dimer threshold for the workup of a pulmonary embolism) is termed as a hyperparameter. Artificial neural networks use a technique known as “backpropagation” where the output serves as feedback to continually refine the model. ⁹ , ¹⁰

FIGURE 2.

Input parameters of patients and generating a decision based on connections for artificial neural networks. ⁵ All the neurons represented by a large circle of a layer are connected to the neurons in the adjacent layer using nodes. The output of each neuron is multiplied with a multiplication factor represented by a dark small circle. This artificial neural network has 2 output modes that represent 1 possible output each

FIGURE 1.

Classification of machine learning models. This figure represents a broad classification of the types of learning, category of algorithm, and methods of machine learning. ML, machine learning; SVR, support vector regression; GPR, Gaussian process regression

In addition, artificial neural network algorithms reorganize themselves during the training process. ¹⁰ Unlike other computer programs, which follow a code or set of instructions, artificial neural networks are capable of developing grounds for working to make decisions. ¹¹ Artificial neural networks have an excellent ability to accurately predict outcomes even when some of the neurons fail to work as required. This ability is called “fault tolerance” and can give comparable and acceptable output even in the absence of some variables forming the input data set. ¹² Disorganized data with “noise” has less effect on artificial neural networks compared to other analytical tools. This also helps artificial neural networks to make decisions more rapidly and precisely than manually performed analytical methods, making it an attractive machine learning technique for use in acute care research. ¹³

Although there are many strengths to artificial neural networks, there are some key limitations. To make a precise decision, an artificial neural network algorithm needs to maximize the number of input prognostic parameters (eg, age, sex, medical history). ¹¹ , ¹⁴ , ¹⁵ Therefore, an artificial neural network often requires more data points than other machine learning techniques. However, large datasets with multiple patients and data points, such as those from a health system's electronic medical record, are uniquely suited for analysis with artificial neural network. ⁵ , ¹¹ , ¹⁴ Clinicians might be unaware of the connection strength between nodes, the multiplication factor, and the threshold value used by artificial neural network at each stage, and the reasoning behind every conclusion might remain unknown. ⁹ Each artificial neural network decision is based on trial and error because a model learns the patterns of interest. ⁸ A “black box” situation occurs when an output is obtained with little or no information on the internal processes leading to the generation of the observed results. Training results in internal changes to parameters like threshold values and the multiplication factor of the artificial neural network. This resultant “black box” can shroud the methodology behind artificial neural network's decision making process. ¹⁶

Artificial neural network algorithms have numerous applications including speech recognition, image processing of biometrics, retina scanning, fingerprint scanning, and capturing of facial features. ¹¹ Artificial neural networks have been applied in medicine to aid in the diagnosis, mortality predictions, and estimation of the cost of treatment. ¹⁷ , ¹⁸ They have also been used to determine the risk of coronary artery disease, using variables relating to ECG characteristics, cardiac image data, and cardiovascular drug dosing, with results often outperforming current techniques. ⁹ A key example of artificial neural network application is its use in the prediction of successful extubation for patients in the ICU. ¹⁸ In this study, the authors used data from 3602 patients split into a 9:1 ratio (n = 3242:n = 360) for training and testing, respectively. Overall, 5.1% of patients were not successfully extubated, and a total of 37 prognostic parameters were evaluated in the artificial neural network algorithm including age, comorbidities, pre‐extubation vital signs, and weaning parameters such as rapid shallow breathing index. ¹⁸ Overall, the artificial neural network outperformed other parameters with an area under the curve (AUC) of 0.85. ¹⁸

3.2. Convolutional neural networks

A similar process to artificial neural network is convolutional neural networks that deal primarily with images, whereas an artificial neural network can work both with images and other forms of data. Convolutional neural networks performs image recognition and decision making through the iterative application of transfer learning. ¹⁹ Transfer learning refers to the derivation of the solution to a problem through dependence on stored solutions to previously resolved problems. Every layer of convolutional neural networks can recognize a feature that can be a pattern, texture, color, shape, etc. ¹⁹ These layers are connected by a single node that transfers data to the next layer. The decision making layers have simple neurons and are connected similar to artificial neural network. ¹⁹ Weights are used to connect each neuron in 1 layer to every neuron in the next layer. Weight determines the strength of the connection between the neurons. Increased weights on the inputs, thereby influences the output. A key advantage is that convolutional neural networks requires less computing hardware compared to artificial neural network. ¹⁹

3.3. Support vector machines

Support vector machines are another commonly used machine learning technique and generally considered to be one of the most accurate and discriminative classification methods. ²⁰ , ²¹ The complexity of the model is balanced against the risk of over fitting it with data in the training or derivation sets. ²⁰ Fitting of complex non‐linear relationships in data are permitted by kernel functions that identify clusters of similar data. ²³ , ²⁴ , ³² Support vector machines are trained to determine boundaries known as “separating hyperplanes” (Figure 3). The hyperplane separates one population (or several) from another along an imaginary boundary that is equidistant from each population. For example, imagine 2 cohorts presenting to the emergency department (ED) for evaluation of chest pain; one group has aortic dissection as the cause of their chest pain and another group does not. Systolic blood pressure may differ between these 2 groups, and support vector machines can be trained to determine the boundary or distance between the systolic blood pressures of these 2 cohorts. This is a 1‐dimensional example using blood pressure as the only variable to create this separating hyperplane. A support vector machine also has the ability to create these hyperplanes using multiple input variables and thereby, multi‐dimensional hyperplanes. ²⁵

FIGURE 3.

Separating hyperplanes in support vector machines. ²⁶ The plane in the figure divides the data set into 2 categories based on the criteria given into the support vector machine model. This plane is called a separating hyperplane. Yellow and purple markers represent different categories of data. There can be multiple hyperplanes depending on the various criteria that are uploaded to the support vector machine

Training a support vector machine model involves learning the weights of different features that maximize the separation, or margin, between different classes, as defined by the separating hyperplanes. The weight of a feature represents how well it describes the object or outcome of interest. For example, a table has 4 vertical legs and a horizontal platform. The legs and the platform are the features that best describe a table's form. In the aortic detection example, a history of connective tissue disorder or widened mediastinum on chest x‐ray may help to better describe the characteristic of the cohort with aortic dissection than systolic blood pressure and therefore carry more weight in the support vector machine model.

Support vector machines can classify large numbers of variables despite small sample sizes. The generalizability, modularity, and scalability of a support vector machine make it more accurate than other existing machine learning methods when applied to datasets with multiple variables. ²⁷ This algorithm can potentially handle millions of features with minimal impact on computational time while minimizing the number of potential misclassifications, compared to other methods. ²⁸ Multi‐class support vector machines with multiple hyperplanes can categorize each of these different data points into different hyperplanes and can increase the accuracy of the resulting outcome. ²⁸ Data points may fall into a common region and create a “grey zone” with limited ability to discriminate between populations. ³ A support vector machine needs datasets with both positive and negative outcomes (eg, the dataset must include both patients who do and do not have the disease or outcome of interest) to allow it to accurately identify the hyperplane that differentiates the outcomes of interest. ³

Support vector machines have been used in acute care research, and in one example, the authors used support vector machines to identify the glottic opening from video laryngoscopes. ²⁹ In this study, 7 providers intubated a mannequin 10 times and recorded the intubation attempts using a video laryngoscope. Each recording was partitioned into 1‐second time epochs and evaluated for the presence or absence of the glottic opening. The authors divided the recordings into training and testing sets and evaluated the accuracy of a number of machine learning algorithms to determine the sensitivity and specificity of each for detecting the glottic opening. ²⁹ Support vector machines outperformed several other machine learning techniques with a sensitivity of 70% and a specificity of 90%. ²⁹ Other studies have used support vector machine models to detect acute coronary syndrome and sepsis with high accuracy. ³⁰ , ³¹

3.4. Naïve Bayes classifier

Naïve Bayes classifier is a well‐established classification technique that has been used and proven reliable for nearly 50 years. ³² Naïve Bayes works on the principle of checking the probability of whether a variable is within a possible class (or outcome). It then identifies the most probable class of the data to assign the variable to and assigns it to that class. Naïve Bayes not only works well for independent parameters in the data set but also variables that are highly dependent on each other. ³³

One key feature to naïve Bayes is transparency in the decision making process of this classifier—investigators can control what variables are included in the classification process. ³⁴ This also allows naïve Bayes to be combined with other probabilistic classifiers thereby improving the accuracy of the results, similar to a Bayesian approach. ³⁵ For example, the use of naïve Bayes is analogous to the identification of fruit as a pear on the basis of the shape, color, size, etc. Each of these parameters is independent of each other; however, they all contribute to the provision of an accurate depiction of a pear as a fruit. Naïve Bayes also works well with small data sets (10–30 observations) and can potentially classify and differentiate data in these situations.

Given that many providers think in a Bayesian manner (ie, is this patient high risk for acute coronary syndrome?), naïve Bayes is uniquely suited for assisting with diagnostic processes in emergency medicine. This model could aid in decision making, and with enough refinement, might even be used as a final check for providers when finalizing a diagnosis. The basis of the decision can be used for development and further study in this field. As an example, a naïve Bayes classifier was used on 82 pediatric patients presenting to an ED with asthma exacerbations to identify which patient would require admission. The data set initially included 240 elements and 5 prediction models were considered. Of the tested models, naïve Bayes had the greatest accuracy for predicting admission (70.7%) and performed almost as well as emergency physicians' judgment (78%). ³⁶

4. DEEP LEARNING

Deep learning is a function of artificial intelligence that mimics the workings of the human neural system to create patterns and process data for usage in decision making. As a subset of machine learning, deep learning has networks capable of learning unsupervised from data that is unstructured or unlabeled. Deep learning models can perform classification tasks directly from images, text, or sound using neural networks and is therefore an extension of artificial neural network. Deep learning is also known as a deep neural network or deep neural learning. ³⁷

5. HARDWARE PLATFORMS USED FOR MACHINE LEARNING

The application of different artificial intelligence and machine learning techniques to data analysis requires extensive use of computing resources. This varies, depending on the model type, software platform, and data size/type. The greatest computational burden falls on the processing and graphics capabilities of computer workstations. In most cases, speed and accurate predictions/inferencing are most important. Most applications of machine learning and artificial intelligence algorithms demand processing speeds in excess of 1 trillion floating‐point operations per second (1 TFLOPS). Computers with regular central processing unit (CPU) capabilities are the least recommended for machine learning in this instance because of their low TFLOPS ceiling (2 TFLOPS uses up about 120 GB of random‐access memory [RAM]). Graphics processing units perform better compared to CPUs. A graphics processing unit can perform remarkably higher FLOPS (∼125 TFLOPS using 16 GB of graphics memory). A tensor processing unit is an accelerated processing unit designed uniquely for machine learning, made available by Google for use on the Cloud. Each board contains 2 tensor processing unit packages of 2 cores with 8 GB of memory in each core. A single package can support 45 TFLOPS, and a default Cloud tensor processing unit configuration has 4 cores supporting 180 TFLOPS. ³⁸ Although tensor processing units are Cloud‐dependent, graphics processing units can be used on the edge within local workstations. In recent times, tensor processing units running TensorFlow Lite have been developed for edge computing. Although these are current approaches, future models may require additional computing power.

6. DIFFERENCES ACROSS CLASSIFICATION MODELS

Because emergency medicine involves time‐sensitive processes and decision making needs to be based on accurate, data‐based inferencing in situations with numerous uncertainties, a rapid, accurate data classification method is ideal, and the implementation of a machine learning technique that is most suited for making inferences on specific health outcomes is of paramount importance. Although this manuscript serves as a starting point for discussing the applications of machine learning in acute care research, it is not a comprehensive review of all machine learning techniques that may be applied to unique or specific situations. This represents a primer for those interested in this line of work and should guide individuals toward examples of how machine learning techniques have been applied. It also highlights key issues encountered when attempting to perform machine learning in acute care research.

The implementation of these machine learning techniques is still being developed, and their practicality is still under investigation. As such, there may be machine learning techniques developed in the future that outperform the methods mentioned here. Although there is no “one‐size‐fits‐all” machine learning technique, support vector machines are uniquely suited to acute care research given their generalizability, modularity, and scalability over variable types of data. ¹⁸ The accuracy of support vector machine can continue to improve over time, if continually trained, and can often be implemented in short order with limited training sets. ³⁹ , ⁴⁰ Ideally, however, investigators may select a panel of machine learning models to ensure consistency with their results much the same as a sensitivity analysis can confirm other findings in a study.

7. LIMITATIONS OF MACHINE LEARNING TECHNIQUES

Although machine learning techniques show promise, there are some unique limitations investigators should be aware of when attempting machine learning analysis. Depending on the machine learning model chosen, large datasets may be required for training to ensure prediction accuracy. Furthermore, most datasets need to be manually labeled (data that have been annotated and formatted with 1 or multiple designations) so machine learning models can use labeled data as a reference for training, testing, and validation. Labeling large amounts of data can be expensive and time‐consuming. To minimize this challenge, technology companies have created, and are constantly improving on solutions that minimize the burden of data labeling. One example is Amazon's Mechanical Turk(R) (Amazon, Inc.). Deep learning does not lend itself to easy, linear explanations, presenting a barrier to adoption.

Machine learning techniques are at risk of over‐fitting the training data and under‐performing with new data. A solution to over‐fitting in a predictive model is cross‐validation, particularly in cases where the amount of data are limited. In cross‐validation, a fixed number of folds (or partitions) of the data are made, analysis on each fold is run, and then the average the overall error is estimated.

8. FUTURE OUTLOOK

Considering the rapidly growing needs of the health care industry, the incorporation of the techniques of artificial intelligence through machine learning can help advance acute care research. Although many types of machine learning models are available and may be used, support vector machine is uniquely suited for acute care research at this time. The application of machine learning techniques in acute care research is at its infancy and will continue to expand, potentially providing new insights into how care can be provided to acutely ill and injured patients. There are key considerations for researchers and providers to consider with machine learning as these techniques become more widely used.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

AUTHOR CONTRIBUTIONS

All authors contibuted to the concept and development of this work. JNC takes responsibility for this work as a whole.

Ohu I, Benny PK, Rodrigues S, Carlson JN. Applications of machine learning in acute care research. JACEP Open. 2020;1:766–772. 10.1002/emp2.12156