Machine Learning, Deep Learning, and Closed Loop Devices—Anesthesia Delivery

BACKGROUND

With the gargantuan volume of data captured during surgeries and procedures, critical care, and pain management, the field of anesthesiology is uniquely suited to the effective application of closed loop technologies, machine learning, and neural networks. In any given aspect of anesthesia practice, be it sedation, the critical care setting, or outpatient pain management, thousands of data points are at the anesthesiologist’s disposal in making decisions. Years of experience help with interpreting these, but no matter what is done to optimize clinical decision making, the human brain is subject to bias, distraction, and fatigue. Computational improvements in the recent past have made development of practical tools to augment human intelligence finally feasible. In the past several years, these areas have expanded immensely in both interest and clinical applications.

Historically, anesthesiologists have been early pioneers of closed loop devices. As early as the 1950s, Bickford¹ and others developed an automated delivery of volatile anesthetic based on electroencephalogram (EEG). Subsequent efforts expanded to sophisticated closed loop systems for achieving optimal end-tidal volatile concentration, neuromuscular blockade, and mean arterial pressure.^2-4 The classic open loop control system in anesthesiology, target-controlled infusion devices, in which a hypothetical plasma or effect-site concentration is targeted based on estimations from a population model of drug distribution and effect, bourgeoned outside the United States, from an emerging technology in the 1990s to a mature one today.⁵ Particularly in the United States, however, concerns with regulatory issues, safety, and liability and a lack of convincing demonstration of significant clinical impact on patient outcomes have been significant impasses for both open loop and closed loop devices.^6-15

Concomitant to progress with closed loop innovations have been significant advances in computational techniques and technology, which have made machine learning and other modalities within artificial intelligence (AI) markedly more accessible in recent years. Early efforts at clinical applications within anesthesiology were focused primarily on EEG analysis and depth of anesthesia monitoring but since have expanded considerably.^16-18 Applications of machine learning and other methods within AI span a vast array of purposes but typically fall into 3 common overall goals^19,20:

1. To analyze large amounts of data in order to search for novel patterns or groups among variables (also known as data mining)

2. To leverage highly complex data sets, such as medical images, EEG waveforms, or multiple hemodynamic signals, over time

3. To generate models or algorithms to predict an event or continuous variable, such as degree of sedation, respiratory depression, or response to nociception

This article provides an overview of the basic tenets of closed loop devices, machine learning, and neural networks. This summary is intended for all audiences within anesthesiology and is by no means exhaustive. Particular emphasis is given to the clinical applications for these technologies. Although the authors have taken efforts to provide as much structure as possible in this text, there inevitably is overlap between these modalities.

WHAT ARE CLOSED LOOP SYSTEMS?

Closed loop control devices are fully automated systems in which a sensor(s) provides feedback to an algorithm that determines the action to take in order to achieve a desired target (Fig. 1). In most cases, the sensor(s) measures and provides feedback to the algorithm repeatedly, and the algorithm repeatedly directs corrective actions, thus creating a closed loop. Also known as automated control systems, closed loop systems act to maintain a given variable at a desired set point via 3 key elements: a sensor, a controller, and an actuator.¹⁵ The sensor or measurement device senses the target parameter and generates a feedback signal that characterizes the status of the controlled variable. The controller queries the disparity between the feedback signal and the desired set point. Then, through a controlled algorithm, the controller generates an output signal for what corrective event should occur. The actuator then converts that signal to actual physical intervention.

Fig. 1.

Basic elements of a closed loop system. A target value range (eg, a mean systolic blood pressure range) is supplied to the controller. These settings then are compared with the output or feedback data (eg, patient blood pressure). The difference between the actual and target values then is processed and action is taken to adjust the input or manipulated variable such that the actual values reach the target range.

The algorithms underlying the closed loop system can be simplistic (eg, if stroke volume variation goes above a certain set threshold, then a bolus is administered) or may utilize AI. In practice, most closed loop systems utilize reinforcement learning, a type of machine learning discussed in more detail later.

Examples of closed loop systems are ubiquitous in daily life—thermostats, clothing dryers, voltage stabilizers, and numerous elements of vehicle navigation, such as cruise control and autopilot systems. In the example of the thermostat, there exists a temperature sensor, a heater or air conditioner, and a unit that allows the user to set a desired temperature; and the heater or air conditioner turns on and off as needed to achieve that temperature based on the measured temperature of the room. These systems have immense potential within medicine. Due to the complexity and variability of inputs, however, not to mention the enormous ramifications of error, the technologies have yet to be rolled out in day-to-day clinical medicine for most practicing anesthesiologists.

WHAT IS ARTIFICIAL INTELLIGENCE?

Just as algorithms underlie closed loop systems, algorithms are at the heart of machine learning and neural networks. The term, algorithm, refers to a systematic procedure or method of solving a problem or accomplishing some end. Put simply, machine learning and neural networks are just methodologies within AI, in which computer systems are created to help perform tasks that normally require human intelligence. With that, some basics of AI are delved into.

Types of Outcomes

One way AI within medicine can be classified is by the types of outcomes that are predicted. Classification involves organizing data into categories or discrete groups. Examples of these are models that aim to predict a binary outcome like mortality after surgery.²¹ Regression, on the other hand, utilizes modeling to predict continuous variables, such as predicting procedural or recovery times for the purposes of optimizing resource utilization.^22,23

Types of Machine Learning Methods

There are several methods that can be used, depending on the type of question and data used. In reading about machine learning algorithms, 3 common methodologies frequently are referred to:

1. Supervised learning

2. Unsupervised learning

3. Reinforcement learning

Supervised learning involves creation of an algorithm that is “trained” to predict a defined entity or outcome. Unsupervised learning, by contrast, does not involve introduction of a priori hypotheses; thus, algorithms then are used to identify patterns, structure, or clusters within a data set. In reinforcement learning, an algorithm is trained to perform an action (eg, deliver an anesthetic to a patient) and to receive feedback and learn from its own errors and successes.

Types of Machine Learning Techniques

Although these descriptive terms help in understanding and assessing AI algorithms, it also is important to delve a bit into the types of techniques commonly used to analyze complex or large data sets. Because supervised learning currently is the most common type of machine learning utilized in medicine, this article describes some of the most common supervised learning machine learning algorithms, including neural networks and bayesian techniques. The techniques described are a small subset of what currently exists, and the choice of a machine learning technique is based on several factors, such as experience, data input, interpretability, and so forth.

One of the most popular types of models applied in medicine is logistic regression. Although it is not considered a complex machine learning technique like neural networks, it is important to understand the distinction in order to understand the potential benefits and pitfalls. Logistic regression is considered to be a more traditional and simpler modeling technique, which contrasts in several ways to some of the more state-of-the art AI models. In logistic regression, the structure is simple, the hypothesized effect of an individual variable is straightforward, and the input variables (or features) do not interact with one other. Other machine learning models, like those described later, can allow for relationships between the features and learning of new features as well as learning between features and outcomes.

Another popular model is the decision tree, which utilizes tree models (akin to a rules-based decision flowchart) with branch-points or nodes to establish a target output based on inputs (Fig. 2). Each node within a decision tree has an assigned value, with the final node representing an outcome as well as the probability of arriving at that outcome based on the preceding decision tree path.²⁴ Random forests then are an extension, in a way, of decision trees (see Fig. 2). Although a decision tree consists of a single sequential decision tree, a random forest model allows for multiple trees, creating an ensemble model. Each individual decision tree incorporates a random subset of features; the individual trees then are combined to generate a final output.

Fig. 2.

A simplified decision tree (A) and random forest (B). A single decision tree is simply a series of sequential questions. A random forest consists of a large number of individual decision trees. Each decision tree is trained on, or utilizes, a random subset of features, or bagged data. These individual decision tree outputs then are aggregated to produce a single model.

Bayesian techniques use a known previous probability distribution of an event along with a probability distribution in a given data set.²⁵ This modality allows for both modeling of uncertainty and updating or learning repeatedly as new data are made available.²⁶ Similar to classical supervised decision tree learning, there are several assumptions that underlie any results produced.

Neural networks allow for an exponentially higher degree of connections and logic. In a typical neural network, each network is composed of an input layer of neurons, which are composed of features that describe the data, as well as a hidden layer of neurons that perform mathematical transformations on input features and an output layer that produces an outcome (Fig. 3). Multiple connections between neurons exist and can be weighted differently depending on input-output maps.²⁴

Fig. 3.

Basic structure of a neural network. The input layer incorporates features supplied by the user. The hidden layer converts inputs into features useable by the network. The output layer then converts the hidden layer results into an interpretable output.

Developing a Machine Learning Model

When describing the development of a machine learning algorithm, it also is common to see the phrases, training data and test data. The training data are used to train the machine learning algorithm to analyze and learn the associations between the inputs and the output of interest. The test data are used to assess the performance of the trained machine learning algorithm on a set of data it has never seen before. For example, 70% to 80% of a data set can be allotted for training, with the remaining 20% to 30% reserved for testing.²⁴

Prior to training an algorithm, the features to be input into the algorithm need to be decided on. Features can be hand-selected by domain experts or a feature selection algorithm (such as lasso regression or sequential feature selection) can be utilized. The purpose of feature selection is to reduce the number of features, which can be limitless, to the most important ones for the model.

CLINICAL APPLICATIONS OF CLOSED LOOP DEVICES AND ARTIFICIAL INTELLIGENCE

There is mounting evidence that the ability to achieve intraoperative goals has significant effects on long-term outcomes.²⁷ When caring for a patient, an anesthesiologist takes in multiple inputs, analyzes the effects these multitudinous variables may have on each other and the patient, and then makes an intervention to ensure the patient stays within a range of goals. These are done in an operating room almost without thinking—adjusting ventilator settings, anesthetic gas delivery, titrating infusions, and so forth. Due to the multitude of inputs, individual practice variations, potential distractors, coverage models, and a frequently high-stakes milieu, however, there are numerous ways in which automated intelligent devices can assist in providing optimized care.¹⁵

There are numerous examples of improved outcomes with reduced interprovider variation and protocol-driven pathways. Kurz and colleagues²⁸ was one of the first randomized clinical trials demonstrating worse outcomes in patients who did not receive extra measures to ensure normothermia intraoperatively. Numerous randomized studies also now show resounding concrete evidence that proactive use of hemodynamic monitoring along with therapies to control hemodynamics significantly reduced mortality and surgical complications.²⁹ And there are numerous studies showing excessive depth of anesthesia associated with mortality and other worse outcomes.^27,30-34 Enhanced recovery after surgery pathways, in which multiple aspects of care are targeted for optimization and protocolized, now seemingly are ubiquitous.³⁵

Speaking in general terms, closed loop devices and AI techniques within anesthesiology are enlisted to achieve a few common goals in order to improve patient care:

1. Keep patients within some kind of physiologic target range.

2. Reduce variability within an individual patient.

3. Reduce variability of care given to one patient versus another, that is, encounter variation, provider variation, and institutional variation.

4. Improve outcomes.

CLOSED LOOP DEVICES IN ANESTHESIOLOGY

Most current applications of closed loop devices in anesthesiology fall into a handful of clinical arenas¹⁵:

1. Anesthetics

2. Intravenous (IV) fluid

3. Vasopressors

4. Mechanical ventilation

5. Glucose control

Some of the relevant studies in each of these clinical areas are reviewed.

Closed Loop Anesthetic Systems

The application of closed loop devices to anesthetic agents and depth of anesthesia is to many the apogee of anesthesia research and innovation. Closed loop systems in anesthesia first were pioneered in the 1980s and since have progressed much nearer to more widespread application.^6,7 These systems have shown immense promise in terms of reduction in clinicians’ workload and improved control of drug delivery.¹⁴

There currently are approximately 20 studies in this area. Most have studied adults during the intraoperative period and IV agents (most commonly propofol and remifentanil), and most utilized bispectral index (BIS) as the target variable for anesthetic depth. A range of cases has been studied, including cardiac, general, gynecologic, vascular, thoracic, and spinal surgeries and procedures.^{8,12,14,36-41} A meta-analysis by Brogi and colleagues¹⁵ of 15 studies showed automated systems increased the percentage of time the outcome variable (depth of anesthesia) was maintained in the desired range by 17.4%. A subset of these studies also examined the proportion of time that the controlled variable was above or below the targeted set point, with meta-analysis showing 12.3% more undershooting or overshooting in the manual groups compared with the closed loop groups.^{8,11,12,14,36,37,39,41-47}

The subset of studies isolated to using BIS and total IV anesthesia showed in meta-analyses that the closed loop anesthetic delivery systems were associated with significant lower doses of propofol at induction of anesthesia and significantly shorter recovery time.⁴⁸

Goal-Directed Fluid Therapy Closed Loop Systems

Although there are several in vivo and in silico studies, currently there have been 2 randomized control trials in humans.^49-51 The smaller of the 2 studies examined 46 moderate-risk to high-risk abdominal surgical patients with an arterial catheter-based cardiac output monitoring system with colloid fluid boluses in response to the closed loop control.⁵⁰ Although no difference was found in this study, the groups were small with notable differences in baseline characteristics. In the larger study, Joosten and colleagues⁵¹ studied 104 patients undergoing elective major abdominal surgery. Stroke volume and stroke volume variation were monitored via arterial catheter–based system, and crystalloid or colloid boluses were administered in response to the algorithm.⁵¹ Patients in the closed loop group had significantly shorter length of stay compared with historical controls and reduced incidence of both major and minor postoperative complications.

Closed Loop Vasopressor Devices

Closed loop vasopressor devices have immense promise to improve the ability to maintain optimal therapeutic control of hemodynamics. There have been 2 large randomized controlled studies in this area, both undertaken in the setting of cesarean section with spinal anesthesia.^52,53 Kee and colleagues⁵² examined blood pressure and heart rate of 214 patients with a computer-controlled phenylephrine delivery system using both intermittent boluses and continuous infusion. They found blood pressure control to be more precise when computer-controlled phenylephrine was delivered using intermittent boluses rather than continuous infusion. Sng and colleagues⁵³ evaluated 216 patients’ noninvasive blood pressure and utilized computer-controlled phenylephrine for maintenance of blood pressure also during spinal anesthesia for cesarean delivery. In pooled analysis of the 2 studies, the automated systems were found to increase the number of measurements within the target range in comparison to manual control.¹⁵

There also are a handful of feasibility studies in this area with major promise. Joosten and colleagues⁵⁴ demonstrated efficacy in a pig model with an induced hypotension model using nitroglycerine and automated controller–titrated norepinephrine. This study showed efficacy in correcting hypotension to keep the mean arterial pressure within 5 mm Hg of the target for 98% of the time. These same investigators also undertook a feasibility study in 20 human subjects undergoing elective moderate-risk and high-risk surgery.⁵⁵ They showed that the closed loop vasopressor control system maintained mean arterial pressure within 5 mm Hg of the target for 91.6% of the intraoperative period and effectively minimized hypotension to 2.6% of the intraoperative period.

Closed Loop Mechanical Ventilation

Nine trials have investigated the accuracy of closed loop ventilation systems in comparison with manual control of ventilation.^56-63 Mechanical ventilation theoretically should be well suited for the application of closed loop systems; however, there are various inputs and outputs clinicians utilize. Thus, it is not surprising that these studies have a fair degree of heterogeneity within variables utilized as inputs and desired outcomes. Of the current studies, approximately half used oxygen saturation as measured by pulse oximetry as the controlled variable, with an automated fraction of inspired oxygen adjustment. The other studies used tidal volume, respiratory rate, and end-tidal CO₂ ranges in order to examine the feasibility of closed loop pressure support ventilation systems to maintain an acceptable ventilation zone.

Subgroup analysis within a recent meta-analysis showed greater maintenance of the controlled variable within the target range in the automated systems versus the control group by approximately 8%.¹⁵ The authors expect in the coming years to see additional progress in this area.

Closed Loop Insulin Administration

More than 10 studies have examined closed loop insulin delivery system for glucose control. This area has made massive strides with both programmable home insulin subcutaneous insulin delivery devices and high-fidelity nearly continuous glucometry. This area also in some ways is much more straightforward than mechanical ventilation, owing to the fact that the controlled therapy (insulin) and target (blood glucose level) are both single, agreed-upon entities.

In patients with type 1 diabetes mellitus, a recent meta-analysis of 8 studies using closed loop insulin delivery systems with pump insulin therapy showed that automated systems were associated with a 21.2% greater time with maintenance in the desired range.¹⁵ Subgroup analyses within the same meta-analysis, demonstrated that compared with the control group, the automated systems also decreased the over-shoots and undershoots, with the automated systems showing a 6.5% reduction in the percentage of time above or below the target range.¹⁵ Meta-analysis of studies in the intensive care setting showed similar benefits; however, these results were not statistically significant.¹⁵

MACHINE LEARNING IN ANESTHESIOLOGY

Depth of Anesthesia

Anesthetic depth has become a particularly valuable clinical target, with several recent studies suggesting poorer outcomes associated with low BIS or excessive depth of anesthesia.^11,63 A wealth of studies have investigated machine learning and neural networks in the context of depth of anesthesia monitoring.^16-18,64,65 Arguably, the increased utilization and prevalence of BIS have seemed to have spurred methodo-logic efforts and sophistication of anesthetic control systems.⁶⁶

Several studies also have proposed and evaluated alternative AI algorithms for depth of anesthesia versus existing measures, including BIS or the response entropy index. Although BIS is a commonly used modality for measuring depth of anesthesia, by no means is it the only modality available and it is possible that better measures of depth are achievable. Others have utilized auditory evoked potentials and heart rate variability.^67,68 Mirsadeghi and colleagues⁶⁴ utilized a method called locally linear embedding, which maps high-dimensional features into a 2-dimensional output space to input direct features from EEG signals. They showed a 88.4% accuracy compared with 84.2% by BIS of identifying awake versus anesthetized patients. Another study by Shalbaf and colleagues⁶⁵ applied EEG features within a neural network model to discriminate different states of anesthetic depth and demonstrated 93% accuracy compared with the BIS index’s 87% accuracy.

Another exciting target in depth of anesthesia monitoring is identification and potential prediction of awareness events. One study by Ranta and colleagues⁶⁸ examined cases of a cohort who had reported intraoperative awareness while under general anesthesia and deployed neural networks using blood pressure, heart rate, and end-tidal carbon dioxide as input features. Although the prediction probability was 66%, the specificity achieved 98%, even with utilization of no EEG features.

Control of Anesthesia

The potential impact of controlled anesthetic delivery systems is vast. Although much emphasis often is placed on depth of anesthesia monitoring in anesthetic control, many forward-thinkers propose systems of complete anesthetic control. A complete system for closed loop control of anesthesia would, in an ideal world, monitor and control hypnosis, nociception, neuromuscular blockade, hemodynamics, ventilation, temperature, and metabolic targets.⁶⁹

Although many aspects of such a system may not require AI, there are many potential places for useful application of such modalities. Not all closed loop systems require AI at all; for example, closed loop temperature control is unlikely to require AI. These methodologies, however, have been applied to helpful use not only in depth of anesthesia systems, discussed previously, but also in systems for maintaining neuromuscular blockade goal as well as systems for controlling or weaning ventilation.^70-75

Event and Risk Prediction

This area is ripe for application of AI modalities and many studies already exist in this area. Models exist for myriad events in the intraoperative, postoperative, and critical care periods. Examples include postinduction hypotension, hypnotic effect of induction dose of propofol, rate of recovery from neuromuscular blockade, American Society of Anesthesiologists (ASA) status, difficult laryngoscopy, identifying respiratory depression during conscious sedation, and assistance in decision making for the optimal method of anesthesia in pediatric surgery.^76-84 Critical care studies have used machine learning to predict morbidity, ventilator weaning, clinical deterioration, mortality readmission, and detection of sepsis.^85-91

In the arena of perioperative risks, investigators Hill and colleagues⁹² recently described a random forest model that predicts postoperative in-hospital mortality based solely on automatically obtained preoperative features with an area under the curve of 0.932. This model was found to markedly outperform other predictors of mortality, such as the Preoperative Score to Predict Postoperative Mortality, Charlson Co-morbidity Index, and ASA physical status. Additionally, deep neural networks have been applied to large data sets in order to predict other common markers of poor postoperative outcomes, such as mortality, readmission, acute kidney injury, and reintubation.^21,93

Ultrasound Guidance

Studies in ultrasound guidance utilizing AI primarily have utilized neural network techniques. Examples include application of neural networks to distinguish femoral artery versus vein and automated identification of vertebral lamina.^94,95 Several studies exist with cardiac echocardiograms as well.⁹⁶ Even in cases of differentiation that seems easy or no better than expert clinical judgment, such simple discriminating abilities potentially could be vastly important and change how to teach trainees learning new imaging modalities.

Pain Management

AI also has significant potential to improve how we understand and treat pain. Examples include development of a nociception-level index based on machine learning analysis of photoplethysmograms and skin conductance waveforms, prediction of opioid dosing, and identification of patients who may benefit from preoperative consultation with a hospital’s acute pain service.^97-99 AI also has been deployed to great success in large data studies with the aim of personalization of medicine.¹⁰⁰ This holds potential to optimize drug selection, dosing, and adverse reactions as well as identifying patients at risk for prolonged opioid use and substance use disorders.

Operating Room Logistics

Just as AI has infiltrated business and management, it also has begun to be applied to operating room and hospital logistics. Examples include prediction of the duration of surgical procedures and optimization of bed use.^22,101-103 Machine learning models have been applied successfully to estimating the duration of robotic-assisted surgery.²² Surgical robotic units are a costly and limited resource; thus, booking robotic cases more accurately represents an example with significant potential business value.

CHALLENGES AND FUTURE DIRECTIONS OF CLOSED LOOP DEVICES AND ARTIFICIAL INTELLIGENCE

Currently, the field of anesthesiology appears to be on an exciting cusp of practical application of AI and closed loop systems. The strides that have been made in computational techniques and hardware, as well as the formation and availability of rich clinical databases, have laid hugely important groundwork. Many experimental systems have been proposed for closed loop control of anesthesia and practical applications of AI in anesthesiology, and the sophistication and intelligent algorithm design and practicalities are improving at lightning speed. Although none of the experimental closed loop systems is commercially available to clinicians at this time, several systems have undergone substantial study and show significant promise, such as the closed loop anesthesia delivery system and Infusion Toolbox 95. To further push for closed loop innovation, significant engagement with regulatory bodies, control, systems, and software engineering will be necessary in the coming future.

With closed loop systems as well as machine learning and neural networks, the next steps seem to be application and demonstration of clear clinical impact and improvement in outcomes. Although numerous studies have shown impressive ability to analyze and predict clinical attributes and outcomes, convincing demonstrations of improved clinically significant outcomes have yet to be seen. Only after clear clinical impact is demonstrated can the cost and other hurdles required of implementation be justified. With the speed of advancement in the past several years, however, studies with demonstration of direct improvement in clinical outcomes are expected to emerge in the near future.

Anesthesiology as a field is genuinely uniquely suited to reap potential benefits and improvements that AI can offer. Anesthesiologists in the operating room are bombarded with digital data points from monitors and anesthesia machines, from within the electronic medical records, and from anesthesia information management systems. This makes incorporation of all these data points all the more challenging in addition to being subject to bias and fatigue. The beauty of AI methodologies is the algorithms’ ability to eliminate potential biases and engage in self-learning rather than needing to be fed the features decided on by expert opinion.

At the same time, these seemingly endless digitized data sources enable having access to rich databases from which how to care for patients can be learned and improved. Similar to the revolution in computing abilities and technology, a revolution in these technologies within anesthesiology is expected in the coming years.

KEY POINTS.

• The application of artificial intelligence in anesthesiology with machine learning, neural networks, and closed loop devices has been advancing in frequency, scope, and sophistication.

• This article summarizes some basic tenets of machine learning (supervised, unsupervised, and reinforcement learning), techniques in artificial intelligence (classical machine learning, neural networks, deep learning, and bayesian methods), and applications of these modalities in clinical anesthesiology.

• This article reviews some history and background of closed loop devices, basic tenets of design and engineering of these devices, and their clinical applications.

• Artificial intelligence has the potential to have an impact on the practice of anesthesiology in aspects ranging from perioperative support to critical care delivery to outpatient pain management.

CLINICS CARE POINTS.

• Given the large amount of data anesthesiologists are required to interpret and prioritize, many closed loop systems and AI modalities have significant potential to reduce unwanted variability in patient care as well as deleterious effects of biases and human error.

• Closed loop systems can be applied to a variety of clinical aspects of the practice of anesthesiology: administration of anesthetics, IV fluids, and vasopressors as well as mechanical ventilation and glucose control.

• AI algorithms have been applied to several clinical arenas within anesthesiology: assessment of anesthetic depth, control of anesthesia, prediction of perioperative events and risks, ultrasound guidance, pain management, and operating room logistics.