Machine Learning Implementation in Clinical Anesthesia: Opportunities and Challenges

Recent Food and Drug Administration (FDA) approval of the first autonomous, diagnostic system¹ heralds the arrival of clinical machine learning (ML). ML is a promising form of artificial intelligence best suited to, but also necessary for, the predictive analytics required for clinical decision-making.^2,3 ML focuses on the development of computer systems that can learn from big data (or data that is of such volume, collection velocity or complexity that it is difficult or impossible to process using traditional methods⁴), identify patterns and make decisions with minimal human intervention.⁵ As ML tools begin to be designed and targeted for clinical anesthesia applications, there will be growing pressure for anesthesiologists to clarify when and how clinicians add value, versus when ML can (and perhaps should) augment clinical practice and clinical decision-making (Table).⁶

Table.

Potential Benefits, challenges and current limitations to implementing Machine Learning (ML) into clinical anesthesia

Potential Benefits	Potential Challenges	Current Limitations
Reduce clinician cognitive load	Clinical skill atrophy: -Maintenance of emergency manual skills - Maintenance of cognitive skills	ML cannot ‘contextualize’ to bedside care of individual patient
Reduction in costs of care Increased access to care (e.g. remote care delivery)	Examination of impact on clinical work flow and work processes (ex. 737 Max) to prevent unintended safety events	Manual tasks (i.e. intubation, vascular access) not easily replaced by machine
Improved evidence supporting care recommendations, through ‘big data’ and real time analytics	Impact on clinician autonomy and clinician-patient relationship	Access to necessary ‘big data’ still being established Bias in data and analysis can have unintended negative consequences
Standardization of care (reduction in care variation between clinicians, clinical centers)	Accountability for ML output or clinical actions undertaken as a result of ML output	Emerging regulation: -Access to Patient Data: privacy protections & data ownership -Set standards to assess and evaluate ML accuracy -Legal liability

For over half a century, progressively shorter acting drugs and improvements in patient monitoring technologies fueled interest in anesthesia delivery as a target for automation.⁷ ML guided anesthesia has already been piloted, including models of remifentanil and propofol interactions with processed electroencephalograms.⁸ In addition to a beneficial impact on quality, cost, and access to care, ML applications for clinical anesthesia will raise unique value-based, ethical challenges, and disrupt established workflow processes, raising safety concerns.⁹ Premature ML implementation causes patient harm.¹⁰ Clinical anesthesiologists are uniquely positioned to consider such systems as they are developed and implemented, working to promote the benefits of ML and reduce potential harms. As pioneers of patient safety, now is the time to consider how anesthesiologists should interact with, define our relationship to, and guide implementation of novel ML systems.

Significant private investment,¹¹ strong research interest, and compatibility with social goals of health care cost reduction all drive continued advancement of ML into healthcare, including clinical anesthesia.¹² Healthcare collaborations such as between Google’s DeepMind and the United Kingdom’s National Health Service, Paige AI and Memorial Sloan Kettering and the International Business Machines (IBM) Corporation’s Watson Oncology and MD Anderson, have all raised ethical concerns. Despite these challenges, global investment in ML for healthcare is predicted to reach $217 billion by 2028.¹¹ To match the speed of development, both ethical and practical guidance for clinical ML implementation needs to be conducted and provided contemporaneously.

As anesthesiologists approach clinical ML implementation, four areas are important to consider: 1) impact on workflow, 2) skill atrophy, 3) accountability, and 4) clinician autonomy.

Impact on Workflow

First, the impact of ML on anesthesia clinical workflow and work processes requires extensive examination. Significant safety and judgment failures have already occurred around implementation of ML systems or output for work processes in non-healthcare contexts. Recent Boeing 737 Max and Tesla Model S crashes are attributable to inadequate assessment of the impact of automated systems on workflow and work processes. In both cases, operators’ lack of familiarity with the automated piloting systems and using them outside their intended design, led to catastrophic adverse events.^13,14 These failures have raised awareness about the potential for ML approaches to cause negative disruptive change within medicine.⁹ These include the potential for similar failures, particularly around clinician and patient interactions with ML systems, and with inadequate in situ assessment of the ML impact on operators and work processes, leading to patient harm.¹⁰ The dynamic and high-stakes clinical environment within anesthesia workflow is vulnerable.

Skill Atrophy

Second, as new technologies replace manual or cognitive tasks, consequent atrophy or loss of those skills occurs. In anesthesia, where a patient’s life may depend on an anesthesiologist’s ability to re-take control from an automated system, maintaining some clinical and cognitive skill will be necessary. Anesthesiologists’ over-reliance on automated anesthesia machine ‘self-check’ systems, has led to patient harm when the automated check failed to identify circuit obstruction.¹⁵ Which clinical skills are paramount and need to be protected from loss should be determined and prioritized.

Literature from non-healthcare, performance-based fields like aviation, recognize the growing challenges involved in maintaining critical emergency skills when operators are routinely functioning in progressively more automated contexts.¹⁶ Most concerning is that, after practicing in largely automated contexts, while pilots’ manual skills to fly by hand largely remain intact (with only moderate, operationally significant “rustiness”) fundamental cognitive skills atrophied significantly, including awareness of plane location, to reference charts, to configure the airplane anew after passing important way-points on a planned route, and to recognize and deal with instrument system failures when they arose.

Recommendations to address these problems all center on increasing pilots’ time practicing these skills, either through repeated simulations or through real-time practice.¹⁶ Unfortunately, co-following or co-flying with an automated system appears to be ineffective at preventing cognitive skill atrophy, with accumulating evidence of the difficulty in pilots maintaining thoughts focused on the activities of an automated system that seldom fails.¹⁶

Simulator training has proven valuable for training anesthesiologists in crisis resource management and later performance in non-simulated crises. However, simulation training to address ML implementation presents several challenges. The first is verisimilitude. For both pilots and clinicians, high fidelity simulator training is necessary to maximize the likelihood that simulation training will cognitively transfer to real environments.¹⁷ Such simulators are costly to construct, maintain, and operate, and provide no guarantee of skill transfer. Since the clinician-computer interactions and points of interface for clinical ML are still being established, simulators and simulations will be unable to depict high fidelity ML-clinician interactions until ML implementation is further established.

An additional, more salient, concern is that scenarios included in simulation training are based on problems that will likely be recognizable to, predictable to (and ultimately addressable by) increasingly complex ML systems. By definition, simulation scenarios are pre-identifiable as likely sources of clinical problems. The real problems anesthesiologists will face and be called to ‘”take over” during would be catastrophic unexpected events that may be difficult to train for without extensive direct clinical experience. This is similar to the performance differences seen between how military-trained or senior pilots were able to compensate for the errors with the Boeing 737 Max MCAS system, while junior, simulator-trained pilots were not as easily able to.¹³ Analyses of Capt. Sullenberger’s landing of the Airbus A320 (U.S. Airways Flight 1549) in the Hudson also showed the importance of experience and judgment relative to how less experienced pilots handled the same situation in simulation.¹⁸

Collecting the necessary knowledge of ML-system failures in order to train clinicians for ML-related crisis training will take time. How much can be predicted from the aviation experience is unknown but, in abstraction, events like the 737 Max are already very valuable for identifying broad target areas. We should ensure clinician familiarity with ML systems prior to clinical deployments, rather than wait for failures to inform training.

If our field decides that maintenance of direct, hands-on patient experience is needed to maintain clinical competency and the ability to “take over,” how many hours, and what types of cases will need to be studied, as do implications for patient care (i.e. how to decide whether a patient receives ML-supported anesthesia or provider-only anesthesia). The aviation field has long recognized that maintenance of skills requires more than simply logging the legally mandated number of flight hours in clear skies. Facing challenging flight conditions is also needed.

Accountability

Third, increasing reliance on ML tools and patient ‘big data’ will impact the physician-patient dyad that has constituted the ethical underpinning of the fiduciary caregiving relationship. This relationship is likely to even further shift into a relationship between patients and a learning healthcare system. Recent ethical concerns around ML applications also indicate that applications in healthcare could raise accountability concerns.⁹ Designers of autonomous systems for healthcare, such as diabetic retinopathy screening, have expressed willingness to assume responsibility for a system’s output (since it is, after all, intended to function autonomously).¹ Because of the potential need for rescuing interventions, it is unlikely that anesthetic delivery systems would ever function fully autonomously, without clinician supervision. However, what accountability, and therefore liability, lies with the anesthesiologist versus with the ML system needs to be established.

Clinician Autonomy

Fourth, the ongoing transition to systems-based anesthesia delivery, including broad adoption of electronic medical record systems (EMRs) and ongoing transition to a shift-based work model, impacts clinician autonomy. ML application to clinical anesthesia has the potential to become the tipping point where a quantitative difference in autonomy becomes a qualitative problem. Whether due to ML exceptionalism (the belief that a result is inherently better because it was produced by a computer) or because the operating room environment has become too data-overwhelmed and clinicians too distracted, ML output may take on an authority never intended. As is already occurring with electronic medical records (EMRs), anesthesiologists may find themselves progressively drawn into a clinical workflow focused on data entry, addressing data output, and reacting to alarms generated by algorithms rather than focusing on the actual patient. It is already recognized as a problem in non-healthcare fields that a person disagreeing with an ML-recommended action is often required to furnish far more and better quality evidence to rebut the ML output than the data used to generate that the output. Such barriers to ML disagreement discourages workers questioning algorithmic output.⁶

Over the past 20 years, American healthcare has seen the rise of a non-clinical, executive class.¹⁹ What bedside clinicians are likely to most value in an ML tool, is unlikely to match what the non-clinician purchasers of ML tools are likely to value. While clinical guidance provided by EMRs and potentially ML systems may improve aspects of care by increasing compliance with evidence-based approaches, ML-driven alarms and guidance may also be used to control a clinician workforce in pursuit of optimizing re-imbursement driven performance metrics and cost impacts of care choices on financial returns.

Current Limitations to ML Implementation

Currently, there are still significant limitations to ML-based anesthesia delivery. Manual tasks fundamental to the delivery of anesthetic care, such as intubation and vascular access, are not yet easily replaced by machines.⁷ Capture of the necessary ‘big data’ on drug delivery and patient physiologic effects still needs to be established in order for ML-targeted drug delivery to improve on current pharmacokinetic and pharmacodynamics models.¹² For clinical knowledge and decision support, with real-time access to current evidence based data, ML-systems are situated to recommend evidence-based clinical actions where data exists, with greater perspective than any individual clinician. However, such systems lack the ability to contextualize a clinical decision to the care of an individual patient. Currently, such systems are better deployed in support of clinician knowledge, rather than as clinician replacement. Capture and analysis of available data is still only, at best, observational data, with all of the inherent limitations of observational study design. Exploration of novel trial designs to integrate research with medical practice and learning health systems are underway.²⁰

Approaches to regulating ML for healthcare are emerging, though far slower than the technology is changing. In the United States, the FDA has recognized that their traditional paradigm of medical device regulation was not designed for adaptive artificial intelligence and machine learning technologies and is currently designing procedures to guide premarket review of proposed clinical ML applications.³ In concept, such review will evaluate that an ML application performs as intended, i.e. that the prediction the ML generates is accurate and any clinical action it undertakes or recommends, efficacious.¹

The rising awareness of the need for access to a large, patient data ecosystem to fuel ML development is being balanced against patient data privacy concerns. With Europe as the vanguard, legal reforms on data protection and privacy are underway in many countries. For example, the European Union has adopted a General Data Protection Regulation (EU 2016/679). Such reforms attempt to increase data subjects’ privacy options and introduce further controls on data uses. These regulations on access to data are covering not only data protection, but also the distribution of any benefits of the exploitation of personal data and the public acceptability of such exploitation (i.e the questions of whether patients have a stake in applications designed from their data and, such as with the Memorial Sloan Kettering-Paige AI, whether clinicians have intellectual property rights to their clinical interpretations of data (such as slide reads by pathologists) used to train ML applications).

These evolving regulatory approaches will address important safety concerns around ML accuracy, patient data privacy protections and data ownership. They will not address the workflow and human/ML interface challenges significant for the practice of anesthesia.

In the near future, clinicians will likely collaborate with and manage ML-systems that aggregate vast amounts of data, generate diagnostic and treatment recommendations, and assign confidence ratings to those recommendations. Systems have already been designed to leverage aggregate patient data for decision-making at the point of care. This integration expands the data to support clinical decisions beyond published studies or even raw data that could be available to an individual clinician. As ML’s influence on the practice of anesthesia approaches, we must thoughtfully and carefully examine how our field will address ML, what impacts we want ML tools to have on clinical anesthesia, what research on ML is needed, and how to anticipate and prevent potential harms to patients and clinicians.

Glossary of Terms:

EMR

Electronic Medical Record

FDA

Food and Drug Administration

IBM

International Business Machines Corporation

ML

Machine Learning