Artificial Intelligence and Surgical Decision-Making

Abstract

IMPORTANCE

Surgeons make complex, high-stakes decisions under time constraints and uncertainty, with significant effect on patient outcomes. This review describes the weaknesses of traditional clinical decision-support systems and proposes that artificial intelligence should be used to augment surgical decision-making.

OBSERVATIONS

Surgical decision-making is dominated by hypothetical-deductive reasoning, individual judgment, and heuristics. These factors can lead to bias, error, and preventable harm. Traditional predictive analytics and clinical decision-support systems are intended to augment surgical decision-making, but their clinical utility is compromised by time-consuming manual data management and suboptimal accuracy. These challenges can be overcome by automated artificial intelligence models fed by livestreaming electronic health record data with mobile device outputs. This approach would require data standardization, advances in model interpretability, careful implementation and monitoring, attention to ethical challenges involving algorithm bias and accountability for errors, and preservation of bedside assessment and human intuition in the decision-making process.

CONCLUSIONS AND RELEVANCE

Integration of artificial intelligence with surgical decision-making has the potential to transform care by augmenting the decision to operate, informed consent process, identification and mitigation of modifiable risk factors, decisions regarding postoperative management, and shared decisions regarding resource use.

Surgeons make complex, high-stakes decisions when offering an operation, addressing modifiable risk factors, managing complications and optimizing resource use, and conducting an operation. Diagnostic and judgment errors are the second most common cause of preventable harm incurred by surgical patients.¹ Surgeons report that lapses in judgment are the most common cause of their major errors.² Surgical decision-making is dominated by hypothetical deductive reasoning and individual judgment, which are highly variable and ill-suited to remedy these errors. Traditional clinical decision support tools, such as the National Surgical Quality Improvement Program (NSQIP) Surgical Risk Calculator, can reduce variability and mitigate risks, but their clinical adoption is hindered by suboptimal accuracy and time-consuming manual data acquisition and entry requirements.^3–8

Although decision-making is one of the most difficult and important tasks that surgeons perform, there is a relative paucity of research investigating surgical decision-making and strategies to improve it. The objectives of this review are to describe challenges in surgical decision-making, review traditional clinical decision-support systems and their weaknesses, and propose that artificial intelligence models fed with live-streaming electronic health record data (EHR) would obviate these weaknesses and should be integrated with bedside assessment and human intuition to augment surgical decision-making.

Methods

PubMed and Cochrane Library databases were searched from their inception to February 2019 (eFigure in the Supplement). Articles were screened by reviewing their abstracts for the following criteria: (1) published in English, (2) published in a peer-reviewed journal, and (3) primary literature or a review article. Articles were selected for inclusion by manually reviewing abstracts and full texts for these criteria: (1) topical relevance, (2) methodologic strength, and (3) novel or meritorious contribution to existing literature. Articles of interest cited by articles identified in the initial search were reviewed using the same criteria. Forty-nine articles were included and assimilated into relevant categories (Table 1).^1–49

Table 1.

Summary of Included Studies

Source	Study Design	Population	Sample Size	Major Findings Pertinent to This Scoping Review	Sources of Funding; Conflicts of Interest
Adhikari et al9	Retrospective	Patients undergoing inpatient surgery	2911	A machine learning algorithm accurately predicted postoperative acute kidney injury using preoperative and intraoperative data	NIGMS, University of Florida CTSI; NCATS; I Heerman Anesthesia Foundation; SCCM Vision Grant
Artis et al10	Observational	Trainee ICU presentations	157	Potentially important data were omitted from 157 of157 presentations; missing an average 42% of all data elements	AHRQ
Bagnall et al11	Retrospective	Patients who had colorectal surgery	1380	Six traditional risk models used to predict postoperative morbidity and mortality had weak accuracy with AUC, 0.46–0.61	St Mark’s Hospital Foundation
Bechara et al12	Observational	Healthy volunteers and participants with prefrontal cortex damage	16	Participants began to decide advantageously before they could consciously explain what they were doing or whythey were doing it	National Institute of Neurological Diseases and Stroke
Bertrand et al13	Prospective	ICU patients and attending surgeons	419	Clinicians thought 45% of all patients had decision-making capacity; a minimental status examination found that 17% had decision-making capacity	Pfizer, Fisher & Paykel; Pfizer; Alexion; Gilead; Jazz Pharma; Baxter; Astellas
Bertsimas et al14	Retrospective	Emergency surgery patients	382 960	A app-based machine learning model accurately predicted mortality and 18 postoperative complications (AUC, 0.92)	None reported
Bihorac et al15	Retrospective	Patients undergoing major surgery	51 457	A machine learning algorithm using automated EHR data predicted 8 postoperative complications (AUC, 0.82–0.94)and predicted mortality at 1,3, 6,12, and 24 mo (AUC, 0.77–0.83)	NIGMS; NCATS
Blumenthal-Barby et al16	Review	Articles about heuristics in medical decision-making	213	Among studies investigating bias and heuristics among medical personnel, 80% identified evidence of bias and heuristics	Greenwall Foundation; Pfizer
Brennan et al17	Prospective	Physicians	20	A machine learning algorithm was significantly more accurate than physicians in predicting postoperative complications (AUC, 0.73–0.85 vsAUC, 0.47–0.69)	NIGMS, University of Florida CTSI; NCATS
Che et al18	Retrospective	Pediatric ICU patients	398	A gradient boosting trees method allowed for quantification of the relative importance of deep model inputs in determining model outputs	NSF; Coulter Translational Research Program
Chen-Ying et al19	Retrospective	Clinic patients	840 487	A deep model predicted 5-y stroke occurrence with greater sensitivity (0.85 vs 0.82) and specificity (0.87 vs 0.86) than logistic regression	None reported
Christie et al20	Retrospective	Trauma patients	28212	A machine learning ensemble accurately predicted mortality among trauma patients in the United States, South Africa, and Cameroon with AUC ≥0.90 inall settings	NIH
Clark et al4	Retrospective	Surgical patients	885 502	The ACS Surgical Risk Calculator accurately predicted mortality (AUC, 0.94) and morbidity (AUC, 0.83)	None reported
Cohen et al6	Retrospective	Studies assessing the ACS Surgical Risk Calculator	3	Externalvalidation studies assessing ACS Surgical Risk Calculator performance may have been compromised by small sample size, case-mix heterogeneity, and use of data from a small number of institutions	None reported
Delahanty et al21	Retrospective	ICU patients	237 173	A machine learning algorithm accurately predicted inpatient death (AUC, 0.94)	Alesky Belcher; Intensix, Advanced ICU
Dybowski et al22	Retrospective	ICU patients	258	An artificial neural network predicted in-hospital mortality more accurately than logistic regression (AUC 0.86 vs 0.75)	Special Trustees for St Thomas’ Hospital
Ellis et al23	Prospective	Volunteers	1948	Induction of fear and anger had unique and significant influences on decisions to take hypothetical medications	NCI
Gage et al24	Prospective	Patients with atrial fibrillation	2580	Models commonly used to predict risk of stroke were moderately accurate with AUC ranging from 0.58–0.70	AHA, NIH, Danish and Netherlands Heart Foundations, Zorg Onderzoek Nederland Prevention Fund, Bayer, UK StrokeAssociation
Gijsberts et al25	Retrospective	Patients with no baseline cardiovascular disease	60 211	Associations between risk factors and development of atherosclerotic cardiovascular disease were different across racial and ethnic groups	Netherlands Organization for Health Research and Development, NIH
Hao et al26	Retrospective	ICU patients	15 647	Deep learning models predicted 28-d mortalitywith 84%−86%accuracy	None reported
Healey et al1	Retrospective	Surgical inpatients	4658	Behind technical errors, diagnostic and judgment errors were the second most common cause of preventable harm	None reported
Henry et al27	Retrospective	ICU patients	16234	A machine learning early warning score accurately predicted the onset of septic shock (AUC 0.83), identifying approximately two-thirds of all cases prior to the onset of organ dysfunction	NSF; Google Research, Gordon and Betty Moore Foundation
Hubbard et al28	Prospective	Trauma patients	980	A machine learning ensemble predicted mortalitymore accurately than logistic regression (5% gain)	US Army Medical Research and Materiel Command, NIH
Hyde et al7	Prospective	Patients undergoing colorectal resections	288	The likelihood of a serious complication was underestimated bythe ACS Surgical Risk Calculator (AUC, 0.69), but the calculator accurately predicted postoperative mortality (AUC, 0.97)	None reported
Kim et al29	Retrospective	ICU admissions	38 474	A decision tree model predicted in-hospital mortalitymore accurately than APACHE III (AUC, 0.89 vs 0.87)	NCRR
Knops et al30	Systematic review	Studies about decision aids in surgery	17	Decision aid use was associated with more knowledge regarding treatment options and preference for less invasive treatment options with no observable differences in anxiety, quality of life, or complications	None reported
Komorowski et al31	Retrospective	Septic ICU patients	96 156	A reinforcement learning model recommending intravenous fluid and vasopressor strategies outperformed human clinicians; mortality was lowest when decisions made by clinicians matched recommendations from the reinforcement learning model	Orion Pharma, Amomed Pharma, Ferring Pharma, Tenax Therapeutics; Baxter Healthcare; Bristol-Myers Squibb; GSK; HCA International
Koyner et al32	Retrospective	Hospital admissions	121 158	A machine learning algorithm accurately predicted development of acute kidney injury within 24 h (AUC, 0.90) and 48 h (AUC, 0.87)	Satellite Healthcare; Philips Healthcare; EarlySense; Quant HC
Leeds et al8	Observational	Surgery residents	124	Residents reported that lack of electronic and clinical workflow integration were major barriers to routine use of risk communication frameworks	NCI; ASCRS; AHRQ
Legare et al33	Systematic review	Studies about shared decision-making	38	Time constraints impair the shared decision-making process among providers, patients, and caregivers	Tier 2 Canada Research Chair
Loftus et al34	Retrospective	Patients with lower intestinal bleeding	147	An artificial neural network predicted severe lower intestinal bleeding more accurately than a traditional clinical prediction rule (AUC, 0.98 vs 0.66)	NIGMS, NCATS
Lubitz et al5	Retrospective	Patients undergoing colorectal surgery	150	The ACS Surgical Risk Calculator accurately predicted morbidityand mortalityfor elective surgery but underestimated risk for emergent surgery	None reported
Ludolph et al35	Systematic review	Articles about debiasing in health care	68	Many debiasing strategies targeting health care clinicians effectively decrease the effect of bias on decision-making	University of Lugano Institute of Communication and Health
Lundgren-Laine et al36	Observational	Academic intensivists	8	Academic intensivists made approximately 56 ad hoc patient care and resource use decisions per day	Finnish Funding Agency for Technology and Innovation; Tekes; Finnish Cultural Foundation
Morris et al37	Interviews	Academic surgeons	20	Younger surgeons felt uncomfortable defining futility and felt pressured to perform operations that were likely futile	AHRQ
Pirracchio et al38	Retrospective	ICU patients	24508	A machine learning ensemble predicted in-hospital mortality (AUC, 0.85) more accurately than SAPS-II (AUC, 0.78) and SOFA (AUC, 0.71)	Fulbright Foundation; Doris Duke Clinical Scientist Development Award; NIH
Pirracchio et al39	Observational	Simulated data sets	1000	A machine learning ensemble predicted propensity scores more accurately than logistic regression and individual machine learning algorithms	Fulbright Foundation; Assistance Publique-Hopitaux de Paris; NIH
Raymond et al3	Prospective	Preoperative clinic patients	150	After reviewing ACS Surgical Risk Calculator results, 70% would participate in prehabilitation and 40% would delay surgery for prehabilitation	GE Foundation; Edwards Lifesciences; Cheetah Medical
Sacks et al40	Observational	Surgeons	767	Facing clinical vignettes for urgent and emergent surgical diseases; surgeons exhibited wide variability in the decision to operate (49%−85%)	Robert Wood Johnson/Veterans Affairs Clinical Scholars program
Schuetz et al41	Retrospective	Clinical encounters in an EHR	32 787	A deep model predicted the onset of heart failure more accurately than logistic regression (AUC, 0.78vsAUC, 0.75)	NSF; NHLBI
Shanafelt et al2	Observational	Members of the ACS	7905	Nine percent of all surgeons reported making a major medical error in the last 3 mo, and lapses in judgment were the most common cause (32%)	None reported
Shickel et al42	Retrospective	ICU admissions	36216	A deep model using SOFA variables predicted in-hospital mortality with greater accuracy than the traditional SOFAscore (AUC 0.90 vs 0.85)	NIGMS; NSF, University of Florida CTSI; NCATS; J Crayton Pruitt Family Department of Biomedical Engineering; Nvidia
Singh et al43	Systematic review	Articles about CRP to predict leak after colorectal surgery	7	The positive predictive value of serum C-reactive protein 3–5 d after surgerywas 21%−23%	Auckland Medical Research Foundation, New Zealand Health Research Council
Stacey et al44	Systematic review	Randomized trials about decision aids	105	Participants exposed to decision aids felt that they were more knowledgeable, informed, and clear about their values and played a more active role in the shared decision-making process	Foundation for Informed Medical Decision Making, Healthwise
Strate et al45	Prospective	Patients with acute lower intestinal bleeding	275	A bedside clinical prediction rule using simple cutoff values predicted severe lower intestinal bleeding (AUC, 0.75)	American College of Gastroenterology, National Research Service Award, American Society for Gastrointestinal Endoscopy
Sun et al46	Observational	Simulated type 1 diabetics	100	A reinforcement learning model performed as well as standard intermittent self-monitoring and continuous glucose monitoring methods, but with fewer episodes of hypoglycemia	Swiss Commission of Technology and Innovation
Van den Bruel et al47	Retrospective	Primary care patients	3890	Clinician intuition identified patients with illness severity that was underrepresented bytraditional clinical parameters	Research Foundation Flanders, Eurogenerics, NIHR
Van den Bruel et al48	Systematic review	Articles about clinical parameters for serious infections	30	Traditional clinical parameters associated with serious infection were often absent among patients with serious infections	Health Technology Assessment, NIHR
Vohs et al49	Observational	Undergraduate students	34	Higher decision-making volume was associated with decreased physical stamina, persistence, qualityand quantity of mathematic calculations, and more procrastination	NIH, Social Sciences and Humanities Research Council, Canada Research Chair Council, McKnight Land-Grant

Abbreviations: ACS, American College of Surgeons; AHA, American Heart Association; AHRQ, Agency for Healthcare Research and Quality; ASCRS, American Society of Colon and Rectal Surgeons; AUC, area under the curve; CRP, C-reactive protein; CTSI, Clinical and Translational Sciences Institute; EHR, electronic health record; GSK, Glaxo Smith Kline; ICU, intensive care unit; NCATS, National Center for Advancing Translational Sciences; NCI, National Cancer Institute; NCRR, National Center for Research Resources, Acute Physiology, and Chronic Health Evaluation; NHLBI, National Heart, Lung, and Blood Institute; NIGMS, National Institute of General Medical Sciences; NIH, National Institutes of Health; NIHR, National Institute for Health Research; NSF, National Science Foundation; SAPS, Simplified Acute Physiology Score; SCCM, Society of Critical Care Medicine; SOFA, Sequential Organ Failure Assessment.

Observations

The quality of surgical decision-making is influenced by patient values and emotions, patient-surgeon interactions, decision-making volume and complexity, time constraints, uncertainty, hypothetical deductive reasoning, and individual judgment. There are effective and ineffective methods for dealing with each of these factors, which lead to positive and negative outcomes, respectively (Figure 1).

Figure 1. Surgical Decision-Making Paradigm.

Challenges in Surgical Decision-Making

Complexity

In the hypothetical-deductive decision-making model that dominates surgical decision-making, initial patient presentations are assessed to develop a list of possible diagnoses that are differentiated by diagnostic testing or response to empirical therapy. This depends on the surgeon’s ability to form a complete list of all likely diagnoses, all life-threatening diagnoses, and all unlikely diagnoses that may be considered if the initial workup excludes other causes. It also requires recognition of strengths and limitations of available tests. Once the diagnosis is established, the surgeon must recommend a plan using sound judgment. Each step introduces variability and opportunities for error.⁴⁰

Values and Emotions

Patient values are individualized by nature, precluding the creation of a criterion standard of optimal decision-making. Understanding and incorporating these values is essential to an effective shared decision-making process.⁵⁰ This may be accomplished by simply asking patients and caregivers about their goals of care and what they value most in life. Shared decision-making improves patient satisfaction and compliance and may reduce costs associated with undesired tests and treatments. However, patients, caregivers, and clinicians often misunderstand one another, their goals may differ, and patients and caregivers are often expected to make decisions with limited background knowledge and no medical training.^13,33,50 Surgical diseases may evoke fear and anger, which influence perceptions of risks and benefits.^23,51 Emotions surrounding an acute surgical condition may also create a sense of urgency and pressure on surgeons to perform futile operations.³⁷

Time Constraints and Uncertainty

Surgical decision-making is often hindered by uncertainty owing to missing or incomplete data. This occurs when decisions regarding an urgent or emergent condition must be made before all relevant data can be gathered and analyzed. Nonurgent decisions may be hindered by time constraints and uncertainty owing to sheer decision-making volume, the time-consuming nature of manual data acquisition, and team dynamics. Academic intensivists make approximately 56 patient care and resource use decisions per day.³⁶ In an assessment of medical student and resident intensive care unit (ICU) patient presentations, potentially important data were omitted from 157 of 157 presentations.¹⁰ Even when data collection and analysis are complete, high decision-making volume begets decision fatigue, manifesting as procrastination, less persistence when facing adversity, decreased physical stamina, and lower quality and quantity of mathematic calculations.⁴⁹ These impairments are exacerbated by acute and chronic sleep deprivation, which occurs in as many as two-thirds of all acute care surgeons taking in-house call.^52,53 For a surgical oncologist with a busy outpatient clinic, automated production of prognostic data from artificial intelligence models could improve efficiency and preserve face-to-face patient-surgeon interactions by obviating manual data acquisition and entry into prognostic models.

Heuristics and Bias

When facing time constraints and uncertainty, decision-making may be influenced by heuristics or cognitive shortcuts.^54,55 Heuristics may lead to bias or predictable and systematic cognitive errors, as described in Table 2.^16,35

Table 2.

Sources of Bias in Surgical Decision-Making

Source of Bias	Examples
Framing effect	Aclinician presents a clinical scenario to a surgeon in different context than the surgeon would have perceived during an independent assessment
Overconfidence bias	Asurgeon falsely perceives that weaknesses and failures disproportionatelyaffect their peers
Commission bias	Asurgeon tends toward action when inaction may be preferable, especially in the context of overconfidence bias
Anchoring bias	Patients are informed of expected outcomes using data from aggregate patient populations without adjusting for their personalized risk profile
Recall bias	Recent experiences with a certain patient population or operation disproportionately affect surgical decision-making relative to remote experiences
Confirmation bias	Outcomes are predicted using personal beliefs rather than evidence-based guidelines

Traditional Predictive Analytics and Clinical Decision Support

Decision Aids

Decision aids provide specific patient populations with background information, options for diagnosis and treatment, risks and benefits for each option, and outcome probabilities. In a systematic review⁴⁴ including 31 043 patients facing screening or treatment decisions, patients exposed to decision aids felt more knowledgeable and played a more active role in the decision-making process. In a systematic review of 17 studies investigating decisions made by surgical patients, decision aids were associated with more knowledge regarding treatment options, preference for less invasive treatments, and no observable differences in anxiety, quality of life, morbidity, or mortality.³⁰ However, because decision aids apply to heterogeneous patient populations with 1 common clinical presentation or choice, they do not consider individual patient physiology and risk factors.

Prognostic Scoring Systems

Traditional prognostic scoring systems use regression modeling on aggregate patient populations to identify static variable risk factor thresholds, which are applied to individual patients. For example, elevated serum levels of C-reactive protein (CRP) are associated with anastomotic leak after colorectal surgery. A meta-analysis⁴³ found that the optimal postoperative day 3 CRP cutoff value was 172 mg/L (to convert to nanomoles per liter, multiply by 9.524). This is easy to apply at the bedside but does not accurately reflect pathophysiology. Serum CRP has a relatively constant half-life, and its production is directly associated with with inflammation along a continuum.⁵⁶ If 4 different patients have CRP levels of 10 mg/L, 171 mg/L, 173 mg/L, and 1000 mg/L 3 days after a colectomy, few clinicians would group these patients according to the 172 mg/L cutoff. The negative predictive value was 97%, such that a low value usually indicates no leak, but the positive predictive value was 21%.

Most diseases are not driven by a single physiologic parameter; therefore, prognostic scoring systems often incorporate multiple parameters for tasks such as measuring illness severity and predicting stroke and severe gastrointestinal bleeding.^24,45,57 Parametric regression prognostic scoring systems assume that relationships among input variables are linear.^22,29 When the relationships are nonlinear, the scoring system is similar to a coin toss.¹¹

To facilitate clinical adoption, prognostic scoring systems have been implemented as online risk calculators. The NSQIP Surgical Risk Calculator is a prominent example. Calculator use may increase the likelihood that patients will participate in risk-reduction strategies such as prehabilitation.³ However, input variables must be entered manually, and its predictive accuracy is suboptimal, especially for nonelective operations, representing opportunities for improvement.^4–7

Artificial Intelligence Predictive Analytics and Augmented Decision-Making

In 1970, William B. Schwartz published a Special Article in the New England Journal of Medicine stating, “Computing science will probably exert its major effects by augmenting and, insomecases, largely replacing the intellectual functions of the physician.”⁵⁸ Despite extraordinary advances in computer technology, this vision has not been realized. Several factors may contribute. Traditional clinical decision-support systems require time-consuming manual data acquisition and entry, which impairs their adoption.^8,33 Even the most successful and widely used static variable cutoff values do not accurately represent individual patient pathophysiology, as reflected by their suboptimal accuracy.^34,43,56 Parametric regression equations also fail to represent the complex, nonlinear associations among input variables, further limiting the accuracy of traditional multivariable regression models.^22,29 The weaknesses of traditional approaches may be overcome by artificial intelligence models fed with livestreaming intraoperative and EHR data to augment surgical decision-making through hpreoperative, intraoperative, and postoperative phases of care (Figure 2).

Figure 2. Optimal and Suboptimal Approaches to Surgical Decision-Making.

Artificial intelligence refers to computer systems that mimic human cognitive functions such as learning and problem-solving. In the broadest sense, a computer program using simple decision tree functions can mimic human intelligence. However, artificial intelligence usually refers to computer systems that learn from raw data with some degree of autonomy, as occurs with machine learning, deep learning, and reinforcement learning (Figure 3). Whereas traditional clinical decision-support systems use rules to generate codes and algorithms, artificial intelligence models learn from examples. Herein lies the strength of artificial intelligence for predictive analytics in medicine: human disease is simply too broad and complex to be explained and interpreted by rules.^59,60

Figure 3. Summary of Artificial Intelligence Techniques.

AI indicates artificial intelligence; EHR, electronic health record.

Machine Learning

Machine learning is a subfield of artificial intelligence in which a computer system performs a task without explicit instructions. Supervised machine learning models require human domain expertise and computer engineering to design handcrafted feature extractors capable of transforming raw data into desired representations. The algorithm learns associations between input data and prescribed output categories. Once trained, a supervised model is capable of classifying new unseen input data. With unsupervised techniques, input data have no corresponding annotated output categories; the algorithm creates its own output categories according to the structure and distribution of the input data. This approach allows discovery of patterns and phenotypes that were unrecognized prior to model development.

Machine learning has been used to accurately predict sepsis, in-hospital mortality, and acute kidney injury using intraoperative time-series data.^9,21,27,32 Each machine learning algorithm has distinct advantages and disadvantages for different tasks such that performance depends on fit between algorithm and task. To capitalize on this phenomenon, Super Learner ranks a set of candidate algorithms^20,28,38,39 by their performance and applies an optimal weight to each, creating ensemble algorithms that can accurately predict transfusion requirements and mortality among trauma patients.^20,28,38,39 Supervised and unsupervised machine learning input features must be handcrafted using domain knowledge. In deep learning, features are extracted by the model itself.

Deep Learning

Deep learning is a subfield of machine learning in which computer systems learn and represent highly dimensional data by adjusting weighted associations among input variables across a layered hierarchy of neurons or artificial neural network. Early warning systems that alert clinicians to unstable vital signs illustrate data dimension ality. As the number of vital sign data sources increases linearly, the combinations of alarm parameters that trigger early warning system alarms increase exponentially, resulting in frequent false alarms. Even without a corresponding exponential increase in observations, data are highly dimensional when many variables are used to represent a single patient or event, especially when the number of patients or events in the data set is relatively low, producing unique and rare mixtures of data. Prediction models are less effective when classifying mixtures of data that are rare or absent in the development or training data set. The ability of deep models to represent highly dimensional data is important to their application to surgical decision-making.

In deep models, the initial input and final output layers are connected by hidden layers containing hidden nodes. Each hidden node is assigned a weight that is influenced by previous layers, affects the output from that neuron, and has the potential to affect the outcome classification of the entire network. An algorithm optimizes and updates weights as the model is trained to achieve the strongest possible association between input and output layers. This structure allows accurate representation of chaotic and nonlinear yet meaningful relationships among input features. Deep models automatically learn optimal feature representations from raw data without handcrafted feature engineering, providing a logistical advantage over machine learning models that require time-intensive feature engineering.⁶¹ Automatic feature extraction also promotes discovery of novel patterns and phenotypes that may have been overlooked by handcrafted feature selection techniques.

Clinical applications of deep learning benefit from the ability to include multiple different types and sources of data as inputs for a single model, including wearable sensors and cameras capturing patient movements and facial expressions with computer vision, an artificial intelligence subfield in which deep models use pixels from images and videos as inputs.^60,62,63 Deep models have successfully performed patient phenotyping, disease prediction, and mortality prediction tasks.^19,26,41,64 When applied to the same variable set used to calculate SOFA scores, deep models outperform traditional SOFA modeling in predicting in-hospital mortality for ICU patients.⁴² Preliminary data suggest that deep models are theoretically capable of accurately predicting risk for perioperative and postoperative complications and augmenting recommendations for operative management and the informed consent process. Despite their utility for predictive analytics, deep learning only provides outcome probabilities that loosely correspond to specific decisions and actions. In contrast, reinforcement learning is well suited to support specific decisions made by patients, caregivers, and surgeons.

Reinforcement Learning

Reinforcement learning is an artificial intelligence subfield in which computer systems identify actions yielding the highest probability of an outcome. Reinforcement models can be trained by series of trial and error scenarios, exposing the model to expert demonstrations, or a combination of these strategies. This occurs in a Markov decision process framework, consisting of a set of states, a set of actions, the probability that a certain action in a certain state will lead to a new state, and the reward that results from the new state. Using this framework, the system creates a policy that identifies the choice or action with the highest probability of a desired outcome, assessing total rewards attributable to multiple actions performed over time and the relative importance of present and future rewards, facilitating application of reinforcement learning to clinical scenarios that evolve over time.

Reinforcement learning has been used to recommend optimal fluid resuscitation and vasopressor administration strategies for patients with sepsis.³¹ Ninety-day mortality was lowest when care provided by clinicians was concordant with model recommendations. Reinforcement learning has also been used to recommend basal and bolus insulin administration for virtual type 1 diabetics.⁴⁶ The algorithm performed as well as standard intermittent self-monitoring and continuous glucose monitoring methods, but with fewer episodes of hypoglycemia. Similar methods could be applied to augment the decision to operate.

Implementation

Automated Electronic Health Record Data

The Health Information Technology for Economic and Clinical Health Act of 2009 incentivized adoption of HER systems.⁶⁵ Within 6 years, more than 4 of 5 US hospitals adopted EHRs.⁶⁶ The volume of data generated by EHRs is staggering and will likely increase over time. Approximately 153 billion GB of data were generated in 2013, with projected growth of 48% per year.⁶⁷ This data volume is ideal for artificial intelligence models, which thrive on large data sets.

Because EHRs are continuously updated as patient data become available, artificial intelligence models can provide real-time predictions and recommendations. Works published within the last year demonstrate the feasibility of this approach. The My Surgery Risk platform uses EHR data for 285 variables to predict 8 postoperative complications with an area under the curve (AUC) of 0.82–0.94 and to predict mortality at 1, 3, 6, 12, and 24 months with an AUC of 0.77–0.83.¹⁵ Electronic health record data feed the algorithm automatically, obviating manual data search and entry and overcoming a major obstacle to clinical adoption. In a prospective study, the algorithm predicted postoperative complications with greater accuracy than physicians.¹⁷

Mobile Device Outputs

To optimize clinical utility and facilitate adoption, automated model outputs could be provided to mobile devices. This would require several elements that communicate with one another reliably and efficiently, including robust quality filters, a public key infrastructure, and encryption that can only be deciphered by the intended receiver.⁶⁸ Model outputs could be provided to mobile devices equipped with the appropriate Rest API client-server relationship and security clearance or through Google Cloud Messaging. To our knowledge, automated surgical risk predictions with mobile device out puts have not yet been reported. However, efforts to use manual data entry to feed machine learning models for surgical risk prediction on mobile devices have been successful.¹⁴

Human Intuition

Human intuition seems to arise from dopaminergic limbic system neurons that modify their connections with one another when a certain pattern or situation leads to a reward or penalty such as pleasure or pain.^69,70 Subsequently, similar patterns or situations evoke positive and negative emotions, or gut feelings, which are powerful and effective decision-making tools. In a sentinel investigation¹² of intuitive decision-making, participants drew cards from 1 of 4 decks for a cash reward. Two decks were rigged to be advantageous and 2 were rigged to be disadvantageous. Participants could explain differences between decks after drawing 80 cards, but demonstrated measurable anxiety and perspiration when reaching for a disadvantageous deck after drawing 10 cards and began to favor the advantageous deck after 50 cards before they could consciously explain what they were doing or why they were doing it. Similar phenomena occur in fight-or-flight survival responses, naval warfare, and financial decision-making.^71,72 Intuition can also identify patients with life-threatening conditions that would be underappreciated by traditional clinical parameters alone.^47,48

Challenges to Adoption

Data Standardization and Technology Infrastructure

To produce models that may be integrated with any EHR in any setting, data must be standardized. The Fast Healthcare Interoperability Resources framework establishes standards for health information exchange using a set of universal components assembled into systems that facilitate data sharing across EHRs and cloud-based communications. In addition, the Epic EHR that dominates the market has exclusive rights to develop new functions. To avoid legal conflicts, virtual models can live outside the EHR.¹⁵ However, this requires technology infrastructure that is not currently available in all clinical settings.

Interpretability

Diligent clinicians and informed patients will want to know why a computer program made a certain prediction or recommendation. Several techniques address this challenge, including attention mechanisms that reveal periods during which model inputs contributed disproportionately to the output, plotting pairwise similarities between data points to display phenotypic clusters, and training models on labeled patient data and then a linear gradient-boosting tree so that the model will assign relative importance to patient data input features.^18,42,73

Safety and Monitoring

If model inputs are flawed or model outputs are not carefully monitored by data scientists and interpreted by astute clinicians, many patients could be harmed in a short time frame. Artificial intelligence models trained on erroneous or misrepresentative data are likely to obscure the truth. Because studies with positive results are more likely to be submitted and published, artificial intelligence literature may be overly optimistic. Prior to clinical implementation, machine and deep learning models must be rigorously analyzed in a retrospective fashion and externally validated to ensure generalizability. Performing a stress test of artificial intelligence models by simulating erroneous and rare model inputs and assessing how the model responds may allow clinicians to better understand how and why failures occur. Initial prospective implementation should occur on a small scale under close monitoring, similar to phase 1 and 2 clinical trials for experimental medications, with analysis of how decision-support tools affect decisions across populations and among individual patients.⁷⁴ In cooperation with the International Medical Device Regulators Forum, the US Food and Drug Administration created the Software as Medical Device category and developed a voluntary Software Precertification Program to aid health care software developers in creating, testing, and implementing Software as Medical Device. Medicolegal regulation of Software as Medical Device is not rigidly defined.

Ethical Challenges

When algorithms are trained on data sets that are influenced by bias, algorithm outputs will likely reflect similar bias. In 1 prominent example, a model designed to augment judicial decision-making by predicting the likelihood of crime recidivism demonstrated predilection for racial/ethnic discrimination.⁷⁵ When data used to train an algorithm are predominantly derived from patient populations with different demographics than the patient for whom the algorithm is applied, accuracy may suffer. For example, the Framingham heart study primarily included white participants. A model trained on this data may reflect racial and ethnic bias because associations between cardiovascular risk factors and events differ by race and ethnicity.²⁵ Accountability for errors poses another challenge. Our justice system is well-equipped to address scenarios in which an individual clinician is responsible for making an errant decision, but it may prove difficult to assign blame to a computer program and its developers.

Conclusions

Surgical decision-making is impaired by time constraints, uncertainty, complexity, decision fatigue, hypothetical-deductive reasoning, and bias, leading to preventable harm. Traditional decision-support systems are compromised by time-consuming manual data entry and suboptimal accuracy. Automated artificial intelligence models fed with livestreaming EHR data can address these weaknesses. Successful integration of artificial intelligence with surgical decision-making would require data standardization, advances in model interpretability, careful implementation and monitoring, attention to ethical challenges, and preservation of bedside assessment and human intuition in the decision-making process. Artificial intelligence models must be rigorously analyzed in a retrospective fashion with robust external validation prior to prospective clinical application under the close scrutiny of astute clinicians and data scientists. Properly applied, artificial intelligence has the potential to transform surgical care by augmenting the decision to operate, the informed consent process, identification and mitigation of modifiable risk factors, recognition and management of complications, and shared decisions regarding resource use.