Table 1:

Summary of included studies.

Primary Author	Study Topic	Study Design	Population	Sample Size	Major Findings Pertinent to this Review	Sources of Funding, Conflicts of Interest
Dijksterhuis3	Decision-making	Observational	Consumers of simple and complex products	75	Conscious deliberation can lead to less satisfactory decision making than “deliberation-without-attention” choices, especially for complex purchases	Netherlands Organization for Scientific Research
Wolf5	Decision-making	Prospective	First-year house officers	89	Bayesian inference can provide a useful complement to clinical judgement, though it is rarely used by first-year house officers	Spencer Foundation
Graber6	Decision-making	Retrospective	Cases of suspected diagnostic error	100	System-related factors and cognitive biases result in diagnostic error	National Patient Safety Foundation
Kirch7	Decision-making	Retrospective	Autopsy reports from 1959, 1969, 1979, and 1989	400	Misdiagnosis rates of were approximately 10% across all decades	None reported
Sonderegger-Iseli8	Decision-making	Retrospective	Autopsy reports from 1972, 1982, and 1992	300	Frequency of major discrepancy between clinical diagnosis and necropsy findings in 1992 was 14%	None reported
Healey9	Decision-making	Prospective	Surgical inpatients	4,658	Half of all adverse events were attributable to provider error, diagnostic and judgement errors were the second most common cause of preventable harm	None reported
Shanafelt10	Decision-making	Cross-sectional	Members of the American College of Surgeons	7,905	70% of all surgeons attributed error to individual, rather than system-level, factors; 9% of all surgeons reported making a major medical error in the last three months, and lapses in judgement were the most common cause (32%)	None reported
Leeds11	Decision-making	Cross-sectional	Surgical trainees at 4 institutions	124	26% of surgical trainees report using validated, contemporary risk communication frameworks; barriers to use included lack of electronic and clinical workflow integration	NIH, NCI, ASCRS, AHRQ
Brotman12	Decision-making	MEDLINE search	Articles with “independent risk factor” or “independent predictor” in their title or abstract	-	Each year, more than 1000 articles are published investigating “independent risk factors” or “independent predictors”	None reported
Legare18	Decision-making	Systematic Review	Articles about implementing shared decision-making practices	38	Barriers to shared decision-making included time constraints and lack of applicability to patient and clinical context; facilitators were provider motivation and positive impact on clinical process and patient outcomes	Canada Research Chair in Implementation of Shared Decision-Making
Bertrand19	Decision-making	Cross-sectional	ICU patients and their providers	419	Decision-making capacity was overestimated by providers, largely due to inappropriate conflation of consciousness and decision-making capacity	Gabriel Montpied Teaching Hospital, Pfizer, Fisher & Paykel, Gilead, Jazz Pharma, Baxter, Astellas, Alexion
de Mik20	Decision-making	Systematic Review	Literature on shared decision-making during surgical consultations	32	Only 36% of all patients and surgeons perceived the consultation as shared decision-making; surgeons were more likely to perceive that interactions represented shared decision-making	AMC Foundation
Wilson21	Decision-making	Systematic Review	Literature on self-reported decisional regret	79	14.4% of patients reported regret, most often associated with type of surgery, disease-specific quality of life, and shared decision-making	None reported
Guyatt31	Decision analysis	Review	-	-	Demonstrates integration of patient values with decision analysis for risks of stroke and hemorrhage when prescribing antiplatelet therapy for patients with atrial fibrillation	None reported
Pauker33	Decision analysis	Review	-	-	Establishment of “testing” threshold and a “test-treatment” threshold to guide decision making on treatment and diagnostic testing	NIGMS, NLM, NIH
O’Brien39	Decision analysis	Review	-	-	Description of how to perform cost-adjusted value measures like quality-adjusted life years (QALY)	None reported
Djulbegovic36	Decision analysis	Review	-	-	Applies patient values to decision analysis for DVT prophylaxis	None reported
Vickers37	Decision analysis	Review	-	-	Description of how decision curve analysis can be used to evaluate diagnostic and prognostic strategies	NCI SPORE
Gage40	Decision analysis	Decision analysis	Patients with nonvalvular atrial fibrillation	-	Warfarin is cost-effective for patients with nonvalvular atrial fibrillation and one additional stroke risk factor. In those without such risk factors, this benefit was lost	None reported
Robbins41	Decision analysis	Cost-benefit analysis	Infants treated with RSV-IG from 3 RCT	1,108	Demonstrates the use of number-needed-to-treat principles in determining RSV-IG treatment for specific infant populations	None reported
Komorowski45	Machine learning	Retrospective review	Patients admitted to ICU with sepsis	96,156	A reinforcement learning model recommended intravenous fluid and vasopressor strategies, mortality was lowest when decisions made by clinicians matched recommendations from the reinforcement learning model	NIHR, EPSRC, Orion Corp, Amomed Pharma, Ferring Pharma, Tenax Therapeutics, Baxter, Bristol-Mysers Squibb, GSK, HCA International, Philips Healthcare, Fresenius-KABI
Silver46	Machine learning	Observational	Go boardgame	-	A deep reinforcement learning model trained by human expert moves and self-play provided high fidelity victories against previous Go algorithms and human experts	Google, Google DeepMind
Mnih47	Machine learning	Observational	Atari games	-	Demonstrated development of a deep Q-network to incorporate highly-dimensional sensory inputs and actions to optimize machine performance in Atari video games	Google, Google DeepMind
Shickel49	Machine learning	Retrospective Review	ICU admissions	79,701	A deep model fed with SOFA variables predicted in-hospital mortality with greater accuracy than the traditional SOFA score (AUC 0.90 vs. 0.85)	NIGMS, NSF, University of Florida CTSI, NCATS, J Crayton Pruitt Family Department of Biomedical Engineering, NVIDIA
Sundaram50	Machine learning	Observational	Actual and simulated animal and human genomes	-	A deep neural network identified pathogenic mutations for rare diseases with 88% accuracy and discovered candidate genes for intellectual disability	Health Innovation Challenge Fund, Wellcome Sanger Institute, NIHR, NIGMS, NSF
Li51	Machine learning	Observational	Actual and simulated protein sequences	-	A deep neural network predicted protein properties by learning from protein sequences, without supervision or domain knowledge	NSF, NIH, Industrial Members of NSF Center for Big Learning
Rajpurkar52	Machine learning	Retrospective Review	Chest radiographs	420	A deep learning algorithm had equivalent performance to board certified Radiologists in 10/14 pathologies, superior performance in 1/14, and inferior performance in 3/14	Stanford AIMI Center, whiterabbit.ai, nines.ai, Nuance communications, Radiological Society of North America, Phillips Healthcare, GE Healthcare
Davoudi53	Machine learning	Observational	Surgical ICU patients	22	Autonomous collection of granular patient and environmental data can identify contributors to delirium	NSF CAREER, NIH/NIGMS, NIH, NIH/NIBIB
Hashimoto54	Machine learning	Observational	Laparoscopic sleeve gastrectomy cases	88	The use of computer vision and deep neural networks can identify quantitative steps in operative procedures	NIH, Olympus Corporation, Toyota Research Institute, Verily Life Sciences, Johnson & Johnson Institute, Gerson Lehrman Group
Silver56	Machine learning	Observational	Go boardgame		A deep reinforcement learning model trained by self-play without human input consistently defeated a version that did use human input	Google, Google DeepMind
Pineau65	Machine learning	Review	Experimental epilepsy models	-	Reinforcement learning paradigms of EEG measurements can be used to optimize electrostimulations patterns in the treatment of epilepsy	NSERC, CIHR
Van Calster66	Decision analysis	Systematic Review	Men undergoing prostate biopsy	3,616	Demonstrates the use of decision curve analysis to identify a range of clinically reasonable risk thresholds for prostate biopsy	Research Foundation–Flanders
Tinetti67	Decision analysis	Review	Disease-specific guidelines	-	Adherence to disease-specific guidelines in patients with multiple chronic conditions may result in clinical harm	NIA, VA HSR&D, Merck, AFAR
Boyd68	Decision analysis	Observational	Clinical practice guidelines for Medicare beneficiaries	-	Adherence to clinical practice guidelines for disease-specific entities may result in suboptimal care for elderly patients with multiple comorbidities	NIH, NIA, HRSA, Roger C. Lipitz Center for Integrated Health Care, Partnership for Solutions
Che70	Machine learning	Retrospective Review	Critically ill children	398	Deep learning models often lack interpretability; shallow models and knowledge-distillation approaches can clarify underlying processes for clinicians	NSF, USC Coulter Translational Research Program
Gal71	Machine learning	Dissertation	-	-	Using a softmax function to map machine learning output layer network activations may overestimate model confidence that its outputs are accurate	Google AI, Qualcomm
Guo72	Machine learning	Review	Machine learning models in published literature	-	Many descriptions of machine learning models do not incorporate and report calibration	NSF, Bill and Melinda Gates Foundation, Office of Naval Research
Vergouwe73	Decision analysis	Observational	Moderate or severe brain injury patients	1,118	Creating benchmark values that incorporate distributions of patient characteristics can improve external validity of prediction models	Netherlands Organization for Scientific Research, NIH
Van Calster74	Decision analysis	Observational	Decision analysis models	-	Miscalibration of a model (overestimation, underestimation, overfitting, and underfitting) to a baseline event rate reduces net benefit and can impair clinical decision-making	Research Foundation-Flanders
Goldstein75	Machine learning	Retrospective review	Hemodialysis patients	18,846	Comparing summary statistics, machine learning methods, functional data analysis, and joint models revealed that complex approaches using highly-dimensional EHR data may impair mortality predictions	NIDDK
McGlynn76	Decision analysis	Cross-sectional study	Randomly selected patients from 12 US metropolitan areas	13,275	Only slightly more than half of all patients surveyed received care recommended by clinical practice guidelines	Robert Wood Johnson Foundation, VA HSR&D

AFAR: American Federation for Aging Research; AHRQ: Agency for Healthcare Research & Quality; AI: Artificial Intelligence; AIMI: Artificial Intelligence in Medicine and Imaging; ASCRS: American Society of Colon and Rectal Surgeons; CIHR: Canadian Institute of Health Research; EPSRC: Engineering and Physical Sciences Research Council; HRSA: Health Resources and Services Administration; NCI: National Cancer Institute; NIA: National Institute on Aging; NIBIB: National Institute on Biomedical Imaging and Bioengineering; NIGMS: National Institute of General Medical Sciences; NIH: National Institutes of Health; NIHR: National Institute for Health Research; NLM: National Library of Medicine; NSERC: Natural Sciences and Engineering Research Council; NSF: National Science Foundation; SOFA: Sequential Organ Failure Assessment; USC: University of Southern California; VA HSR&D: Veterans Affairs Health Services Research and Development