Abstract
Clinical decision-making is based on knowledge, expertise, and authority, with clinicians approving almost every intervention—the starting point for delivery of “All the right care, but only the right care,” an unachieved healthcare quality improvement goal. Unaided clinicians suffer from human cognitive limitations and biases when decisions are based only on their training, expertise, and experience. Electronic health records (EHRs) could improve healthcare with robust decision-support tools that reduce unwarranted variation of clinician decisions and actions. Current EHRs, focused on results review, documentation, and accounting, are awkward, time-consuming, and contribute to clinician stress and burnout. Decision-support tools could reduce clinician burden and enable replicable clinician decisions and actions that personalize patient care. Most current clinical decision-support tools or aids lack detail and neither reduce burden nor enable replicable actions. Clinicians must provide subjective interpretation and missing logic, thus introducing personal biases and mindless, unwarranted, variation from evidence-based practice. Replicability occurs when different clinicians, with the same patient information and context, come to the same decision and action. We propose a feasible subset of therapeutic decision-support tools based on credible clinical outcome evidence: computer protocols leading to replicable clinician actions (eActions). eActions enable different clinicians to make consistent decisions and actions when faced with the same patient input data. eActions embrace good everyday decision-making informed by evidence, experience, EHR data, and individual patient status. eActions can reduce unwarranted variation, increase quality of clinical care and research, reduce EHR noise, and could enable a learning healthcare system.
INTRODUCTION
Our healthcare culture blocks development of a learning healthcare system
The current decision-making model is based on clinician knowledge, expertise, and authority/autonomy, with clinicians approving almost every intervention. Healthcare culture is inexorably linked to this expert-based model (“Era 1”1). Culture has been described as the “software of the brain.”2 New information indicates that culture can also modify the structure of the brain, changing people’s biology independent of genetics, thus adding a new dimension to cultural considerations.3 However, this culture is only the starting point for delivery of credible evidence-based healthcare interventions. The expert-based model is necessary but not sufficient for delivery of “All the right care, but only the right care” (p. 14)—a healthcare quality improvement goal yet to be closely approached. In this paper we focus on treatment (similar to a management system5) not diagnosis, since replicable clinician actions are currently more easily achieved for treatment.
Decades of compelling evidence indicate unaided clinicians suffer from cognitive limitations and biases when decisions are based only on training, expertise, and experience.3,5–12 Almost all resources for improving healthcare outcomes support ordinary guidelines and protocols and processes (eg, quality improvement,4,13 education,14–16 and training17) (Table 1, row 1). These produce improvements but fail to approach achieving “All the right care, but only the right care.” The behavioral economics strategy of making it “easy to do it right,” consistently and rationally linking action to best evidence, is in natural tension with healthcare culture and the clinician’s concept of decision autonomy.1,18–20 Healthcare culture is thus a barrier to implementation of many potentially useful computer applications.12,21 We argue that surmounting the cultural barrier and generating a true learning healthcare system will require a “bottom-up”22,23 clinical problem-focused redesign that virtually eliminates random and systematic noise due to variable clinician decisions. Such a system can produce almost noise-free clinical EHRs and replicable clinical outcome results.24–27
Table 1.
Variation |
|||||||
---|---|---|---|---|---|---|---|
Strategy | Intervention (Transfer Function) | Automated Action? | Clinician Compliancea | Clinical Outcome Credibility | Warranted Mindful | Unwarranted Mindless | Outcome Replicability |
Top-down, system focus | CQI, 6-σ, Zero patient harm, care process, guideline, protocol | ① No | Low (15%–40%) (31, 32) | Low | Yes: defaultb | Common ∼50% | Low (15%–40%) |
Bottom-up, clinical problem focus | eActions→ Adaptive, personalized patient care decision/ action | ② No | High (∼95%) | High | Yes: defaultb | Small (0.3%) | High (∼95%) |
③ Yes | Highest (∼100%) | Highest | Yes: not default | No | Highest (∼100%) |
Abbreviation: CQI, continuous quality improvement.
with best evidence-based care.
Yes: possible; Default: deciding clinician must approve each decision/action.
Clinician variability in treatment decisions/actions
Clinician treatment decisions/actions vary widely.1,28–30 Adults and children only receive recommended care about 50% of the time,31–33 reflecting a significant treatment gap.12,34–39 Desired, mindful, clinician variation can contribute new insights in some situations, such as the COVID-19 pandemic.11 However, individual clinician decision-making is commonly associated with mindless11,12 or unwarranted variation (deviations from best practice, not based on evidence or patient preference),40,41 and associated with waste, morbidity, and mortality.4,28,42–47 Even specialists claiming to follow best evidence do not consistently do what they say.5,8,48–52 Widespread overestimation of professional self-confidence seems to contribute to this variability in clinician decision-making,9,10,53 for example “…professionals often make decisions that deviated significantly from… peers, from their…prior decisions, and from rules… they… claim to follow” (p. 4010). This variation increases with urgency or acuity and can result both from variable clinician responses to task and information overload and incentive misalignment.54–56 These responses reflect undefined and unrecorded individual clinician beliefs, biases,3,5–12,29,30 and decision rules that, while preserving limited cognitive resources,56–59 impede deep attention required to address complex problems.60 The responses lack underlying clinician decision rationales and confound assessment of clinical outcomes of trials, observations, and large data sets (big data).55,61,62
Unwarranted variation, for example, in transfusion or blood glucose targets,29,30 impedes replicability and introduces systematic noise (bias) into clinician decision-making and thus into the EHR database. This noise, unlike random noise, cannot be overcome by increasing the number of observations. All noise impedes accurate machine learning61,62 and thus impedes achievement of a sorely needed learning healthcare system. Limited availability of appropriately granular and interoperable EHR data is also an impediment. In addition, choice of machine learning strategy introduces variability.63 Some computer applications can reduce a major source of both random and systematic (bias) EHR noise by almost eliminating variation in clinician decision-making called “unwarranted40,41” or “mindless11,12”. This is a long-recognized problem.28,64 We assert that reducing noise will increase the EHR signal-to-noise ratio65,66 enabling a learning healthcare system21,67–75 and improving important clinical outcomes.52
Replicability
Replication of results is a cornerstone of science and experimentation.76–81 This is not always possible in clinical settings82 but is generally recognized as an important goal even in settings difficult to control, like surgery.83,84 In many clinical settings where evidence and heuristics are available, thoughtful and replicable detailed clinician decision-making methods produce more consistent evidence-based care across clinicians, time, and institutions.5,24–27
This construct began in earnest with paper-based protocols in clinical psychology85,86 and automatic electroencephalographic control of general anesthesia87 about 70 years ago, without broad adoption. Over forty years ago, closed-loop66,88 computer protocols automatically controlled left atrial pressure post-operatively89 with vasoactive drugs90 and blood products.91Replicability occurs when the same patient information and context lead different clinicians to the same decision and action.24,25,27,55,92–100 Such replicability of clinician actions (interventions) helps assure scientific validity of experimental and observational studies.76,101–103 This replicability links interventions with evidence, increasing appropriate and effective care, enhancing research by reducing noise from unwarranted variation, supporting artificial intelligence utility, and perhaps increasing teaching consistency.104 Widely distributed replicability might also reduce racial and ethnic care disparities, as long as rules appropriately attend to racial differences.
Detailed computer protocols
In seeking benefits that outweigh risks, clinicians practice clinical “arts” through desired, mindful, clinician variation.11,12 Absent guiding evidence or patient preferences, “artful” clinical decisions/actions must be determined exclusively by clinicians, usually during patient–clinician interactions, with contextual integration of clinician expertise and available information. However, complex healthcare challenges include diversity, quality, access, information overload, provider burnout, ethical dilemmas, and cost. Clinicians often make decisions and take actions without incorporating all available, and often overwhelming, evidence.11,12,20,54,55,93,105,106 Well-designed computerized decision-support tools, based on comprehensive evidence and expert clinician logic/judgment, can help clinicians incorporate this evidence.107,108 Expert clinician logic/judgment is captured through an iterative development process that includes elements of conflict resolution and enough comprehensive patient input detail to generate both patient and time-specific care instructions.92,96 This overcomes the shortcomings of most decision-support attempts.6 Decision-support rules are thus developed asynchronously, during tool generation before the patient–clinician encounter.96 During the encounter, these tools function synchronously as a surrogate expert consultation and can unburden the overburdened clinician.20 These tools generate replicable, detailed, patient and context-specific, evidence-based decisions/actions distributed across clinician roles and disciplines. Schwartz discussed, 50 years ago, the yet unrealized potential for computers to change medical practice. He claimed “…computer as an intellectual tool can reshape the present system of health care” (italics added, p. 1257109).
Unfortunately, current EHRs do not fulfill this vision. Indeed, by requiring many detailed, task-oriented interactions, they can be counterproductive and contribute to clinician burnout,20 and impede experienced clinicians from deep clinical thinking and from applying clinical “arts.”20,110,111 Appropriate and automated tools can maximally relieve clinicians of this current EHR burden. We call these tools eActions to emphasize replicable clinician actions in contrast with decision aids that merely deliver replicable messages—an important distinction commonly overlooked.112eActions also enable rapid, detailed, and rigorous translation of clinical research results to usual clinical care24 thus overcoming many barriers to diffusion of innovations that rely on learning and performance by individuals within complex institutions.34,113–116 Translation of eActions from research studies to usual care is an extreme and effective version of the “make it happen” reengineering imperative for the spread of innovations in service organizations.117 In addition, eActions operating automatically in a closed-loop manner66,88 could diminish, or even eliminate, some important research strategy differences between detailed explanatory randomized controlled clinical trials (RCTs) and large simple, pragmatic, and comparative effectiveness trials. eActions could enable replicable methods to produce more robust explanatory trial results and enable more robust large multi-institutional trials with the same replicable method (replacing some pragmatic trials).55 After completion of a trial, eActions can be immediately introduced into usual care and could become the basis of quasi-experimental or more rigorous comparative effectiveness trials118–120 of modified eActions, thereby generating a learning healthcare system. A learning healthcare system could ameliorate the research uncertainties, and subsequent inappropriate care, associated with the use of biomarkers and other intermediate outcomes as surrogate endpoints for ultimate clinical outcomes.121eActions would be particularly helpful during difficult or novel situations that can overwhelm healthcare resources. Finally, automated eActions could extend automated computerized physician order entry,122 respond to complexity60 and decision-support challenges,123 and maximally unburden clinicians during major threats like the COVID-19 pandemic.124,125
Human decision-making systems
Human decision-making involves 2 information processing systems: an automatic effortless cognitive system using large stores of knowledge and experience, and a deliberative conscious effortful system used for extreme decisions and requiring great attention.56–59 The automatic effortless system often functions below awareness and resists change. The conscious effortful system is limited, more easily changed with information, and functions with full awareness. It is strictly conserved by avoiding deep rational thought whenever possible, a human behavior recognized for millennia.126–130 Efforts to alter clinical context and improve decision-making generally address both systems and include: clinical guidelines and protocols; process improvement (eg, Continuous Quality Improvement,13 Toyota Lean Methodology,131 Zero patient harm,132 6-sigma133,134); ergonomic approaches like those pioneered by anesthesiologists to preclude incorrect device connections;135 choice constraints (safe defaults requiring opting-out rather than opting-in); and team science studying organization, staffing, safety culture, and checklists.11 “Nudging decision-makers toward default choices” is a common practice.19,136,137 Many of these efforts (Table 1) are predominantly “top-down” indirect efforts that use marketing, training, and incentives to influence clinician decisions, but produce little change to internal motivation20 because bedside decision-making remains largely done automatically with the effortless cognitive system. Conversely, rapid and large change can follow efforts that directly provide clinicians with evidence-based decisions/actions, but these require more strictly limited conscious cognitive clinician resources and require a change in the expert-based healthcare culture.1,22,23 Removing some clinician task-focused decisions with decision-support tools would unburden clinicians and free cognitive resources to address higher-level complex situations and major therapeutic strategy decisions (the clinical “art”20). More simply, “bottom-up” eActions23 that directly alter bedside clinician decision-making can lead to ∼95% clinician compliance, even when not automated. eActions produce clinician actions tightly linked to credible evidence (Table 1, row 2).24,25,27,55,92,96–100
COMPUTER-PROTOCOLS LEADING TO REPLICABLE CLINICIAN ACTIONS
For eActions to be widely adopted they must be detailed, specific, and acceptable to clinicians. Acceptability depends primarily on demonstration of credible clinical outcomes and usability in practice. Unlike eActions, most clinical computer applications are difficult for clinicians to employ in practice.138,139 Our previous publications labeled eActions with different names25,27,55,92–94,96–99,140–142 (see Supplementary Materials for synonyms). eActions influence decisions/actions directly and indirectly by altering context (changing expectations, social motivation, administrative goals, etc).24,25,27,55,92–94,96–100eActions communicate specific decisions/actions based on the patient’s current state determined by multiple sources of data (eg, laboratory, imaging, the literature, patient trajectory, etc). Though they remain rare, we used eActions in multiple ways: as HELP system143–145 embedded EHR rules or external EHR interfaced rules and as independent eActions computer applications.24,25,27,55,92–100eActions are usually activated by EHR data, avoiding double data entry. However, 1 independent eActions blood glucose management study required double,25 and another required additional,100 data entry. Remarkably, the bedside clinicians in 1 center25 wanted to continue the double data entry for clinical care that they believed was improved, even though it increased their workload.
Usual guidelines, protocols, and most other decision-support tools or aids are not eActions. They encourage clinician decision independence with inadequately detailed messages (Table 1, row 1).1,20,107,108 While these messages can be replicated, clinicians execute such messages differently except in simple circumstances.11,146,147 Although commonly called medical algorithms,107,148–159 a broad and loosely used term,160 they are usually ambiguous flow-diagrams, tables, or rules and contain neither a precise list of steps, with temporal determinants, nor a clear stopping point (characteristics of mathematics and computer science algorithms).107,108,161 For example, the declarative guideline message “achieve a normal hemoglobin A1c” is replicable but is insufficiently detailed to lead to replicable clinician actions. Similarly, an expert protocol goal for mechanical ventilation of acute respiratory distress syndrome (ARDS) patients (Supplementary Figure 1) is “… use the least PEEP and tidal volume necessary to achieve acceptable gas exchange while avoiding tidal collapse and reopening of unstable lung units.”162 This replicable statement can lead to many different specific interventions by different clinicians, or even by the same clinician at different times, because it lacks the detail required for the replicable clinician actions achieved by eActions (Figure 1). Replicable clinician actions are the target here. Replicability is more important than the specific clinical care eActions strategy chosen from the many available acceptable clinical care strategies.164
The patient, clinician, and healthcare delivery system are all complex systems, each with modeling challenges.60,165 The rules and logic of eActions address and model only the clinician decision-maker.55,96 This does not require that we know the “truth,” or even all the input variables that might ultimately be revealed important. It requires only that we extract from clinicians, using robust established techniques, the way they make their decisions based on the information at hand.55,92,96 Clinicians, like all humans, are cognitively limited with average short-term (working) memory limited to 4 ± 1 psychological constructs before decisions become degraded.56–59,166 While many seem unaware of this ubiquitous limitation, it is expressed in multiple human activities including slide preparation,167 risk assessment,168 and stock trades.169 Most studied subjects overestimate their competence and inflate their self-assessments.49,50 Even personal beliefs are not consistently reflected in decisions.5,8,48–51 Paradoxically, these human cognitive limitations reduce the number of input variables developers need to consider for any single eActions decision and thus enable individual decisions (protocol branch points) to evolve from structured and focused clinician thinking.92,96Any singleeActiondecision usually requires only 1–3 data inputs.55,96 Because eActions may contain thousands of single decision rules, the total eActions information for all of its rules includes detail and integration typically difficult for humans to access and utilize11,12,20,55,93,96,98,104–106,166 (see mapping terms, drug classes, administration routes, equivalent doses, step changes, etc, and detailed rules, in Supplementary Table 1). For example, the first eActions, for management of arterial oxygenation in a clinical trial of extracorporeal support for ARDS, consisted of approximately 40 flow-diagram pages.94 In addition to treatment decisions, eActions rules could also indicate evaluations for rare causes of common problems, like inherited metabolic causes of pediatric seizures.
eActions embrace the nuances of defensible clinician decision-making informed by evidence, experience, laboratory and other EHR data and by individual patient trajectory including treatment response. Because eActions, automated or not, can electronically capture clinician responses to instructions, the detailed clinician decision/action directed care method is clearly documented.24,25,96–98,100,170 Such knowledge of detailed methods is rare.56eActions not only produce replicable clinician actions, they also produce replicable clinical outcomes (Figure 2a and 2b).24–26,94,171 For example, eActions eliminated all variation in blood glucose management and blood glucose value distributions between 4 ICUs in 3 US states and Singapore (Figure 2b). Interestingly, systems that incorporate clinician decisions, like all second-order or higher systems, can oscillate and fail if not properly damped.65,66 We incorporated 3 common clinician decision strategies to damp eActions and prevent failure due to oscillation: waiting times, dead zones, and fuzzy logic membership.172
Understanding is a requirement for clinician acceptance of eActions.140 Past work focused on if–then rule-based eActions because clinicians intuitively and easily understand if–then protocol rule logic. This was particularly important when eActions development began in 1986173,174 and clinicians questioned eActions’ ability to appropriately manage mechanical ventilation.27,94,96,97,175 However, decision strategies other than if–then rules can also produce successful eActions. These include physiology-based computer models of bodily function for blood glucose homeostasis,100,170 acid–base balance,176–178 oxygenation,179 and mechanical ventilation.142,180 In fact, these computer model eActions may be easier to develop and more adaptable to changing clinical contexts than rule-based eActions.100,141,142,170
WHY HAVE eACTIONS LED TO REPLICABLE CLINICAL OUTCOMES?
We implemented eActions in multiple clinical contexts and at multiple institutions (Table 1, Figure 2).24,25,27,55,92,96–100 The 95% clinician compliance with eActions instructions seems explained by 2 linked factors: initial credible favorable clinical outcome evidence24,25,27,55,96 and clinician beliefs.140
We obtained favorable clinical outcomes, in the intended clinical care setting, during the initial iterative refinement of eActions96 for mechanical ventilation,27,94 blood glucose control,25 and sepsis management.98 These results armed eActions developers with favorable outcome evidence.24,25,55,96,98 This outcome evidence attracted colleagues to join randomized eActions clinical trials and to review and critique the readily available eActions if–then rule logic.26,27 In contrast, well-conducted and important RCTs of other detailed if–then rule-based computer-protocols did not appear to have presented favorable initial clinical outcome evidence to participating clinicians.181,182 These computer protocols used EHR data including vital signs, symptoms, and functional class to generate guideline-based cardiac care suggestions like “Treat systolic dysfunction with ACE inhibitor unless allergic—ORDER Lisinopril 10 mg PO qAM.”181,182 In the absence of favorable initial prestudy clinical outcome evidence, participating clinicians might have lacked helpful motivation and beliefs that enable compliance with computer protocol instructions.140 This may explain their disappointing RCT results reported for both heart181 and lung disease.182 In spite of documented differences in clinician patient management styles,29,30 clinician beliefs in self-efficacy (the ability to reach an intended goal) and in eActions information accuracy, timeliness and trustworthiness, were associated with high compliance with individual patient-specific decisions/actions.20,140
ELEMENTS FOR SUCCESS
Based on past successes, we believe broad dissemination of eActions will likely require 7 elements (Table 2). The first 4 are supported by evidence.55,92,93,96,98,100,141 The remaining 3 need to be systematically evaluated. We believe that healthcare business leaders are good assessors of returns on investment for value-based medicine economies. The healthcare industry, itself, should invest in eActions as they currently do for care processes (Table 1).
Table 2.
Element | Explanation |
---|---|
SUPPORTED BY PUBLISHED RESULTS | |
1.“Bottom-up” clinician leadership23 dedicated to solving a specific clinician decision-making challenge55,93,96,98,100 | “Bottom-up” eActions leadership is a simpler, problem-focused strategy and complements the more complex “top-down” information technology efforts common in healthcare. eActions development, iterative refinement, validation, and safety assurance is resource intensive and enabled by clinicians with passion for solving a particular problem. It is this clinician focus on specific discrete problems that makes eActions challenges soluble. |
2.Multidisciplinary team dedicated to development and validation in the intended clinical use setting26,55,92,96,98,183 | This team requires frontline clinicians interested in solving the specific clinical problem, information scientists and technicians, multiple professional disciplines, and administrators. |
3.Easily understood intuitive screens and messages, using 1 unique self-explanatory term and appearance for each protocol construct or button | Many proprietary EHR systems fail to achieve this requirement of self-explanation that is important for ease of use by clinicians. Past eActions have been initiated clinically with as little instruction as the following: “put the blood glucose value here and follow the displayed instructions.” This reduces time-consuming, expensive, current clinician education as clinical unit staff members change. |
4.Iterative refinement and validation in a well-supervised clinical setting functioning as a human outcomes research laboratory, and producing credible clinical outcome data during refinement and validation of eActions rules25,55,93,96,98 | Clinical care environments that have adequate supervision and control to function as a human outcomes research laboratory are rare. Past myocardial infarction research units attempted to do this by complementing the clinical care units, but they did not reach the level necessary for eActions. It is the intended clinical care delivery unit that should be the human outcomes research laboratory. |
NOT YET SUPPORTED BY PUBLISHED RESULTS | |
5.EHR-platform-independent, public domain eActions | Future widespread distribution of eActions will not be achievable or capable of proper curation if applications are installed within proprietary EHR systems. These EHR systems are unlikely to achieve the level of interoperability necessary to allow rapid revision of eActions rules when new evidence appears. A single cloud-based application using SMART on FHIR and other strategies now seems a solution that would allow proper curation, like the Agency for Healthcare Research and Quality “CDS Connect” website (https://cds.ahrq.gov/cdsconnect). |
6.Academic curation of the protocol | Academic curation requires constant vigilance of eActions, a requirement consistent with the concern raised by Schwartz regarding the danger of centralized clinical decision-making.109 Such curation will likely require a change in current healthcare organizational culture and social pressure,140 especially at academic sites. |
7.Funding from healthcare industry sources | Funding from healthcare industry sources seem necessary since neither the initial rule development investment required for eActions96,98 nor eActions curation needs are met by current research funding sources. The return on investment can justify the business decision by the healthcare industry. For example, the original eActions mechanical ventilation research protocol27 was subsequently seamlessly translated to clinical care and consistently delivered lung-protective ventilation to Acute Respiratory Distress Syndrome patients.24 This eActions generated about 2,000,000 protocol care instructions in about 22,000 patients during about 3 decades at the original development hospital27,55,92 and at 1 of the collaborating research sites.97 |
WHERE MIGHT eACTIONS BECOME SUCCESSFUL?
The guideline for management of extracellular fluid retention in heart failure patients is mature and widely accepted,184 though inconsistently applied.185 We developed an eAction for managing fluid retention that could be formally studied (Supplementary Table 1). It could populate EHRs with complete, valid, and almost noise-free clinician decision-making data. This eAction could be automated via telemedicine to manage patients at home, unburdening not only clinicians but also patients and clinics through reduced visits. eActions would enable rigorous comparative studies of alternative care strategies proposed by thoughtful clinicians. This future heart failure work could enhance, for example, the current groundbreaking comparative study achievements of the Learning Healthcare System at Vanderbilt University Medical Center.186–189 A hint of comparative studies potential was provided by the reuse of EHR data populated by an if–then rule-based blood glucose and insulin eAction.25 These EHR data were reused as input data for a competing physiology-based blood glucose homeostasis eAction.170 The investigators concluded the results from the competing physiology-based eAction were likely preferable to the original results from the if–then rule-based eAction.141 In addition, we were able to explore, with the same eActions rules, the impacts of 2 different blood glucose targets for ICU patients managed with intravenous insulin190 and compare performance in diabetic with that in nondiabetic patients.191,192
CHALLENGES AND LIMITATIONS
All decision-support tools, including eActions, incorporate beliefs, interpretations, and biases.3,5–12,29,30,55,56,193,194 Consequently, rules must be defined wisely and consensually, be curated, incorporate clinical setting performance feedback, and deal with conflicts of interest and bias that may be present in guidelines adopted by prominent professional societies195,196 or developed by commercial interests.197 Credible clinically pertinent initial outcome data24,25,55,96,98 provide important checks against suboptimal protocols that include improper bias and that may lead to incorrect instructions and patient harm.195–197 Elimination of all clinician decision-making variation seems both impossible and undesirable.11,20 Indeed, in situations where important uncertainty exists, alternative eAction designs that allow causal inference may be crucial to learning and reducing uncertainty. Circumstances outside the scope of eActions rules and knowledge may suddenly emerge and be more easily captured with widespread use of eActions, than with current dependence on individual clinician recognition and reporting. The rules would subsequently, when appropriate, be modified through iterative refinement.96eActions would complement, not replace, other strategies currently employed in healthcare.
Complexity theory indicates that patient trajectories might, under some circumstances, be exquisitely sensitive to initial conditions, as in chaotic systems that sometimes make weather prediction difficult.165 This exquisite sensitivity is imperceptible and hidden below measurement resolution. This theoretically might preclude patients with identical problems and the same comprehensive clinical input data from following the same trajectory.60 We are unaware of any systematic clinical explorations of this potential complexity theory limitation. eActions would still enable different clinicians to achieve replicable actions and eliminate unwarranted variation. We believe that using the best available evidence to initiate therapy of even complex diseases, like glaucoma or insulin-dependent diabetes with multiorgan dysfunction and heart failure, is reasonable.52 As long as we make relatively short-term decisions and use iterative refinement and intelligent decision-making rules to deal with those patients who do not respond in the expected manner to eActions instructions,96 we assert this strategy will advance patient care.60
The totality of clinical problems amenable to eActions and to its widespread scaling both need exploration. Exploring more eActions should help drive collection of more relevant and important clinical EHR data. We believe eActions will be widely applicable because human cognitive limitations56–59,166 are ubiquitous and independent of healthcare discipline, disease, and clinical context complexity.25,27,55,92–94,97,104 Both automated and nonautomated eActions are disruptive innovations not well addressed by mature businesses such as healthcare.22,23,70 Funding for development, validation, implementation, and curation of eActions for multiple clinical problems will be important and should be provided by the healthcare industry. Not only will patient care quality increase (the face value imperative for moving ahead with this work) but profits might increase as well. One-quarter or more of clinical expenses are nonbeneficial wasted resources.46,47,198–202 Broad application of eActions could reduce clinical waste. In addition to providing resources, healthcare industry acceptance will be key to providing environmental support, social pressure, and organizational cultural contributions that are important for clinician acceptance and compliance with eActions decisions/actions.1,4,140 Admittedly, convincing the healthcare industry to fund work on eActions is challenging.22,23
Key to achieving healthcare culture change is not just individual compliance but also acceptance of the utility and value of our proposed eActions approach and linking it to clinician goals.20,52,140 Clinicians will alter their behavior to comply with demands of employers, a widely recognized human response. Clinician deskilling203,204 may occur and must be incorporated in assessment of risks and benefits of our proposed strategy. Although deskilling is declared an unavoidable consequence of civilization’s advance (p. 42205 and p. 29206), clinician deskilling does not have to occur.207 Instructions for the use of our first eActions in the HELP computer system144 clearly identified consistently applied principles and rules for mechanical ventilation of ARDS patients.27,94,96,97 This enhanced our trainees’ understanding, as was anticipated decades ago.107,108
CONCLUSION
Computer protocols leading to replicable clinician actions (eActions) reduce mindless, unwarranted variation in clinical care and research. eActions could improve both clinical care quality and research by unburdening overtasked clinicians and by reducing noise in clinical databases. Common current clinical decision-support tools or aids, including guidelines and protocols, neither enable replicable evidence-based actions, nor provide individual patient-specific care. eActions should be formally discussed and pursued, nationally and internationally, in our efforts to reduce clinician variation, improve care, and establish a learning healthcare system.
FUNDING
This work was supported by NIH grant numbers HL36787, HHSN268200425210C/NO1-HC-45210, HHSN268200425210C/N01-HR-56171, HHSN268200536179C/N01-HR-56179, HHSN268200536170C/N01-HR-56170, HHSN268200536175C/N01-HR-56175, HHSN268200536165C/N01-HR-56165, and by the Deseret Foundation, and Intermountain Healthcare, Inc.
AUTHOR CONTRIBUTIONS
All authors made substantial contributions to the conception or design of this work, revised the drafts critically for important content, approved the final version, and are accountable for the arguments contained therein. AHM wrote the initial draft. The following authors participated as investigators in the past work discussed in this JAMIA Perspective: AHM, ML, JO, Jr, TPC, LKW, FT, CG, EH, TDE, CJW, MPY, DFS, MB, EB, KAS, SSP, GRB, BTT, RB, JDT, JS, RDH, DFW, JJZ, VMN, CJLN, JL, KHL, BPD, DKS, AW, PJH, UP, SR, DK, SA, DAS, DA, and MRP.
SUPPLEMENTARY MATERIAL
Supplementary material is available at Journal of the American Medical Informatics Association online.
DATA AVAILABILITY
No new data were generated or analyzed in support of this perspective manuscript.
Supplementary Material
ACKNOWLEDGMENTS
We are indebted to many RN, RRT, MD and informatics technology colleagues who enabled the execution of this work during the past 35 years. We acknowledge important arguments by Roberta Goldring, MD, Waldemar Johanson, MD, and Robert Rogers, MD, in support of initial NIH funding; incisive editorial critiques by Gordon Rubenfeld, MD, MSc; past investigator contributions by Bruce A. McKinley, PhD, whom we were unable to contact; and the central role of the HELP system medical informatics infrastructure provided by Homer Warner, MD, PhD, T. Alan Pryor, PhD, and Reed M. Gardner, PhD. Finally, Marilyn S. Morris, PT, MS, convinced Alan Morris, MD, to teach a course titled “Human Cognitive Limitations: Why Humans Overestimate Performance and So Commonly Make Bad Decisions.” This led to exploration of many constructs included in this publication.
CONFLICT OF INTEREST STATEMENT:
None declared.
References
- 1. Berwick DM. Era 3 for medicine and health care. JAMA 2016; 315 (13): 1329–30. [DOI] [PubMed] [Google Scholar]
- 2. Stanovich KE. What Intelligence Tests Miss: The Psychology of Rational Thought. New Haven, CT: Yale University Press; 2009: xv, 308. [Google Scholar]
- 3. Henrich JP. The WEIRDest People in the World: How the West Became Psychologically Peculiar and Particularly Prosperous. New York: Farrar, Straus and Giroux; 2020. [Google Scholar]
- 4.Bernard S, James B, Bayley KB. Cost of Poor Quality or Waste in Integrated Delivery System Settings. Agency for Healthcare research and Quality: RTI International; 2006 September 2006. Report No.: AHRQ Publication No. 08-0096-EF Contract No.: RTI Project No. 0207897.011.
- 5. Diamond GA, Pollock BH, Work JW.. Clinician decisions and computers. J Am Coll Cardiol 1987; 9 (6): 1385–96. [DOI] [PubMed] [Google Scholar]
- 6. Wasson JH, Sox HC, Neff RK, Goldman L.. Clinical prediction rules. Applications and methodological standards. N Engl J Med 1985; 313 (13): 793–9. [DOI] [PubMed] [Google Scholar]
- 7. Hammond KR. Human Judgment and Social Policy: irreducible Uncertainty, Inevitable Error, Unavoidable Injustice. New York: Oxford University Press; 1996: xi, 436. [Google Scholar]
- 8. Arkes H, Hammond K.. Judgment and Decision Making: An Interdisciplinary Reader. Cambridge, UK: Cambridge University Press; 1986. [Google Scholar]
- 9. Kleinmuntz B. Why we still use our heads instead of formulas: toward an integrative approach In: Connolly T, Arkes HR, Hammond KR, eds. Judgment and Decision Making: An Interdisciplinary Reader. 2nd ed. Cambridge, UK: Cambridge University Press; 2000: 681–711. [Google Scholar]
- 10. Kahneman D, Rosenfield A, Gandhi L, Blaser T.. NOISE: How to overcome the high, hidden cost of inconsistent decision making. Harv Bus Rev 2016; 94 (10): 38–46. [Google Scholar]
- 11. Sutcliffe KM, Paine L, Pronovost PJ.. Re-examining high reliability: actively organising for safety. BMJ Qual Saf 2017; 26 (3): 248–51. [DOI] [PubMed] [Google Scholar]
- 12. Croskerry P. From mindless to mindful practice–cognitive bias and clinical decision making. N Engl J Med 2013; 368 (26): 2445–8. [DOI] [PubMed] [Google Scholar]
- 13. Deming W. Out of the Crisis. Cambridge, MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study; 1986. [Google Scholar]
- 14. Williamson J, German P, Weiss R, Skinner E, Bowes F.. Health science information management and continuing education of physicians. Ann Intern Med 1989; 110 (2): 151–60. [DOI] [PubMed] [Google Scholar]
- 15. Beshay A, Liu M, Fox L, Shinkai K.. Inpatient dermatology consultative programs: A continued need, tools for needs assessment for curriculum development, and a call for new methods of teaching. J Am Acad Dermatol 2016; 74 (4): 769–71. [DOI] [PubMed] [Google Scholar]
- 16. Porter-Armstrong AP, Moore ZE, Bradbury I, McDonough S.. Education of healthcare professionals for preventing pressure ulcers. Cochrane Database Syst Rev 2018; 5: CD011620. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Jena AB, Farid M, Blumenthal D, Bhattacharya J.. Association of residency work hour reform with long term quality and costs of care of US physicians: observational study. BMJ 2019; 366: l4134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. James BC. Making it easy to do it right. N Engl J Med 2001; 345 (13): 991–3. [DOI] [PubMed] [Google Scholar]
- 19. Thaler RH, Sunstein CR.. Nudge: Improving Decisions about Health, Wealth, and Happiness. New Haven, CT: Yale University Press; 2008: 293. [Google Scholar]
- 20. Weir CR, Taber T, Reese TR, Del Fiol G.. Feeling and thinking: can theories of human motivation explain how EHR design impacts clinician burn-out? J Am Med Inform Assoc. doi: 10.1093/jamia/ocaa270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. McLachlan S, Buchanan D, Lean S, et al. Learning health systems: the research community awareness challenge. BMJ Health Care Inform 2018; 25 (1): 38–40. [DOI] [PubMed] [Google Scholar]
- 22. Christensen CM, Grossman JH, Hwang J.. The Innovator's Prescription: A Disruptive Solution for Health Care. New York: McGraw-Hill; 2009: 441. [Google Scholar]
- 23. Heath C, Heath D.. Switch: How to Change Things When Change is Hard. 1st ed. New York: Broadway Books; 2010. [Google Scholar]
- 24. Morris AH, Orme J, Rocha BH; for the Reengineering Critical Care Clinical Research Investigators, et al. An electronic protocol for translation of research results to clinical practice: a preliminary report. J Diabetes Sci Technol 2008; 2 (5): 802–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Morris AH, Orme J Jr., Truwit JD, et al. A replicable method for blood glucose control in critically Ill patients. Crit Care Med 2008; 36 (6): 1787–95. [DOI] [PubMed] [Google Scholar]
- 26. Thompson B, Orme J, Zheng H; for the Reengineering Critical Care Clinical Research Investigators, et al. Multicenter validation of a computer-based clinical decision support tool for glucose control in adult and pediatric intensive care units. J Diabetes Sci Technol 2008; 2 (3): 357–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. East T, Heermann L, Bradshaw R, et al. Efficacy of computerized decision support for mechanical ventilation: results of a prospective multi-center randomized trial. Proc AMIA Symp 1999; 1999: 251–5; Washington, DC. [PMC free article] [PubMed] [Google Scholar]
- 28. Bakwin H. Pseudodoxia pediatrica. N Engl J Med 1945; 232 (24): 691–7. [Google Scholar]
- 29. Hebert PC, Wells G, Martin C, et al. A Canadian survey of transfusion practices in critically ill patients. Transfusion Requirements in Critical Care Investigators and the Canadian Critical Care Trials Group. Crit Care Med 1998; 26 (3): 482–7. [DOI] [PubMed] [Google Scholar]
- 30. Hirshberg E, Lacroix J, Sward K, Willson D, Morris AH.. Blood glucose control in critically ill adults and children: a survey on stated practice. Chest 2008; 133 (6): 1328–35. [DOI] [PubMed] [Google Scholar]
- 31. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348 (26): 2635–45. [DOI] [PubMed] [Google Scholar]
- 32. Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med 2007; 357 (15): 1515–23. [DOI] [PubMed] [Google Scholar]
- 33.IOM (Institute of Medicine Medicine). The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary. Washington, DC: The National Academies Press; 2010: 828. [PubMed] [Google Scholar]
- 34. Glasgow RE, Emmons KM.. How can we increase translation of research into practice? types of evidence needed. Annu Rev Public Health 2007; 28 (1): 413–33. [DOI] [PubMed] [Google Scholar]
- 35. Thirunavukarasu M. Closing the treatment gap. Indian J Psychiatry 2011; 53 (3): 199–201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Jansen S, White R, Hogwood J, et al. The “treatment gap” in global mental health reconsidered: sociotherapy for collective trauma in Rwanda. Eur J Psychotraumatol 2015; 6 (1): 28706. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Office of the Surgeon General. Facing Addiction in America—The Surgeon General’s Report on Alcohol, Drugs, and Health. Washington, DC: US Department of Health and Human Services; 2016. [PubMed] [Google Scholar]
- 38. Spivak ES, Kendall B, Orlando P, et al. Evaluation of outpatient parenteral antimicrobial therapy at a veterans affairs hospital. Infect Control Hosp Epidemiol 2015; 36 (9): 1103–5. [DOI] [PubMed] [Google Scholar]
- 39. Silverman M, Povitz M, Sontrop JM, et al. Antibiotic prescribing for nonbacterial acute upper respiratory infections in elderly persons. Ann Intern Med 2017; 166 (11): 765–74. [DOI] [PubMed] [Google Scholar]
- 40. Wennberg JE, Gittelsohn A.. Small area variation analysis in health care delivery. Science 1973; 142: 1102–8. [DOI] [PubMed] [Google Scholar]
- 41. Wennberg JE. Time to tackle unwarranted variations in practice. BMJ 2011; 342 (mar17 3): d1513. [DOI] [PubMed] [Google Scholar]
- 42. Newth CJL, Sward KA, Khemani RG, et al. Variability in usual care mechanical ventilation for pediatric acute respiratory distress syndrome: time for a decision support protocol? Pediatric Critical Care Medicine 2017; 18 (11): e521–e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Sward KA, Newth CJL, Khemani RG; Eunice Kennedy Shriver National Institute for Child Health and Human Development Collaborative Pediatric Critical Care Research Network (CPCCRN), et al. Potential acceptability of a pediatric ventilator management computer protocol. Pediatr Crit Care Med 2017; 18 (11): 1027–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Yong PL, Olsen L.. Roundtable on Evidence-Based Medicine, Institute of Medicine. The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary. Washington, DC: The National Academies Press; 2010. [PubMed] [Google Scholar]
- 45. Joynt Maddox KE, McClellan MB.. Toward evidence-based policy making to reduce wasteful health care spending. JAMA 2019; 322 (15): 1460–2. [DOI] [PubMed] [Google Scholar]
- 46. Shrank WH, Rogstad TL, Parekh N.. Waste in the US health care system: estimated costs and potential for savings. JAMA 2019; 322 (15): 1501–9. [DOI] [PubMed] [Google Scholar]
- 47. Berwick DM. Elusive waste: the fermi paradox in US health care. JAMA 2019; 322 (15): 1458–9. [DOI] [PubMed] [Google Scholar]
- 48. Arkes H. Impediments to accurate clinical judgment and possible ways to minimize their impact In: Arkes H, Hammond K, eds. Judgment and Decision Making: An Interdisciplinary Reader. Cambridge: Cambridge University Press; 1986: 582–92. [Google Scholar]
- 49. Kruger J, Dunning D.. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol 1999; 77 (6): 1121–34. [DOI] [PubMed] [Google Scholar]
- 50. Kruger J. Lake Wobegon be gone! The “below-average effect” and the egocentric nature of comparative ability judgments. J Pers Soc Psychol 1999; 77 (2): 221–32. [DOI] [PubMed] [Google Scholar]
- 51. Brunkhorst F, Engel C, Ragaller M, et al. Practice and perception—A nationwide survey of therapy habits in sepsis. Crit Care Med 2008; 36 (10): 2719–25. [DOI] [PubMed] [Google Scholar]
- 52. Stagg BC, Stein JD, Medeiros FA, et al. Special commentary: using clinical decision support systems to bring predictive models to the glaucoma clinic. Ophthalmol Glaucoma 2020; Aug 15:S2589-4196(20)30212-X. doi: 10.1016/j.ogla.2020.08.006. Epub ahead of print. PMID: 32810611 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Fischoff B, Slovic P, Lichtenstein S.. Knowing with certainty: the appropriateness of extreme confidence. J Exper Psychol 1977; 3 (4): 551–64. [Google Scholar]
- 54. Ostbye T, Yarnall KS, Krause KM, Pollak KI, Gradison M, Michener JL.. Is there time for management of patients with chronic diseases in primary care? Ann Fam Med 2005; 3 (3): 209–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55. Morris A. The importance of protocol-directed patient management for research on lung-protective ventilation In: Dreyfuss D, Saumon G, Hubamyr R, eds. Ventilator-Induced Lung Injury. Lung Biology in Health and Disease. 215 New York: Taylor & Francis Group; 2006: 537–610. [Google Scholar]
- 56. Marewski JN, Gigerenzer G.. Heuristic decision making in medicine. Dialogues Clin Neurosci 2012; 14 (1): 77–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. Bailey RW. Human Performance Engineering. 3rd ed.Upper Saddle River, NJ: Prentice Hall; 1996: 636. [Google Scholar]
- 58. Norretranders T. The User Illusion: Cutting Consciousness down to Size. New York, New York: Penguin Books; 1998. [Google Scholar]
- 59. Kahneman D. Thinking, Fast and Slow. 1st pbk. ed. New York: Farrar, Straus and Giroux; 2013: 499. [Google Scholar]
- 60. Sweeney K, Griffiths F.. Complexity and Healthcare: An Introduction. Abingdon, Oxon, UK: Radcliffe Medical Press; 2002. [Google Scholar]
- 61. Weber GM, Mandl KD, Kohane IS.. Finding the missing link for big biomedical data. JAMA 2014; 311 (24): 2479–80. [DOI] [PubMed] [Google Scholar]
- 62. Obermeyer Z, Lee TH.. Lost in thought—the limits of the human mind and the future of medicine. N Engl J Med 2017; 377 (13): 1209–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63. Li Y, Sperrin M, Ashcroft DM, van Staa TP.. Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar. BMJ 2020; 371: m3919. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Editorial. Pseudodoxia pediatrica. JAMA 2020; 324 (4): 402. [DOI] [PubMed] [Google Scholar]
- 65. Bertalanffy L, LaViolette PA.. A Systems View of Man. Boulder, CO: Westview Press; 1981: 180. [Google Scholar]
- 66. Bertalanffy L. General System Theory. New York: George Braziller; 1968: 295. [Google Scholar]
- 67. Stead W. Clinicians and the electronic health record as a learning tool. In: Olsen L, Aisner D, McGinnis JM, eds. The Learning Healthcare System: Workshop Summary (Roundtable on Evidence-Based Medicine). Washington, DC: The National Academies Press; 2007: 268–75. [PubMed] [Google Scholar]
- 68. Morris A. Computerized protocols to assist clinical research. In: Olsen L, Aisner D, McGinnis JM, eds. The Learning Healthcare System: Workshop Summary (Roundtable on Evidence-Based Medicine). Washington, DC: The National Academies Press; 2007: 61–80. [PubMed] [Google Scholar]
- 69.Committee on the Learning Health Care System in America (Institute of Medicine). Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press; 2012. [PubMed] [Google Scholar]
- 70. Faden RR, Kass NE, Goodman SN, Pronovost P, Tunis S, Beauchamp TL.. An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics. Hastings Cent Rep 2013; 43 (s1): S16–S27. [DOI] [PubMed] [Google Scholar]
- 71. Friedman C, Rubin J, Brown J, et al. Toward a science of learning systems: a research agenda for the high-functioning Learning Health System. J Am Med Inform Assoc 2015; 22 (1): 43–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72. Friedman CP, Rubin JC, Sullivan KJ.. Toward an information infrastructure for global health improvement. Yearb Med Inform 2017; 26 (01): 16–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73. Friedman CP, Allee NJ, Delaney BC, et al. The science of learning health systems: foundations for a new journal. Learn Health Sys 2017; 1 (1): e10020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74. McLachlan S, Dube K, Johnson O, et al. A framework for analysing learning health systems: Are we removing the most impactful barriers? Learn Health Sys 2019; 3 (4): e10189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75. Platt JE, Raj M, Wienroth M.. An analysis of the learning health system in its first decade in practice: scoping review. J Med Internet Res 2020; 22 (3): e17026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76. Jasny BR, Chin G, Chong L, Vignieri S.. Data replication & reproducibility. Again, and again, and again …. Introduction. Science 2011; 334 (6060): 1225. [DOI] [PubMed] [Google Scholar]
- 77. Stodden V, Guo P, Ma Z.. Toward reproducible computational research: an empirical analysis of data and code policy adoption by journals. PLoS One 2013; 8 (6): e67111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78. Manolagas SC, Kronenberg HM.. Reproducibility of results in preclinical studies: a perspective from the bone field. J Bone Miner Res 2014; 29 (10): 2131–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79. Flier JS. Irreproducibility of published bioscience research: Diagnosis, pathogenesis and therapy. Molecular Metabolism 2017; 6 (1): 2–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80. Heesen R. Why the reward structure of science makes reproducibility problems inevitable. The Journal of Philosophy 2018; cxv (12): 661–74. [Google Scholar]
- 81. Beam AL, Manrai AK, Ghassemi M.. Challenges to the reproducibility of machine learning models in health care. JAMA 2020; 323 (4): 305–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82. Hawe P, Shiell A, Riley T.. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ 2004; 328 (7455): 1561–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83. Atkins H. The three pillars of clinical research. BMJ 1958; 2 (5112): 1547–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84. Yang LJ-S, Chang KW-C, Chung KC.. Methodologically rigorous clinical research. Plast Reconstr Surg 2012; 129 (6): 979e–88e. 10.1097/PRS.0b013e31824eccb7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85. Meehl P. Clinical versus Statistical Prediction - a Theoretical Analysis and a Review of the Evidence. Minneapolis, MN: University of Minnesota Press; 1954: 149. [Google Scholar]
- 86. Dawes R, Faust D, Meehl P.. Clinical versus actuarial judgement. Science 1989; 243 (4899): 1668–74. [DOI] [PubMed] [Google Scholar]
- 87. Bickford RG. Automatic electroencephalographic control of general anesthesia. Electroencephalogr Clin Neurophysiol 1950; 2 (1-4): 93–6. [DOI] [PubMed] [Google Scholar]
- 88. Blesser W. A Systems Approach to Biomedicine. New York: McGraw-Hill Book Company; 1969. [Google Scholar]
- 89. Sheppard L, Kirklin J, Kouchoukos N.. Chapter 6-Computer-controlled interventions for the Acutely Ill Patient. Computers in Biomedical Research. vol. 4. New York: Academic Press; 1974: 135–48. [Google Scholar]
- 90. Sheppard L. Computer control of the infusion of vasoactive drugs. Ann Biomed Eng 1980; 8 (4-6): 431–4. [DOI] [PubMed] [Google Scholar]
- 91. Sheppard LC, Kouchoukos NT.. Automation of measurements and interventions in the systematic care of postoperative cardiac surgical patients. Med Instrum 1977; 11 (5): 296–301. [PubMed] [Google Scholar]
- 92. East T, Morris A, Wallace C, et al. A strategy for development of computerized critical care decision support systems (Adapted from an article in the Int J Clin Monit Comput 8:263–269, 1992. Reprinted with permission of Kluwer Academic Publishers.). In: Shabot M, Gardner R, eds. Computers and Medicine: Decision Support Systems in Critical Care. New York: Springer-Verlag; 1993: 343–53. [Google Scholar]
- 93. Morris AH. Paradigms in management In: Pinsky M, Dhainaut J, eds. Pathophysiologic Foundations of Critical Care Medicine. Baltimore, MD: Williams and Wilkens; 1993: 193–206. [Google Scholar]
- 94. Morris A, Wallace C, Menlove R, et al. Randomized clinical trial of pressure-controlled inverse ratio ventilation and extracorporeal CO2 removal for ARDS [erratum 1994; 149(3, Pt 1):838, Letters to the editor 1995; 151(1): 255–6, 1995; 151(4): 1269–70, and 1997; 156(3):1016–7]. Am J Respir Crit Care Med 1994; 149 (2): 295–305. [DOI] [PubMed] [Google Scholar]
- 95. East T, Morris A, Gardner R.. Chapter 101 computerized management of mechanical ventilation In: Grenvik A, Ayres S, eds. Textbook of Critical Care. 3rd ed.Philadelphia, PA: W. B. Saunders; 1995: 895–911. [Google Scholar]
- 96. Morris A. Developing and implementing computerized protocols for standardization of clinical decisions. Ann Intern Med 2000; 132 (5): 373–83. [DOI] [PubMed] [Google Scholar]
- 97. McKinley BA, Moore FA, Sailors RM, et al. Computerized decision support for mechanical ventilation of trauma induced ARDS: results of a randomized clinical trial. J Trauma 2001; 50 (3): 415–24. [DOI] [PubMed] [Google Scholar]
- 98. McKinley BA, Moore LJ, Sucher JF, et al. Computer protocol facilitates evidence-based care of sepsis in the surgical intensive care unit. J Trauma 2011; 70 (5): 1153–67. [DOI] [PubMed] [Google Scholar]
- 99. Blagev DP, Hirshberg EL, Sward K, et al. The evolution of eProtocols that enable reproducible clinical research and care methods. J Clin Monit Comput 2012; 26 (4): 305–17. [DOI] [PubMed] [Google Scholar]
- 100. Agus MSD, Wypij D, Hirshberg EL, et al. Tight glycemic control in critically ill children. N Engl J Med 2017; 376 (8): 729–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 101. Weaver W. Science and complexity. Am Sci 1948; 36 (4): 536–44. [PubMed] [Google Scholar]
- 102. Ioannidis JPA. Why most published research findings are false. PLoS Med 2005; 2 (8): e124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 103. Ioannidis JP. Why most clinical research is not useful. PLoS Med 2016; 13 (6): e1002049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 104. Morris AH. Human cognitive limitations. broad, consistent, clinical application of physiological principles will require decision support. Annals Ats 2018; 15 (Supplement_1): S53–S6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 105. Fraser AG, Dunstan FD.. On the impossibility of being expert. BMJ 2010; 341 (dec14 1): c6815. [DOI] [PubMed] [Google Scholar]
- 106. Kroth PJ, Morioka-Douglas N, Veres S, et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (8): e199609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 107. Margolis CZ. Uses of clinical algorithms. JAMA 1983; 249 (5): 627–32. [PubMed] [Google Scholar]
- 108. Herbert SI, Gordon CJ, Jackson-Smale A, Salis JL.. Protocols for clinical care. Comput Methods Programs Biomed 1995; 48 (1-2): 21–6. [DOI] [PubMed] [Google Scholar]
- 109. Schwartz WB. Medicine and the computer. The promise and problems of change. N Engl J Med 1970; 283 (23): 1257–64. [DOI] [PubMed] [Google Scholar]
- 110. Overhage JM, McCallie D Jr.. Physician time spent using the electronic health record during outpatient encounters: a descriptive study. Ann Intern Med 2020; 172 (3): 169–74. [DOI] [PubMed] [Google Scholar]
- 111. Overhage JM, McCallie D.. Physician time spent using the electronic health record during outpatient encounters. Ann Intern Med 2020; 173 (7): 594–5. [DOI] [PubMed] [Google Scholar]
- 112. Matheny ME, Whicher D, Thadaney Israni S.. Artificial intelligence in health care: a report from the National Academy of Medicine. JAMA 2020; 323 (6): 509. [DOI] [PubMed] [Google Scholar]
- 113. Rogers EM. Diffusion of Innovations. 4th ed.New York: Free Press; 1995: 518. [Google Scholar]
- 114. Southon FCG, Sauer C, Dampney CNG.. Information technology in complex health services: organizational impediments to successful technology transfer and diffusion. J Am Med Inform Assoc 1997; 4 (2): 112–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115. Savitz LA, Kaluzny AD.. Assessing the implementation of clinical process innovations: a cross-case comparison. J Healthc Manag 2000; 45 (6): 366–79. [PubMed] [Google Scholar]
- 116. Dobbin F, Simmons B, Garrett G.. The global diffusion of public policies: social construction. Annu Rev Sociol 2007; 33 (1): 449–72. [Google Scholar]
- 117. Greenhalgh T, Robert G, MacFarlane F, Bate P, Kyriakidou O.. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82: 581–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 118.The Editors. OHRP and standard-of-care research. N Engl J Med 2014; 371 (22): 2125–6. [DOI] [PubMed] [Google Scholar]
- 119. Lantos JD, Spertus JA.. The concept of risk in comparative-effectiveness research. N Engl J Med 2014; 371 (22): 2129–30. [DOI] [PubMed] [Google Scholar]
- 120. Grossmann C, Powers B, McGinnis JM, (Eds.), Institute of Medicine. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: The National Academies Press; 2009: 253 p. [Google Scholar]
- 121. DeMets DL, Psaty BM, Fleming TR.. When can intermediate outcomes be used as surrogate outcomes? JAMA 2020; 323 (12): 1184–5. [DOI] [PubMed] [Google Scholar]
- 122. Kawamoto K, Lobach DF.. Clinical decision support provided within physician order entry systems: a systematic review of features effective for changing clinician behavior. AMIA Annu Symp Proc 2003; 2003: 361–5. [PMC free article] [PubMed] [Google Scholar]
- 123. Tcheng J, Bakken S, Bates D, Bonner H III, Ghandi T, Josephs M, eds. Optimizing Strategies for Clinical Decision Support: Summary of a Meeting Series. Washington, DC: National Academy of Medicine, National Academies Press; 2017. [PubMed] [Google Scholar]
- 124. Gostin LO, Hodge JG Jr.. US emergency legal responses to novel coronavirus: balancing public health and civil liberties. JAMA 2020; 323 (12): 1131–2. [DOI] [PubMed] [Google Scholar]
- 125. Berwick DM, Shine K.. Enhancing private sector health system preparedness for 21st-century health threats: foundational principles from a national academies initiative. JAMA 2020; 323 (12): 1133–4. [DOI] [PubMed] [Google Scholar]
- 126.Epictetus. The Enchiridion. New York: The Liberal Arts Press; 1958. [Google Scholar]
- 127. Bacon F, Fowler T.. Novum Organum. 2nd. ed. corr. & rev. ed. Oxford, UK: Clarendon Press; 1889. [Google Scholar]
- 128. Bacon R, Burke RB.. The Opus Majus of Roger Bacon. Bristol, UK: Thoemmes; 2000. [Google Scholar]
- 129. Browne T. Pseudodoxia Epidemica: Or, Enquiries into Very Many Received Tenents, and Commonly Presumed Truths. 4th ed.London: Printed for E. Dod; 1658: 468. [Google Scholar]
- 130. Risen JL. Believing what we do not believe: acquiescence to superstitious beliefs and other powerful intuitions. Psychol Rev 2016; 123 (2): 182–207. [DOI] [PubMed] [Google Scholar]
- 131. Cassel CK, Saunders RS.. Engineering a better health care system: a report from the president’s council of advisors on science and technology. JAMA 2014; 312 (8): 787. [DOI] [PubMed] [Google Scholar]
- 132. Chassin MR, Loeb JM.. High-reliability health care: getting there from here. Milbank Quarterly 2013; 91 (3): 459–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 133. Chassin MR. Improving the quality of health care: what's taking so long? Health Aff (Millwood) 2013; 32 (10): 1761–5. [DOI] [PubMed] [Google Scholar]
- 134. Padula WV, Duffy MP, Yilmaz T, Mishra MK.. Integrating systems engineering practice with health-care delivery. Health Systems 2014; 3 (3): 159–64. [Google Scholar]
- 135. Cooper JB, Newbower RS, Long CD, McPeek B.. Preventable anesthesia mishaps: a study of human factors. Anesthesiology 1978; 49 (6): 399–406. [DOI] [PubMed] [Google Scholar]
- 136. Halpern SD, Ubel PA, Asch DA.. Harnessing the power of default options to improve health care. N Engl J Med 2007; 357 (13): 1340–4. [DOI] [PubMed] [Google Scholar]
- 137. Blumenthal-Barby JS, Cantor SB, Russell HV, Naik AD, Volk RJ.. Decision aids: when ‘nudging’ patients to make a particular choice is more ethical than balanced, nondirective content. Health Affairs (Project Hope) 2013; 32 (2): 303–10. [DOI] [PubMed] [Google Scholar]
- 138. Bangor A, Kortum P, Miller J.. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Studies 2009; 4 (3): 114–23. [Google Scholar]
- 139. Melnick ER, Dyrbye LN, Sinsky CA, et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (3): 476–87. [DOI] [PubMed] [Google Scholar]
- 140. Phansalkar S, Weir CR, Morris AH, Warner HR.. Clinicians’ perceptions about use of computerized protocols: a multicenter study. Int J Med Inform 2008; 77 (3): 184–93. [DOI] [PubMed] [Google Scholar]
- 141. Wong AF, Pielmeier U, Haug PJ, Andreassen S, Morris AH.. An in silico method to identify computer-based protocols worthy of clinical study: An insulin infusion protocol use case. J Am Med Inform Assoc 2016; 23 (2): 283–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 142. Karbing DS, Spadaro S, Dey N, et al. An open-loop, physiologic model-based decision support system can provide appropriate ventilator settings. Crit Care Med 2018; 46 (7): e642–e8. [DOI] [PubMed] [Google Scholar]
- 143. Pryor TA, Warner HR, Gardner RM, Clayton PD, Haug PJ.. The HELP system development tools In: Orthner H, Blum B, eds. Implementing Health Care Information Systems. New York: Springer-Verlag; 1989: 365–83. [Google Scholar]
- 144. Kuperman GJ, Garder RM, Pryor TA.. HELP: A Dynamic Hospital Information System. Orthner HF, ed. New York: Springer-Verlag; 1991: xxiii, 334 p. [Google Scholar]
- 145. Morris AH, Gardner RM.. Computer applications In: Hall J, Schmidt G, Wood L, eds. Principles of Critical Care. New York: McGraw-Hill; 1992: 500–14. [Google Scholar]
- 146. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282 (15): 1458–65. [DOI] [PubMed] [Google Scholar]
- 147. Bello AK, Ronksley PE, Tangri N, et al. Quality of chronic kidney disease management in canadian primary care. JAMA Netw Open 2019; 2 (9): e1910704-e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 148. Moseby-Knappe M, Westhall E, Backman S, et al. Performance of a guideline-recommended algorithm for prognostication of poor neurological outcome after cardiac arrest. Intensive Care Med 2020; 46 (10): 1852–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 149. Grote T, Berens P.. On the ethics of algorithmic decision-making in healthcare. J Med Ethics 2020; 46 (3): 205–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 150. Keitel K, Kagoro F, Samaka J, et al. A novel electronic algorithm using host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-POCT): a randomized, controlled non-inferiority trial. PLoS Med 2017; 14 (10): e1002411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 151. Bihorac A, Ozrazgat-Baslanti T, Ebadi A, et al. MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann Surg 2019; 269 (4): 652–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 152. Cordingley JJ, Vlasselaers D, Dormand NC, et al. Intensive insulin therapy: enhanced Model Predictive Control algorithm versus standard care. Intensive Care Med 2009; 35 (1): 123–8. [DOI] [PubMed] [Google Scholar]
- 153. Meijering S, Corstjens AM, Tulleken JE, Meertens JH, Zijlstra JG, Ligtenberg JJ.. Towards a feasible algorithm for tight glycaemic control in critically ill patients: a systematic review of the literature. Crit Care 2006; 10 (1): R19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 154. Davidson M, Mayer B, Castellanos M, Duran P, Karlan V.. Effective diabetes care by a registered nurse following treatment algorithms in a minority population. Am J Manag Care 2006; 12: 226–32. [PubMed] [Google Scholar]
- 155. Armstrong R, Bullen C, Cohen S, Singer M, Webb A.. Critical Care Algorithms. New York, NY: Oxford University Press; 1991: 100. [Google Scholar]
- 156. Shoemaker WC. A patient care algorithm for cardiac arrest. Crit Care Med 1978; 4: 157. [DOI] [PubMed] [Google Scholar]
- 157. Boutros AR, Hoyt JL, Boyd WC, Harford CE.. Algorithm for management of pulmonary complications in burn patients. Crit Care Med 1977; 5 (2): 89–92. [DOI] [PubMed] [Google Scholar]
- 158. Mittelstadt BD, Allo P, Taddeo M, Wachter S, Floridi L.. The ethics of algorithms: mapping the debate. Big Data & Society 2016; 3 (2): 205395171667967. [Google Scholar]
- 159. Eaneff S, Obermeyer Z, Butte AJ.. The case for algorithmic stewardship for artificial intelligence and machine learning technologies. JAMA 2020; 324 (14): 1397. [DOI] [PubMed] [Google Scholar]
- 160.Wikipedia. Medical Algorithm 2020. https://en.wikipedia.org/wiki/Medical_algorithm Accessed July 2020.
- 161. Dykes A. Algorithm Definition & Meaning 2020. http://www.webopedia.com/TERM/A/algorithm.html Accessed July 2020.
- 162. Marini JJ, Gattinoni L.. Ventilatory management of acute respiratory distress syndrome: a consensus of two. Crit Care Med 2004; 32 (1): 250–5. [DOI] [PubMed] [Google Scholar]
- 163. East TD, Böhm SH, Wallace CJ, et al. A successful computerized protocol for clinical management of pressure control inverse ratio ventilation in ARDS patients. Chest 1992; 101 (3): 697–710. [DOI] [PubMed] [Google Scholar]
- 164. Khanji MY, Bicalho VV, van Waardhuizen CN, Ferket BS, Petersen SE, Hunink MG.. Cardiovascular risk assessment: a systematic review of guidelines. Ann Intern Med 2016; 165 (10): 713–22. [DOI] [PubMed] [Google Scholar]
- 165. Mitchell M. Complexity: A Guided Tour. Oxford, UK: Oxford University Press; 2009: xvi, 349. [Google Scholar]
- 166. Cowan N. The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav Brain Sci 2001; 24 (1): 87–114. [DOI] [PubMed] [Google Scholar]
- 167. Grant B. Pimp your PowerPoint: Epitomics, Inc. (an Abcam Company); 2010. http://www.the-scientist.com/? articles.view/articleNo/28818/title/Pimp-your-PowerPoint/ Accessed March 1 2010
- 168.An Initiative of the Risk Response Network. Global Risks 2013. 8th ed. Howell L, ed. Geneva: World Economic Forum; 2013. [Google Scholar]
- 169. Bogaisky J. Quants-R-us? Algorithmic trading trickles down to individual investors. Forbes. September 4, 2013.
- 170. Pielmeier U, Rousing M, Andreassen S, Nielsen B, Haure P.. Decision support for optimized blood glucose control and nutrition in a neurotrauma intensive care unit: preliminary results of clinical advice and prediction accuracy of the Glucosafe system. J Clin Monit Comput 2012; 26 (4): 319–28. [DOI] [PubMed] [Google Scholar]
- 171. Orme J Jr, Romney JS, Hopkins RO, et al. Pulmonary function and health-related quality of life in survivors of acute respiratory distress syndrome. Am J Respir Crit Care Med 2003; 167 (5): 690–4. [DOI] [PubMed] [Google Scholar]
- 172. Morris A. Oscillation and damping in human decision-making systems. Am J Resp and Crit Care Med (ATS Proceedings Abstracts) Suppl 2000; 161 (3): A552. [Google Scholar]
- 173. Sittig DF, Morris AH, Beck E, Pace NL, Wallace CJ, Gardner RM.. A computerized respiratory care algorithm to standardize patient care in a controlled clinical trial. Methodology for Medical Decision Making 1987; 7 (4): 285. [Google Scholar]
- 174. Sittig DF, Pace NL, Gardner RM, Beck E, Morris AH.. Implementation of a computerized patient advice system using the HELP clinical information system. Comp Biomed Res 1989; 22 (5): 474–87. [DOI] [PubMed] [Google Scholar]
- 175. Hébert PC, Cook DJ, Wells G, Marshall J.. The design of randomized clinical trials in critically ill patients. Chest 2002; 121 (4): 1290–300. [DOI] [PubMed] [Google Scholar]
- 176. Rees SE, Toftegaard M, Andreassen S.. A method for calculation of arterial acid-base and blood gas status from measurements in the peripheral venous blood. Comput Methods Programs Biomed 2006; 81 (1): 18–25. [DOI] [PubMed] [Google Scholar]
- 177. Rees SE, Hansen A, Toftegaard M, Pedersen J, Kristiensen SR, Harving H.. Converting venous acid-base and oxygen status to arterial in patients with lung disease. Eur Respir J 2009; 33 (5): 1141–7. [DOI] [PubMed] [Google Scholar]
- 178. Khemani RG, Celikkaya EB, Shelton CR, et al. Algorithms to estimate PaCO2 and pH using noninvasive parameters for children with hypoxemic respiratory failure. Respir Care 2014; 59 (8): 1248–57. [DOI] [PubMed] [Google Scholar]
- 179. Karbing DS, Allerød C, Thorgaard P, et al. Prospective evaluation of a decision support system for setting inspired oxygen in intensive care patients. J Crit Care 2010; 25 (3): 367–74. [DOI] [PubMed] [Google Scholar]
- 180. Rees SE, Allerod C, Murley D, et al. Using physiological models and decision theory for selecting appropriate ventilator settings. J Clin Monit Comput 2006; 20 (6): 421–9. [DOI] [PubMed] [Google Scholar]
- 181. Tierney WM, Overhage JM, Murray MD, et al. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med 2003; 18 (12): 967–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 182. Tierney WM, Overhage JM, Murray MD, et al. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res 2005; 40 (2): 477–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 183. Sorenson D, Grissom C, Carpenter L, et al. A frame-based representation for a bedside ventilation weaning protocol. J Biomedical Informatics 2008; 41 (3): 461–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 184. Hunt SA, Abraham WT, Chin MH, et al. 2009 focused update incorporated into the ACC/AHA 2005 guidelines for the diagnosis and management of heart failure in adults: a report of the American College of Cardiology Foundation/American Heart Association Task Force on practice guidelines: developed in collaboration with the international society for heart and lung transplantation. Circulation 2009; 119 (14): e391-479. [DOI] [PubMed] [Google Scholar]
- 185. Fonarow GC, Yancy CW, Albert NM, et al. Heart failure care in the outpatient cardiology practice setting: findings from IMPROVE HF. Circ Heart Fail 2008; 1 (2): 98–106. [DOI] [PubMed] [Google Scholar]
- 186. Noto MJ, Domenico HJ, Byrne DW, et al. Chlorhexidine bathing and health care-associated infections: a randomized clinical trial. JAMA 2015; 313 (4): 369–78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 187. Janz DR, Semler MW, Lentz RJ; Facilitating EndotracheaL intubation by Laryngoscopy technique and apneic Oxygenation within the ICU Investigators and the Pragmatic Critical Care Research Group, et al. Randomized trial of video laryngoscopy for endotracheal intubation of critically Ill adults. Crit Care Med 2016; 44 (11): 1980–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 188. Semler MW, Self WH, Wanderer JP, et al. Balanced crystalloids versus saline in critically Ill adults. N Engl J Med 2018; 378 (9): 829–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 189. Casey JD, Janz DR, Russell DW, Vonderhaar DJ, et al. Bag-mask ventilation during tracheal intubation of critically Ill adults. N Engl J Med 2019; 380 (9): 811–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 190. Hersh AM, Hirshberg EL, Wilson EL, Orme JF, Morris AH, Lanspa MJ.. Lower glucose target is associated with improved 30-day mortality in cardiac and cardiothoracic patients. Chest 2018; 154 (5): 1044–51. [DOI] [PubMed] [Google Scholar]
- 191. Lanspa M, Hirshberg E, Phillips G, Holmen J, Stoddard G, Orme J.. Moderate glucose control is associated with increased mortality compared with tight glucose control in critically ill patients without diabetes. Chest 2013; 143(): (5): 1226–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 192. Lanspa MJ, Dickerson J, Morris AH, Orme JF, Holmen J, Hirshberg EL.. Coefficient of glucose variation is independently associated with mortality in critically ill patients receiving intravenous insulin. Crit Care 2014; 18 (2): R86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 193. Roux D, Moreau R, Dreyfuss D.. Albumin infusion in spontaneous bacterial peritonitis: another brick off the wall? Ann Intensive Care 2018; 8 (1): 99. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 194. Greenfield S. Can expert bias be reduced in medical guidelines? BMJ 2019; 367: l6882. [DOI] [PubMed] [Google Scholar]
- 195. Dreyfuss D, Ricard JD, Gaudry S.. Amphotericin B deoxycholate for candidiasis in intensive care unit patients revisited: medical, ethical, and financial implications. Am J Respir Crit Care Med 2013; 187 (6): 661–3. [DOI] [PubMed] [Google Scholar]
- 196. Dreyfuss D, Ricard JD, Gaudry S.. Reply: continuous infusion of amphotericin B deoxycholate for the treatment of life-threatening Candida infections. Am J Respir Crit Care Med 2013; 188 (8): 1033–4. [DOI] [PubMed] [Google Scholar]
- 197. Taitsman JK, VanLandingham A, Grimm CA.. Commercial influences on electronic health records and adverse effects on clinical decision making. JAMA Intern Med 2020; 180 (7): 925. [DOI] [PubMed] [Google Scholar]
- 198. Glasziou PP, Sanders S, Hoffmann T.. Waste in covid-19 research. BMJ 2020; 369: m1847. [DOI] [PubMed] [Google Scholar]
- 199. Lum F, Lee P.. Waste in the US health care system—insights for vision health. JAMA Ophthalmol 2019; 137 (12): 1351. [DOI] [PubMed] [Google Scholar]
- 200. Bauchner H, Fontanarosa PB.. Waste in the US health care system. JAMA 2019; 322 (15): 1463. [DOI] [PubMed] [Google Scholar]
- 201. Figueroa JF, Wadhera RK, Jha AK.. Eliminating wasteful health care spending—is the united states simply spinning its wheels? JAMA Cardiol 2020; 5 (1): 9. [DOI] [PubMed] [Google Scholar]
- 202. Macleod MR, Michie S, Roberts I, et al. Biomedical research: increasing value, reducing waste. Lancet 2014; 383 (9912): 101–4. [DOI] [PubMed] [Google Scholar]
- 203. Hoff T. Deskilling and adaptation among primary care physicians using two work innovations. Health Care Manage Rev 2011; 36 (4): 338–48. [DOI] [PubMed] [Google Scholar]
- 204. Girard TD, Alhazzani W, Kress JP, et al. An official American Thoracic Society/American College of Chest Physicians Clinical Practice Guideline: liberation from mechanical ventilation in critically ill adults. rehabilitation protocols, ventilator liberation protocols, and cuff leak tests. Am J Respir Crit Care Med 2017; 195 (1): 120–33. [DOI] [PubMed] [Google Scholar]
- 205. Whitehead AN. An Introduction to Mathematics. London: Williams and Norgate; 1911: 256. [Google Scholar]
- 206. Russell B. Education and the Good Life. New York: Boni & Liveright; 1926: vi, 319. [Google Scholar]
- 207. Prasad M, Holmboe ES, Lipner RS, et al. Clinical protocols and trainee knowledge about mechanical ventilation. JAMA 2011; 306 (9): 935–41. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
No new data were generated or analyzed in support of this perspective manuscript.