Decision-Making During High-Risk Events: A Systematic Literature Review

Abstract

Effective decision-making in crisis events is challenging due to time pressure, uncertainty, and dynamic decisional environments. We conducted a systematic literature review in PubMed and PsycINFO, identifying 32 empiric research papers that examine how trained professionals make naturalistic decisions under pressure. We used structured qualitative analysis methods to extract key themes. The studies explored different aspects of decision-making across multiple domains. The majority (19) focused on healthcare; military, fire and rescue, oil installation, and aviation domains were also represented. We found appreciable variability in research focus, methodology, and decision-making descriptions. We identified five main themes: (1) decision-making strategy, (2) time pressure, (3) stress, (4) uncertainty, and (5) errors. Recognition-primed decision-making (RPD) strategies were reported in all studies that analyzed this aspect. Analytical strategies were also prominent, appearing more frequently in contexts with less time pressure and explicit training to generate multiple explanations. Practitioner experience, time pressure, stress, and uncertainty were major influencing factors. Professionals must adapt to the time available, types of uncertainty, and individual skills when making decisions in high-risk situations. Improved understanding of these decisional factors can inform evidence-based enhancements to training, technology, and process design.

Introduction

Decision-making is particularly challenging in acute situations such as trauma care and fire and rescue, due to the time pressured nature of the events, and the uncertain and dynamic environments in which they occur. An important line of research has been the study of naturalistic decision-making in complex, high-risk settings, yielding descriptive models at different levels of generalizability. Four high-level categories of decision-making strategies have been described (Kahneman & Klein, 2009; Klein, 1993; Pauley et al., 2011, 2013; Rasmussen, 1983), including: (1) Recognition-primed decision-making (RPD) (sometimes referred to as “intuitive”), (2) analytical, (3) rule-based, and (4) creative or innovative. The RPD model describes the intuitive, pattern-based situational decision-making made by experts in many domains (Klein, 1993). Analytical decision-making involves systematically collecting and analyzing relevant information to decide the course of action (Kahneman & Klein, 2009). Rule-based decision-making involves following a known protocol or algorithm to respond to an event or situation (Rasmussen, 1983). Creative or innovative decision-making occurs in unusual situations where standard approaches do not apply, and adaptation is required (Kaempf et al., 1996).

These decision-making strategies are not necessarily distinct. For example, rule-based decisions are sometimes made as part of RPD. When decision makers recognize a situation as routine, they may choose actions based on standard operating procedures or rules. In other cases, decision makers recognize that standard rules do not apply and come up with an innovative decision as part of RPD. Rule-based decision-making is also used to describe situations in which a (generally inexperienced) decision maker blindly applied rules without adequately understanding the situation.

Healthcare routinely requires decision-making within inherently uncertain situations; however, medical decision-making varies depending on the nature of the patient care. Studies have found that clinical decision-making in slow-paced situations (e.g., chronic care) frequently includes normative (analytical) decision-making (Eddy, 1996). Less is known about decision-making in acute, high-consequence naturalistic environments. For instance, in the field of anesthesiology, clinicians must respond, often quickly, to acute multivariate alterations in patient physiology during dynamic conditions. An early model of anesthesiology decision-making by Gaba and colleagues (2014) identified multiple phases of decision-making, including event recognition, problem solving, ongoing situation awareness (or sense-making), iterative cycles of treatment and diagnosis, escalation of care, and resource management, and suggested that anesthesiology decision-making incorporates recognition-primed, analytic, and rule-based decision strategies. Trauma resuscitations represent another high-risk clinical domain in which decisions must be made quickly, often with limited information. Groombridge et al., (2019) conducted a systematic review to evaluate the effect of stress on decision-making during resuscitations, finding both stressors (e.g., fatigue and noise) and mediators (e.g., checklists) that affected decision-making.

To improve patient outcomes, we need to better understand strategies skilled performers use to manage critical events and the complexities that make errors more likely. In this study, we sought to identify decision-making strategies used by skilled performers in responding to critical events across domains through a systematic review of the literature.

Methods

We conducted a systematic literature review to identify decision-making strategies used during such events. Our primary inclusion criteria included empirical studies of the cognitive and behavioral aspects of naturalistic decision-making by professionals during high-risk events. We excluded studies of routine decision-making and those exploring decision-making by teams or laypersons. Table 1 lists our full inclusion and exclusion criteria.

TABLE 1:

Inclusion and Exclusion Criteria for the Literature Review.

Inclusion	Exclusion
Focused on naturalistic decision-making in high-risk and stressful settings	Routine decision-making or decision-making not involving time pressure (i.e., where the participant has hours or days to act, rather than seconds or minutes)
Focused on the behavioral and cognitive aspects of decision-making	Physiological studies (e.g., changes in brain chemistry)
Studies of individual decision-making	Studies solely focused on team cognition or shared decision-making
Empirical, original research	Theoretical or non-original research (e.g., theoretical models and literature reviews)
Decision-making by trained professionals or professional trainees (e.g., nursing students, residents, and physicians in specialty training)	Decision-making by laypersons (e.g., patients, consumers, and civilians)
English language	Non-English language

Data Collection

In consultation with a health science information specialist in the Eskind Biomedical Library at Vanderbilt University School of Medicine, we identified and pilot-tested candidate search terms in PubMed and PsycINFO. We used an automated strategy (i.e., adding “NOT” logic statements) to electronically exclude decision-making topics not relevant to our search that otherwise would have been captured by our key inclusion search terms to help reduce the amount of manual title and abstract review required. The final search strategy was a combination of free-text and controlled vocabulary (MESH terms), yielding 759 papers in PubMed and 258 in PsycINFO (1017 total from our search term strategy) published by October 26, 2021 (see Online Appendix 1 for detailed search terms). We then added 44 papers identified by reference list review (13) or recommended by members of the research team (31) to supplement the papers identified by our search terms and reduce the risk of missing relevant studies (Flottorp et al., 2013; Horsley et al., 2011). Two research team members (CR and SA) independently reviewed the titles and abstracts for relevance, reaching consensus for any disagreement. This process eliminated papers that did not meet our inclusion criteria (951), as well as duplicates (26), non-peer reviewed papers (4), and papers without an abstract available (11). All papers deemed relevant to the research question were subjected to a complete review (69).

Although decision-making during high-risk events has been studied in many domains, findings are difficult to generalize because investigators use different methods and characterize findings using different labels (Militello & Anders, 2018). To facilitate our synthesis of these 69 diverse studies, we developed a structured rubric (see Online Appendix 2), based on a synthesis of 24 existing models (Anders et al., 2022; Reale et al., 2021) relevant to naturalistic decision-making. We analyzed each model, extracted its essential components, and synthesized them into a consolidated working model for naturalistic decision-making. The major components of this synthesized model informed the rubric’s content (e.g., factors affecting decision-making, such as the environment or participant characteristics; phases of the decision-making process such as recognizing, critiquing, or implementing; and factors associated with successful or poor decisional performance). Following iterative pilot testing, we used this rubric to qualitatively code key findings and characteristics of the studies under review (e.g., domain, methods, and research objectives) to facilitate synthesis of results across domains and different research approaches.

The research team completed an initial review of five papers using the proposed rubric and met regularly throughout this process to reach consensus on rubric structure and code application. Six research team members reviewed the remaining 64 papers using the final rubric. Each reviewer flagged any papers that warranted a second reviewer. Two additional research team members reviewed any papers that were flagged, including those for which the reviewer was unsure whether it met inclusion criteria. Thirty-two of the sixty-nine reviewed papers (46%) were included in the final analysis (see Figure 1).

Figure 1.

PRISMA flow chart of paper selection process.

Data Analysis

We divided the 32 articles across three research team members who each systematically analyzed their assigned papers (Table 2). Each researcher read the paper and extracted the key findings to the rubric. The three researchers (CR, SA, and MS) discussed the findings and identified themes that emerged from the data. We then organized the extracted results into the identified themes. One researcher (CR) then reviewed the team’s extracted results to identify key similarities and differences within each theme. The same researchers met regularly during this synthesis process to review progress and reach consensus on the interpretation of the extracted findings.

TABLE 2:

Thirty-two Papers Included in this Systematic Review.

Author (Year)	Relevant Theme	Domain: Subjects (n)	Study Type: Methods	Stated Research Objectives
DeAnda and Gaba (1991)	Decision-making strategy, errors	Healthcare: Anesthesiologists in the operating room (27)	Experimental: High fidelity simulation, debrief interviews, and video analysis	To investigate experienced and trainee physicians’ detection of and response to problems during crisis events
Orasanu et al. (1993)	Time pressure and errors	Aviation: Pilots (not specified)	Qualitative: Simulated flight data and NTSB accident report analysis	To understand the cognitive requirements of pilots’ decision-making tasks and associated errors in the cockpit
Xiao et al., (1995)	Errors	Healthcare: Anesthesiologist in a trauma resuscitation (1)	Qualitative: Video analysis	To analyze fixation errors during a real-world trauma patient resuscitation
Flin et al. (1996)	Decision-making strategy, time pressure	Offshore drilling: Offshore installation managers on an oil rig (16)	Experimental: Paper-based simulation and critical decision method interviews	To examine offshore installation managers’ decision-making during a crisis event
Kaempf, et al. (1996)	Decision-making strategy, uncertainty	Military: Naval officers in combat incident command center (31)	Qualitative: Critical decision method interviews	To study how experienced naval officers make decisions in a complex, time-pressured command-and-control setting
McLennan And Omodei (1996)	Decision-making strategy	Fire and rescue: Firefighters (7)	Qualitative: Interviews	To explore decision makers’ engagement in mental simulation of a situation and potential actions prior to encountering the actual incident
Johnston et al. (1997)	Decision-making strategy, stress	Military: Navy personnel (90)	Experimental: Computer-based simulation	To examine the effectiveness of vigilant and hypervigilant decision-making strategies on a command-and-control task
Cioffi (2000)	Decision-making strategy, uncertainty	Healthcare: Nurses in acute care (32)	Qualitative: Unstructured interviews	To explore nurses’ experience calling the hospital’s emergency assistance and early intervention team for patients with signs of clinical deterioration
Dominguez (2001)	Decision-making strategy, errors	Healthcare: Surgeons (20)	Qualitative: Cognitive task analysis, field observations, and video case-based interviews	To study surgeons’ decision-making regarding the need to convert from a laparoscopic to an open incision case
Wetzel, et al. (2006)	Stress and uncertainty	Healthcare: Surgeons (16)	Qualitative: Interviews	To explore the impact of surgical stressors on surgeons’ performance and coping strategies
Mosier, et al. (2007)	Time pressure and uncertainty	Aviation: Pilots (93)	Experimental: Computer-based simulation	To examine the impact of certain operational variables on pilots’ diagnosis and decision-making processes in an automated cockpit
Cohen (2008)	Time pressure and stress	Military: Fighter pilot (1); army physician (1)	Qualitative: Case studies	To compare decision-making processes in time-critical and non-time-critical situations
Farnan, et al. (2008)	Uncertainty	Healthcare: Residents in general medicine (50)	Qualitative: Critical incident technique interviews	To describe the types of uncertainty resident physicians encounter, the impact on patient care, and their management strategies
Thompson, et al. (2008)	Time pressure	Healthcare: Nurse in acute care (241)	Experimental: Low fidelity simulation scenarios	To explore the impact of clinical experience and time pressure on nurses’ decision-making performance
Bearman et al. (2009)	Errors	Aviation: Pilots (24)	Qualitative: Critical incident technique interviews	To explore the influence of situational factors on pilots’ decision-making
Thompson, et al. (2009)	Decision-making strategy	Healthcare: Nurses in acute and critical care (245)	Experimental: Paper-based simulation	To understand nurses’ use of clinical information when making decisions about whether a patient is at risk of a critical event
Pauley, et al. (2011)	Decision-making strategy, time pressure	Healthcare: Surgeons (24)	Qualitative: Critical decision method interviews	To examine cognitive processes underlying surgeons’ intraoperative decision-making and risk management strategies
Tallentire, et al. (2011)	Stress and uncertainty	Healthcare: Residents in acute care (36)	Qualitative: Focus groups	To explore factors that affect new doctors’ behavior when caring for acutely ill patients
Bearman and Bremner (2013)	Errors	Fire and rescue: Volunteer fire fighters (10)	Qualitative: Critical decision method interviews, hierarchical task analysis, and SHERPA analysis	To investigate incident commanders’ decision-making tasks and potential errors and how errors are managed
Cristancho, et al. (2013)	Decision-making strategy	Healthcare: Surgeons (7)	Qualitative: Observations and critical decision method interviews	To explore surgeons’ processes for assessing and responding to non-routine challenges during surgery
Pauley et al. (2013)	Decision-making strategy	Healthcare: Surgeons (12)	Qualitative: Critical decision method interviews	To examine cognitive processes underlying surgeons’ intraoperative decision-making and risk management strategies
Pelaccia, et al. (2014)	Decision-making strategy, uncertainty	Healthcare: Physicians in emergency medicine (15)	Qualitative: Video-cued recall interviews	To identify when and how physicians generate and evaluate diagnostic hypotheses during initial patient encounters
Yang et al. (2014)	Time pressure	Healthcare: Nurses and nursing students in acute care (97)	Experimental: High fidelity simulation	To assess nurses’ decision-making strategies and outcomes under time pressure
Cohen-Hatton et al. (2015)	Decision-making strategy, time pressure	Fire and rescue: Incident commanders (23)	Qualitative: Video-cued recall interviews and video analysis	To understand fire and rescue incident commanders’ decision-making during real-world events
Cohen-Hatton and Honey (2015)	Decision-making strategy	Fire and rescue: Incident commanders (82)	Experimental: Virtual reality and in-person simulation scenarios	To assess whether goal-oriented training affects incident commanders’ development of explicit plans during emergency incidents
Hamui-Sutton, et al. (2015)	Uncertainty	Healthcare: Residents in internal medicine, OBGYN, surgery, and pediatrics (248)	Qualitative: Questionnaires	To identify the types of uncertainty resident physicians encounter during critical incidents and their coping strategies
Hausmann, et al. (2016)	Decision-making strategy, time pressure	Healthcare: Residents in emergency medicine (11)	Qualitative: Observations and interviews	To map physicians’ diagnostic decision-making processes during real emergency patient cases
Axelsson and Jansson (2018)	Time pressure	Multiple: Train drivers and dispatchers, high-speed ferry operators, long-haul truck drivers, and intensive care nurses (not specified)	Qualitative: Collegial verbalization method	To examine how decision-makers adapt their expertise under time pressure
White, et al. (2018)	Time pressure, uncertainty	Healthcare: Physicians in emergency medicine or critical care (10)	Qualitative: Eye-tracking video-cued recall interviews and cognitive task analysis	To study physicians’ decision-making processes during trauma resuscitations
Ilgen, Teunissen, et al. (2021)	Uncertainty	Healthcare: Physicians in emergency medicine (12)	Qualitative: Critical incident technique interviews	To investigate how physicians iteratively manage and respond to feelings of discomfort during patient care experiences of uncertainty
Shortland, et al. (2020)	Decision-making strategy, uncertainty	Military: Active duty and veteran members of the air force, army, navy, and marine corps (27)	Qualitative: Critical decision method interviews	To explore the psychological processes involved when deployed military personnel make “least-worst” decisions during critical incidents
Ilgen, Regehr, et al. (2021)	Uncertainty	Healthcare: Residents in emergency medicine (13)	Qualitative: Critical incident technique interviews	To understand the experiences of new physicians managing clinical situations that generate uncertainty

Results

Across the different domains of study in the 32 papers, we found appreciable variability in research focus, methodology, and descriptions of decision-making factors, which made synthesis challenging. Nevertheless, the structured analysis identified five main themes: (1) type of decision-making strategy, (2) time pressure, (3) stress, (4) uncertainty, and (5) errors. We present the relevant findings within each of these themes, followed by a summary of the evidence of how each theme impacts decision-making performance.

Decision-Making Strategies

Ten studies explored the overall strategy or strategies used to make decisions in acute, dynamic, high-risk events. Table 3 summarizes the domains, types of events studied, frequency of strategies observed, and descriptions used to classify decision-making strategies across the 10 studies. Four high-level categories of decision-making strategies were described: (1) recognition primed, (2) analytical, (3) rule-based, and (4) creative or innovative. Not all studies included all four categories in their analysis, and some did not separate rule-based from RPD. The level of detail provided about the decision-making strategies used ranged from detailed analysis of decision-making processes by incident (Cohen-Hatton et al., 2015; Kaempf et al., 1996) to summary statements in the Discussion (DeAnda & Gaba, 1991; Thompson et al., 2009).

TABLE 3:

Frequency and Descriptions of Decision-Making Strategies Identified Across Multiple Domains and Event Types.

Domain and Decision Context	RPD	Analytical	Rule-Based	Innovative/Creative
Naval officers identified a challenging incident from their CIC antiair warfare experience (Kaempf et al., 1996)	Most (95%, 74 actions) Used a recognitional process to identify a course of action	Few (4%, 3 actions) Selected a course of action from multiple options	Not separated from RPD Coded as a subcategory of RPD	Few (1%, 1 action) Generated a novel course of action
Oil installation managers completed a paper-based scenario of an offshore explosion (Flin et al., 1996)	Most Generated a single course of action based on SOP	Few Generated multiple options	Not separated from RPD	Not studied Scenario did not include this type of situation
Firefighters responded to actual fire and rescue service incidents (Cohen-Hatton et al., 2015)	Most Moved from situation assessment to plan execution (reflexive process)	Few Moved from situation assessment to plan formulation to plan execution (reflective process)	Not discussed	Not discussed
Emergency physicians treated actual patients admitted to the emergency department for a potentially life-threatening reason (Pelaccia et al., 2014)	Most Generated diagnostic hypotheses quickly without conscious effort	Few Generated diagnostic hypotheses through conscious effort	Not discussed	Not discussed
Anesthesiologists completed a high-fidelity simulation of an intraoperative procedure (DeAnda & Gaba, 1991)	Most Matched patterns based on most frequent explanation	Not enough evidence to confirm Formally generated and tested specific hypotheses	Not discussed	Not discussed
Surgeons identified an important decision point in a challenging case from their operative experience (Pauley et al., 2013)	Half (50%, 6 cases) Did not compare risks and benefits of more than one option	Half (50%, 6 cases) Weighed risks and benefits of more than one possible course of action	None Applied a documented procedure from guidelines	None Devised a novel solution in an unfamiliar situation
Surgeons identified an important decision point in a challenging case from their operative experience (Pauley et al., 2011)	Almost half (46%, 11 cases) Recalled only one option	Half (50%, 12 cases) Compared risks and benefits of more than one option	Few (4%, 1 case) Applied a documented procedure from guidelines	None Devised a novel solution in an unfamiliar situation
Surgeons identified a challenging case in which they anticipated making important decisions about the proper course of an operation (Cristancho et al., 2013)	Only process described Used a cyclical process to recognize the need to change the preoperative plan and mentally simulate alternative actions	Not discussed	Not discussed	Not discussed
Nurses described their experience calling the hospital’s early intervention team for patients with signs of deterioration (Cioffi, 2000)	Only process described Matched patterns and recognized a subjective ℌgut feeling” that something was wrong	Not discussed	Not discussed	Not discussed
Nurses determined the need to intervene in paper-based scenarios of patients exhibiting potential signs of deterioration (Thompson et al., 2009)	Most Used intuitive, non-linear reasoning	Not discussed	Not discussed	Not discussed

Healthcare dominated the domains studied (7 of 10 papers) with additional studies representing military commanders, firefighters, and offshore oil installation managers. Research methods and study design varied although 9 of 10 studies used interview methods to explore this question. Critical decision method (CDM) was the most common interview technique (5) (Flin et al., 1996; Kaempf et al., 1996; Pauley et al., 2011, 2013), followed by video-cued recall (2) (Cohen-Hatton et al., 2015; Pelaccia et al., 2014), post-simulation debrief interviews (1) (DeAnda & Gaba, 1991), and unstructured interviews (1) (Cioffi, 2000). Three studies involved simulations of varying fidelity, with two of these including a post-simulation interview as part of the data collection methods (DeAnda & Gaba, 1991; Flin et al., 1996; Thompson et al., 2009). Two studies complemented interview data with separate video analysis (Cohen-Hatton et al., 2015; DeAnda & Gaba, 1991) and one study involved direct observation of real-world cases followed by post-event interviews (Cristancho et al., 2013).

While it is likely that not all the events examined in this subset of studies rose to the level of a “crisis,” they appear to reasonably represent situations where the decision maker had to perform under stress and with a degree of inherent risk to the outcome. Furthermore, not all the events under study were time pressured. Several studies included a mix of high and low time pressure incidents (Cohen-Hatton et al., 2015; Pauley et al., 2011, 2013), and one study manipulated this factor as an intervention in their experimental design (Thompson et al., 2009).

When is RPD Used?

All 10 studies found that trained professionals use strategies consistent with RPD during acute event management. Of these, seven referred to RPD specifically while the other three used the more general term “intuitive” decision-making. We will use the term RPD herein for all 10 studies’ findings. Recognition-primed decision-making was the most frequently identified strategy across all of the studies except in the surgery domain.

Two of the three studies involving surgeons found an RPD strategy was employed about half of the time (Pauley et al., 2011, 2013). In the third study of surgeons (Cristancho et al., 2013), the authors developed a decision-making model that parallels RPD while noting that a key difference in the surgical domain is the presence of a preoperative plan the surgeon developed, based on known patient information before starting the procedure. This model describes a three-stage process that included a “reconciliation cycle” or cognitive shift that occurred when the surgeon detected any challenges to the pre-op plan. Aspects of this “reconciliation cycle” were consistent with RPD—obtaining information (relevant cues), weighing information against expected operative course (expectancies), and projecting future actions (mental simulation).

When is Analytical Decision-Making Used?

The two studies of surgeons’ decision-making processes found analytical strategies were employed in half of the cases (Pauley et al., 2011, 2013). In one study (Pauley et al., 2011), surgeons reported only brief consideration of multiple options. In other domains, analytical strategies were infrequently observed. Among naval officers (Kaempf et al., 1996), the few cases of analytical strategy occurred at less critical decision points. Emergency physicians’ use of analytical strategies occurred when they were ensuring no possible diagnoses had been missed (Pelaccia et al., 2014).

When is Rule-Based Decision-Making Used?

Strictly rule-based strategies were rarely identified, although it’s unclear how frequently rule-based factors may have come into play in the studies that failed to separate this strategy from RPD (Flin et al., 1996; Kaempf et al., 1996). In the oil installation study (Flin et al., 1996), following standard operating procedure was a frequent aspect of the RPD strategies identified. One of the studies of surgeons (Pauley et al., 2011) categorized a single case as using a rule-based strategy, but this case also had RPD characteristics. While not explicitly identifying rule-based processes, it was noted that many nurses used the organization’s early warning score system’s protocol guidance in their decision-making (Thompson et al., 2009). This suggests that both RPD and rule-based strategies may be used during the same incident.

When is Innovative/Creative Decision-Making Used?

Innovative or creative decision-making strategies were rarely identified, although most studies did not explicitly look for them. One study reported a single occurrence of a novel strategy (Kaempf et al., 1996). However, in a separate interview-based study of “least-worst” decision-making in critical incidents not included in Table 3, Shortland et al. specifically inquired about situations encountered by Armed Forces personnel where there was no established policy to follow nor any previous experience or training to draw upon when deciding between options. Although rare, most participants had encountered a unique decision-making situation in their military career and, given the ambiguous nature of the situation, a “more adaptive, potentially creative” decision-making strategy was required (Shortland et al., 2020).

How do Decision-Makers Implement RPD Strategies?

Recognition-primed decision-making-based strategies were by far the most identified decision-making strategies employed by trained professionals in high-risk events. Eleven studies (seven of the ten studies listed in Table 3 plus four additional studies) explored specific aspects of RPD strategies in more detail. Several studies suggested that the phases of the RPD model do not require completion in a rigidly stepwise manner; they may be skipped, overlap, or performed in different orders as the situation evolves (Cohen-Hatton et al., 2015; Cohen-Hatton & Honey, 2015; Cristancho et al., 2013; Kaempf et al., 1996).

Mental simulation, or evaluating a recognition generated course of action, is a step of the RPD model that may frequently be skipped. In a study of naval tactical action officers, Kaempf et al. (1996) delineate differences between simple forms of RPD where the decision-maker confidently recognizes the situation and generates an appropriate course of action without further evaluation of options, and more complex forms of RPD that call for mental simulation of the generated action to test how it will play out in that context. In that study, 78% of the recognition-generated actions were implemented without further evaluation. The naval officers’ primary concerns appeared to be situation assessment, not weighing alternative actions. Many of the incidents included determining whether an incoming track was hostile or friendly. High consequences, intense time pressure, as well as standardized procedures and rules of engagement likely contribute to the emphasis on assessment (Kaempf et al., 1996). In contrast, in the largely unstandardized surgical domain, where minutes rather than seconds may be available, mental simulation is often described as an element of decision-making. Although the prevalence was not quantified, surgeons commonly reported a process of thinking ahead about the pros and cons of their preferred option to ensure it was safe (Cristancho et al., 2013; Pauley et al., 2011, 2013).

McLennan and Omodei (1996) describe how firefighters, on the drive to the scene, mentally simulate aspects of the situation they are likely to confront upon arrival based on the initial available information. This finding extends the traditional RPD model, which described “experiencing the situation” as the starting point of the RPD process, to account for this “prepriming” that allows the individual to arrive on the scene with several likely scenarios and possible responses in mind. Likewise, Pelaccia et al. (2014) found that almost all of the emergency physicians generated at least one diagnostic hypothesis when they received initial patient information before meeting the patient, and one quarter of all hypotheses captured during the study were generated before the patient had been seen. Novice emergency medicine physicians reported an analogous pre-event process in which they mentally rehearsed their information-gathering plan and considered potential differential diagnoses before seeing the patient to decrease their discomfort from uncertainty and improve their chances for successful decision-making (Ilgen, Regehr, et al., 2021). Thus, the RPD process frequently begins upon receipt of any situational information.

Cognitive activities during fire and rescue incident commanders’ situation assessment were primarily perception and understanding, not anticipating or formative planning activities (Cohen-Hatton et al., 2015). Eighty-seven percent of naval officers’ situation assessment involved feature-matching (i.e., matching specific patterns of available environmental cues to identify the presenting situation) (Kaempf et al., 1996). If there was insufficient information to trigger recognition or available cues were contradictory, they relied on story building (12% of the time) by mentally simulating from prior experience to make sense of the current situation.

Experience was important to the effectiveness of situation assessment. Experienced fire and rescue incident commanders were more likely to follow initial plan formulation activities with plan execution rather than delaying to further assess the situation, especially in higher risk incidents with greater time pressure (Cohen-Hatton et al., 2015). In contrast, less experienced firefighters gathered additional information to better understand the situation before acting. More experienced surgeons detected critical intraoperative cues (i.e., a pulsating artery to confidently identify key anatomical structures) more frequently than residents (Dominguez, 2001). Similarly, the time it took anesthesiologists to detect a problem in a simulated crisis event generally decreased with increased professional experience (DeAnda & Gaba, 1991).

Kaempf et al.’s (1996) work with naval commanders also expanded the traditional RPD model to incorporate more complex forms of recognitional decision-making by including an iterative “diagnosing the situation” loop within the situation assessment phase; this occurs when a situation is not confidently recognized as typical. The authors state, “Even though a comparison is needed between hypotheses, this still fits the intent of the RPD model to explain how people can make decisions without comparing different courses of action” (Kaempf et al., 1996). Physicians’ differential diagnosis process when encountering emergency patient cases may share characteristics with this “diagnostic cycle” (Hausmann et al., 2016; Pelaccia et al., 2014). However, emergency physicians’ diagnostic testing and ranking process appears to occur continuously throughout a crisis event and not just as an initial step. The available studies provide insufficient evidence to determine how the differential diagnostic process relates or extends to decisions about actions.

How does Time Pressure Affect Decision-Making?

Time criticality may vary considerably between domains and types of crisis events even when the situation is extremely high risk (Cohen-Hatton et al., 2015). Time availability is a critical aspect of situation assessment that the decision maker must consider (Axelsson & Jansson, 2018; Cohen, 2008; Flin et al., 1996; Orasanu et al., 1993). Cohen-Hatton et al. (2015) describe an incident where firefighters responding to a road traffic collision that trapped a seriously injured driver in the vehicle determined that the injuries were not “time-critical” and therefore the safer, but more time-consuming, extraction plan could be enacted. Eye-tracking data from expert physicians leading trauma resuscitations revealed that a frequent focus of cognitive activity was maintaining temporal awareness and recognizing the need to expedite tasks based on the time available (White et al., 2018). Orasanu et al. (1993) identified poor estimation of time constraints and time availability as potential sources of error.

A related finding is the effect of time pressure more generally. Case studies of decision-making by a fighter pilot and a battlefield physician concluded that time pressure impacts decision-making strategy along two dimensions: the degree of time pressure to: (1) make a decision and (2) implement the decision (Cohen, 2008), suggesting that time pressure is associated with more automatic decisions and implementation of actions. Conversely, when time pressure is lower (Weinger et al., 1994), other cognitive decision-making approaches and systematic implementation are possible. Similarly, a task analysis study spanning five different domains (truck drivers, train engineers, train dispatchers, high-speed ferry drivers, and intensive care nurses) found that decision-maker’s assessment of time criticality forms a “mental time frame” that impacts decision-making strategies (Axelsson & Jansson, 2018). When time pressure was higher, expert decision-makers tended to drop non-critical tasks to focus on the most pressing matters; however, when time pressure was low, these experts were able to employ proactive planning.

Studies of pilots, oil installation managers, and nurses reported similar findings. They relied on fewer available cues to make satisfactory, rather than optimal, decisions when under time pressure (Flin et al., 1996; Mosier et al., 2007; Yang et al., 2014). Two studies where participants self-reported subjective time pressure after the event found it had no significant effect on emergency physicians’ diagnostic or surgeons’ intraoperative decision-making strategies (Hausmann et al., 2016; Pauley et al., 2011).

How does Stress Affect Decision-Making?

Stress is a subjective state inherent to decision-making in acute crisis situations (Cohen, 2008). Although one qualitative interview study identified some positive effects of stress on decision-making (Wetzel et al., 2006), namely, enhancing alertness, concentration, or focus, most relevant studies found stress to negatively impact decision-making.

High levels of stress disrupt cognition (Tallentire et al., 2011; Wetzel et al., 2006). In a focus group study, resident physicians reported feeling overwhelmed, panicked, unable to think, or paralyzed by stress when faced with diagnostic uncertainty in high pressure situations (Tallentire et al., 2011), described as pervasive and sometimes debilitating. Similarly, an interview study reported that “most surgeons had experienced situations in which they were unable to think clearly” when experiencing high stress (Wetzel et al., 2006). As a result of excessive stress, the surgeons described difficulty analyzing problems logically and determining the next step, even in situations where decision-making would normally be straightforward. Non-productive emotional responses (e.g., feeling the need to rush to think, decide, or act) and behavioral responses (e.g., fixation on a technical problem or reduced communication) contributed to decision-making difficulties under stress. Experience can play an important role in an individual’s ability to use effective coping strategies to mitigate the impact of stress on decision-making; senior surgeons described sophisticated coping techniques (e.g., using deliberate physical relaxation techniques as soon as internal stress cues were detected) while junior surgeons described more fragmented responses and less certainty about their ability to cope (Wetzel et al., 2006).

Uncertainty

Twelve studies explored aspects of uncertainty during decision-making, characterized as either an attribute of the environment (i.e., the decision-maker faced incomplete, outdated, inaccurate, or conflicting information) or personal (i.e., the decision-maker was uncertain about what to do in the situation). However, Shortland et al. identified a third category—“exogenous” uncertainty, or confusion about the roles and actions of others involved in the incident—as an understudied factor influencing decision-making (Shortland et al., 2020).

How does Environmental Uncertainty Affect Decision-Making?

There is generally limited time available to gather information during a crisis, so managing uncertainty is a necessity. When a situation was poorly understood, naval officers responded by preparing for the worst-case scenario, such as taking defensive actions to reduce the effects of an attack on the ship (Kaempf et al., 1996). In the face of informational conflicts, aircraft pilots took more time gathering and confirming information compared with situations with consistent situational cues (Mosier et al., 2007). Physicians reported diagnostic uncertainty in emergency encounters and trauma resuscitations (Pelaccia et al., 2014; White et al., 2018). Experienced emergency physicians actively remained open to alternative hypotheses as new information emerged to avoid premature closure (failing to consider reasonable alternatives after identifying an initial diagnosis) or anchoring bias (overreliance on the initial information received) (Pelaccia et al., 2014). Newly trained physicians described discomfort initiating treatment when faced with diagnostic uncertainty (Tallentire et al., 2011). This led to prematurely choosing a diagnosis and fixation on this initial diagnosis, even when subsequent information was disconfirming or inconsistent. Alternatively, experienced emergency physicians used those feelings of uncertainty as a trigger to focus their cognitive resources more intentionally, such as increased cue monitoring, stepping back to deliberately review the situation, and anticipating resources needed to manage potential consequences (Ilgen, Teunissen, et al., 2021).

How does Personal Uncertainty Affect Decision-Making?

A study exploring how residents dealt with uncertainty during actual critical incidents found stress and uncertainty to be strongly correlated (Hamui-Sutton et al., 2015). Other studies found that knowledge and skill-based uncertainty among newly trained physicians hinder decision-making performance, including making decisions about clinical interpretations, indications for procedures, goals of care, and when to escalate care (Farnan et al., 2008; Hamui-Sutton et al., 2015; Ilgen, Regehr, et al., 2021; Tallentire et al., 2011). When caring for acutely unwell patients or during critical incidents, resident physicians were frequently uncertain about how to apply abstract knowledge to a specific clinical situation (Hamui-Sutton et al., 2015; Tallentire et al., 2011). In response, these residents reported focusing on a specific task, such as locating equipment or placing an intravenous line, to distract them from the diagnostic uncertainty and avoid making tough decisions (Tallentire et al., 2011). These inexperienced physicians reported being reluctant to make decisions in acute situations because their fear of doing the wrong thing was worse than the consequences of doing nothing.

In one study, nurses sought input from their peers to help determine whether patients exhibiting potential signs of deterioration warranted a call to the emergency response team; less experienced nurses typically sought guidance from more experienced nurses (Cioffi, 2000). Similarly, Farnan et al. (2008) found that American medical residents followed a distinct “hierarchy of assistance” to attempt to resolve uncertainty that started with the literature and their peers and then moved sequentially to individuals with increasing seniority (Farnan et al., 2008). In contrast, Hamui-Sutton et al. (2015) reported that medical residents in Mexico City commonly coped with decision-making uncertainty by seeking input from senior physicians (almost half of the cases) and much less frequently consulted informational resources (9%) or their peers (8%). These coping strategies appeared to operate in parallel and did not follow a clear stepwise progression. In 9% of decisions made during critical incidents in the face of uncertainty, the residents did not review any additional resources or seek help from others. We do not know if the different findings of Franan et al. (2008) versus Hamui-Sutton et al. (2015) are due to differences in culture, medical training, healthcare system expectations, or other factors.

Several studies reported that newly trained surgeons, resident physicians, and nurses experienced uncertainty and reluctance to call for help in stressful acute patient care situations (Cioffi, 2000; Tallentire et al., 2011; Wetzel et al., 2006). Resident physicians described feeling the need to prove themselves before seeking help, and that they had to take a certain number of actions before calling a senior physician for help to avoid perceptions of “weak” performance (Tallentire et al., 2011). Similarly, nurses worried not only whether they were making the right decision to call for help but also feared feeling foolish if they did call and it was later determined that intervention was unnecessary (Cioffi, 2000).

What Factors Contribute to Decision-Making Errors?

Orasanu et al.’s (1993) study suggested that the primary cause of pilot decision errors was limitations in human cognition, specifically limited attention, working memory, and long-term memory retrieval. Stress can disrupt these processes, increasing the potential for decision-making errors during high-risk, high-pressure crisis events. Their analysis of simulated flight crew events and aviation accident reports revealed errors across the entire decision-making process, including failures to recognize a situation or relevant options, neglecting important information or tasks, and misjudging the degree of time pressure or risk.

Poor risk assessment may be due to “cognitive biases such as expectancy effects and over-reliance on representativeness” (Orasanu et al., 1993). Experience also plays an important mediating role. In a study of fire incident commanders, experienced commanders maintained awareness of the hazards and actively monitored for and employed strategies to mitigate them as incidents progressed (Bearman & Bremner, 2013). More experienced surgeons were more aware of safe versus unsafe operating conditions than were residents during review of a critical incident vignette (Dominguez, 2001). While most surgeons who did not detect a critical situational cue (anatomical landmark) made the decision to revise the surgical plan due to the increased risk of patient injury, 30% of residents who failed to detect the critical cue continued with the original plan. The inexperienced surgeons failed to recognize or discuss the increased risk of the situation and exhibited an inappropriately high comfort level with continuing with the original plan.

Two studies of anesthesiologists analyzed fixation errors (i.e., persistent attention on something other than the most critical problem) (DeAnda & Gaba, 1991; Xiao et al., 1995). DeAnda and Gaba (1991) found that experienced anesthesiologists made fewer overall errors, but of the same type, as anesthesiologist residents during simulated crisis events. In fact, the frequency of fixation errors did not improve with experience. A video analysis of anesthesiologists’ fixation errors during a trauma patient resuscitation concluded that these errors were partially due to the complexity of the work environment and to the strategies individuals used to deal with that complexity (Xiao et al., 1995). Key contributory factors included unreliable monitoring devices that prompted repeated redundancy checks and ongoing uncertainty about the patient’s often evolving status.

Social and contextual factors can exert subtle and diverse effects on decision-making and will interact with other factors that also affect decision-making (Bearman et al., 2009). Some contextual situations motivated pilots and fire incident commanders toward unsafe behaviors (what the authors termed “goal seduction” or where the objective sought exerted excessive pressure on their decision-making), while other situations motivated rejection of safer decisions (“goal aversion” or trying to avoid situations perceived as averse) (Bearman & Bremner, 2013; Bearman et al., 2009). In these studies, goal seduction often stemmed from urgency (e.g., search and rescue flights) or time pressure (e.g., the need to get to the destination or to a desired end outcome) (Bearman & Bremner, 2013; Bearman et al., 2009; Orasanu et al., 1993). Situation aversion stemmed from personal comfort or inconvenience (e.g., no overnight lodging in an area, or the desire to leave the incident as soon as the fire was extinguished). These factors may contribute to a tendency to persist with a plan despite evidence suggesting it should be revised.

Performance

In addition to the findings about decision-making errors described above, a few studies examined the impact of the other major themes we identified on decision-making performance, namely, decision-making strategy, time pressure, and stress. None of the reviewed studies measured how uncertainty affects decision-making performance.

How does the Decision-Making Strategy Affect Performance?

We did not find much evidence to specifically connect a particular strategy to performance outcomes. Two studies (Cohen-Hatton & Honey, 2015; Johnston et al., 1997) attempted to manipulate decision-making strategies through training. In a simulated naval task to classify radar targets (Johnston et al., 1997), participants were trained to use either an RPD or analytical strategy. Participants using an RPD strategy performed better (i.e., more accurate target identification) than those using an analytical strategy. In another study describing three experiments (virtual reality and in-person simulations) to measure the impact of goal-oriented training on fire incident commanders’ decision-making processes (Cohen-Hatton & Honey, 2015), some participants were trained to use a more analytical strategy. The trained participants exhibited more reflective processes (analytical plan formulation activities) without increased response time. However, this study did not assess if the training impacted decision effectiveness.

When assessing patient risk to determine whether to intervene during simulated patient deterioration, nurses tended to rely on intuitive decision-making strategies (Thompson et al., 2009). However, these same nurses underestimated the importance of key clinical information, and their resulting intervention decisions were largely inaccurate when compared with patients’ actual outcomes. However, the study could not conclude whether the intuitive decision-making strategy was the cause of the poor performance.

How does Time Pressure Affect Performance?

Two studies found a negative impact of time pressure on decision-making. Pilots completing scenarios in simulated cockpits were less accurate in diagnosing problems and deciding on appropriate responses when under time pressure, especially when informational cues were conflicting (Mosier et al., 2007). Since pilots who checked more information were more accurate overall, the authors attributed the poorer performance to reduced information search in response to time pressure, causing the pilots to miss relevant information cues.

Thompson et al. (2008) found that time pressure decreased the likelihood that nurses would intervene in low fidelity simulated scenarios of patients exhibiting potential signs of deterioration. As a result, there were slightly fewer false alarms but a significantly greater number of misses (Thompson et al., 2008). All nurses performed the decision task better without time pressure regardless of general years of clinical experience. While more experienced critical care nurses performed slightly better than general ward nurses without time pressure, under time pressure there was no difference in performance between these groups. However, a high-fidelity simulation study by Yang et al. (2014) produced contradictory results regarding the effect of time pressure on nurses’ ability to assess patient’s risk of deterioration: Decision-making accuracy was similar whether under time pressure or not (Yang et al., 2014). The authors suggested that the nurses may have perceived the time pressure intervention as “mild,” and thus had less impact on performance than expected.

How does Stress Affect Performance?

Only one study quantitatively measured the impact of stress on decision-making performance (Johnston et al., 1997). Johnston et al. trained naval personnel to use either an intuitive or an analytical decision-making strategy when performing simulated tasks to classify radar targets. High stress levels were added to some scenarios through a combination of auditory distraction, time pressure, and task load. Independent of the decision-making strategy, participants reported significantly greater stress and made fewer accurate decisions when performing tasks under the high-stress condition.

Discussion

The wide array of study foci and methods made synthesis of this evidence base challenging. Despite this, we found intuitive decision-making strategies consistent with RPD the most prevalent approach used by trained professionals during acute, dynamic situations. This is consistent with Kahneman’s characterization of system one and system two (Kahneman, 2011). We found little support for Hammond’s notion of a cognitive continuum suggesting that decision-making may not be simply intuitive or analytic, but in many cases may include a combination of intuition and analysis (Hammond, 1980). An important exception is Keampf et al.’s description of analysis of competing hypotheses within the RPD process (Kaempf et al., 1996). Time pressure, stress, and individual uncertainty may have important negative effects on a practitioner’s decision-making abilities in crisis situations if appropriate coping skills are lacking.

Perhaps the biggest takeaway from this effort is that substantial knowledge gaps persist in the literature. Table 4 highlights some of the specific gaps we uncovered through our review and suggests potential opportunities for future research design to fill in these gaps. As depicted in Table 4, there may be value in a taxonomy of decision strategies that facilitates communication across domains of study so that consumers of decision-making studies can better assess how relevant a specific study is to their own domains. A taxonomy with standardized definitions of decision strategies might lead to more understanding and discussion of boundary conditions for various decision strategies, how operators shift decision strategies as the demands of a situation change, and how prepriming might influence decision-making.

TABLE 4:

Research Gaps Identified.

Theme	Research Gap	Future Research Opportunity
Decision-making strategy	There is an inconsistency in how decision-making strategies are defined and a lack of clear descriptions of what was categorized as analytical versus RPD versus rule-based	• Development of a naturalistic decision-making taxonomy • Additional theoretical scholarship articulating the limits of RPD versus other decision-making strategies, including what aspects of decision-making can be generalized and what aspects are unique to specific types of work, as well as potential revisions to or expansion of the RPD model • Studies using cognitive engineering/psychology tools to delineate decision-making during or after actual or simulated events • Simulation-based studies that vary the information available “pre-event” to assess effects on in-event decision-making
	Boundary conditions of decision strategies are not well understood. Are some strategies maladaptive in some situations?
	The phenomenon of using mixed strategies (e.g., decision-makers flex between RPD and analytical) and what factors influence this are not well understood
	The role of rule-based decision-making in RPD is not well understood—is it a core part of RPD, or a distinct category?
	The role of pre-priming (i.e., initial assessments and mental simulation before actual engagement in the situation) in the decision process is not well understood
Time pressure	Strategies for assessing and coping with time pressure and their impact on decision-making performance during crisis events are not well understood. How to teach and sustain these skills?	• Studies of decision-making during or after actual or simulated events under different levels of time pressure, stress, or uncertainty (knowledge and outcomes) • Studies focused on understanding the role of time pressure, other stressors, and uncertainty • Studies focused on designing training to support stress management and evaluating training impact
Stress	Strategies for coping with stressors and their impact on performance during crisis events are not well understood. How to teach and sustain these skills?
Uncertainty	Strategies for managing uncertainty and their impact on performance during crisis events are not well understood—especially managing informational conflicts when diagnosing (e.g., to avoid premature closure). How to teach and sustain these skills? Can these skills be used to cue to focus cognitive resources effectively?
	The role of different types of uncertainty (information, event status, and environmental) on decision-making strategy and performance is not well understood
Performance	Whether specific decision-making strategies correlate with certain types of errors or performance failures is not well understood	• Studies of actual and simulated highly dynamic and high-risk events that focus on conditions that increase/decrease the likelihood of error and/or performance failures. May require large studies to explore the many contributing factors and contend with inter-scenario variability • Studies focused on how to best support different decision processes during high-risk events, including exploration of what skills should be trained and what decision support tools or organizational structures are needed in different contexts • Studies focused on understanding common errors, designing training to support error recognition and recovery, and evaluating training impact
	Whether training focused on error recognition and recovery supports performers in acquiring the adaptive, real-time, decision strategies needed for skilled performance in dynamic, high-risk events

Of particular interest is how to support operators in managing time pressure, stress, and uncertainty. One’s ability to cope with these demands is important for effective decision-making in crisis management and should be more tangibly incorporated into training and decision aids to improve performance outcomes. Perception of time criticality appears to impact decision-making strategies and performance. The ability to effectively assess and adapt to time pressure during critical or high-stakes events is a characteristic of expertise (Tole et al., 1982). Groombridge et al. (2019) conducted a systematic review of stress and decision-making during resuscitation that found exposure to stressors, such as patient illness severity, noise, time-pressure, and fatigue, negatively impacted decision-making. These stressors were found to be mitigated using a combination of resuscitation cognitive aids and teaching stress management (Groombridge et al., 2019). Consistent with Lowenstein’s (Loewenstein et al., 2001) concept of risk as feelings, studies have found nonproductive emotional responses (e.g., feeling the need to rush, feelings of urgency to think, decide, or act) and behavioral responses (e.g., fixation on a technical problem or reduced communication) contributed to decision-making difficulties under stress (Chrouser et al., 2018).

Therefore, it seems reasonable to deliberately train less experienced practitioners to assess situational time constraints and the degree of associated stress, for example, in realistic simulated acute scenarios. Indeed, there is a long tradition of stress inoculation in the military in which trainees are educated about stress response, coached regarding cognitive and behavioral strategies for managing time pressure and other stressors, and asked to perform in a range of stressful contexts (Robson & Manacapilli, 2014). For example, they may be asked to accomplish key tasks when fatigued, in the dark, while being shot at, with loud noises, or wearing bulky equipment. Although research on the effects of stress inoculation on performance is limited, there are some promising results. One study found that military personnel exposed to stress inoculation training performed better on a flight task in a virtual environment than those who did not (McClernon, 2009). Another found that participants exposed to stress inoculation training were able to hold their breath during cold water immersion longer than the control group (Barwood et al., 2006). Combining training with decision aids that reduce cognitive workload under stress may amplify positive effects.

The discussion of the effects of stress on decision-making begs the question of whether different stressors affect performance differently. Future studies exploring this issue should consider that in reality stressors, such as uncertainty, time pressure, and fatigue, often co-occur and their impact is likely mediated by experience. Rather than taking a decompositional approach, research to characterize the effects of stressors may benefit from a focus on adaptive skill as a component of expertise to be understood and trained. In fact, some argue that adaptive skill is a requirement for expertise, and therefore exposing study participants and learners to training scenarios that retain real-world complexities such as time pressure, uncertainty, and other stressors is critical to skill acquisition (Ward et al., 2018).

More research is needed to better understand how to best support decision-making in critical situations, especially the effectiveness of training on modifying decision-making approaches. The effectiveness of a decision-making strategy depends on many individuals, task, and contextual factors, which could be linked to performance to better understand decision-making in context (Kahneman & Klein, 2009). Studies that characterize stressors and complexities are likely to inform both our understanding of how skilled practitioners manage these demands and effective strategies for training adaptive performance. Error management training has shown encouraging results (Dyre et al., 2017; Franklin et al., 2021; Keith & Frese, 2008; Ziv et al., 2005). This approach could be expanded to develop training that focuses on presenting conditions likely to induce errors, including time pressure, stress, and uncertainty, so that learners have an opportunity to practice adapting decision strategies to cope with these demands. The RPD model implies that training that includes both recognizing critical cues and acting is likely to support skill acquisition. This notion of assessment-action pairing is believed to support development of the executive function needed to quickly size up a situation and act (Militello et al., 2023). Part-task training that separates activities that occur in parallel (such as assessment and action in time-pressured, dynamic tasks) may have detrimental effects (Wickens et al., 2013).

Although there was insufficient evidence to draw further conclusions about differences in decision-making strategies in acute or crisis situations across professional domains, we did note two nuances within the healthcare context. First, the surgical domain appears to rely more heavily on analytical strategies than other domains, even other healthcare specialties (Pauley et al., 2011, 2013). Cristancho et al. (2013) assert that the existence of a preoperative plan may be a particularly important component of decision-making in elective surgical cases. The presence of a patient-specific pre-op plan developed based on all available relevant information prior to initiating the surgery does shape the surgeon’s subsequent decision-making during the actual case. However, the degree to which this preplanning influences decision-making strategies when something unexpected occurs (or during emergencies when such plans are less well informed) remains unclear. This preoperative decision-making process may be analogous to an extended form of the advance cognitive work McLennan and Omodei (1996) identified in their study of firefighters prior to arriving on the scene, or the “mission rehearsal” activities commonly used in military settings in which team members mentally simulate a mission before it begins (Krebs et al., 1999). It is possible that a more formalized preparatory cognitive process may be useful if employed in other domains.

Second, physicians’ process of generating diagnostic hypotheses (e.g., differential diagnoses) is largely consistent with RPD—use of intuitive reasoning, pattern matching, and recognition based on past experiences (Farnan et al., 2008). However, the emergency physician studies reviewed (Hausmann et al., 2016; Pelaccia et al., 2014) suggest there may be some important aspects of this cognitive process that do not fit neatly within the RPD model. For example, although physicians often generate a leading diagnostic hypothesis rapidly based on their situation assessment, they typically keep other diagnoses as open options as they act while striving to confirm or disconfirm their leading diagnosis over time. This differential diagnosis process reflects different medical training paradigms. More research is needed to better understand how RPD overlaps with and differs from this ongoing differential diagnosis weighting and ranking activity.

In this literature review, we did not explore research investigating the limitations of specific decision strategies or how to avoid pitfalls associated with RPD, analytical, rule-based, and innovative strategies. Our focus was on the characterization of skilled decision-making in challenging domains; research focusing on poor performance and common pitfalls may provide additional insights.

Conclusion

More than 30 years since Klein introduced the RPD model for naturalistic decision-making (Klein, 1989), empirical evidence included in our systematic review continues to support its ongoing relevance for decision-making during critical events across a wide range of professional contexts. Our review of decision-making strategies used by skilled performers in responding to critical events did not identify clear differences between domains. Recognition-primed decision-making strategies were consistently reported across the 32 studies we reviewed regardless of domain. Analytical strategies also play an important role and appear to be more frequent in certain critical event tasks and contexts that generally have less time pressure, and in those in which generating multiple explanations is explicitly trained, with rule-based and creative strategies less frequently employed. Time pressure, stress, and uncertainty are major factors in decision-making during crisis events. These findings have important implications for training as skilled professionals must adapt decision processes depending on the time available, the level and types of uncertainty, their own experience and skill level, but also point to gaps requiring further research. A better understanding of the role of these factors can help us target evidence-based enhancements to practitioner training, and technology (e.g., decision support systems) and system design.

Biographies

Carrie Reale, MSN, RN-BC, is a board-certified nurse informaticist and human factors researcher with the Center for Research and Innovation in Systems Safety (CRISS) at Vanderbilt University Medical Center. Her research focus is on the application of human factors methods to improve the safety and usability of healthcare technology, including electronic health record user interfaces, clinical decision support tools, and medical devices.

Megan E. Salwei, PhD, is a Research Assistant Professor in the Department of Anesthesiology and the Department of Biomedical Informatics at Vanderbilt University Medical Center. She received her PhD in Industrial and Systems Engineering from the University of Wisconsin-Madison.

Laura Militello is a co-founder and CEO at Applied Decision Science, LLC, and Chief Scientist at Unveil, LLC. She is a leading authority on cognitive task analysis (CTA) methods—the tools and techniques for uncovering human cognition and decision-making behavior in complex settings. Her research includes applying cognitive engineering principles and naturalistic decision-making methods to design for medical informatics, medical devices, and military systems.

Matthew B. Weinger, MD, holds the Norman Ty Smith Chair in Patient Safety and Medical Simulation and is a Professor of Anesthesiology, Biomedical Informatics, Medical Education, and of Civil & Environmental Engineering at Vanderbilt University. He is the Director of the Center for Research and Innovation in Systems Safety (CRISS) at Vanderbilt University Medical Center.

Amanda Burden, MD, is a Professor of Anesthesiology, Vice Chair, Faculty Affairs, and Director of Clinical Skills and Simulation at Cooper Medical School of Rowan University and Cooper University Healthcare. Her areas of interest include patient safety, ending disparities in healthcare and education, and the use of simulation in healthcare education and assessment.

Christen Sushereba, MS, is a research associate at Applied Decision Science, LLC, where she aids in all aspects of research including data collection, analysis, and design of solutions. For the past 10 years, she has applied cognitive engineering and human factors methods to a variety of research domains, including military pararescue, emergency response, cybersecurity, electronic health record design, emergency medicine training, air traffic control, workload, and human-automation teaming.

Laurence C. Torsher, MD, is an Assistant Professor of Anesthesiology and Medical Education with the Department of Anesthesiology and Perioperative Medicine at Mayo Clinic in Rochester MN. His research is geared toward using simulation in education of medical students, residents, and practicing health care professionals.

Michael H. Andreae, MD, is a Professor of Anesthesiology at the University of Utah, focuses his use of immersive simulation on research in healthcare disparity and research ethics. Serving as the oral board examiner for the American Board of Anesthesiology, and senior editor for the Journal of Clinical Anesthesia, Dr. Andreae is most interested to leverage simulation to improve team performance and reflective practice in socially charged medical crisis.

David M. Gaba, MD, is a Staff Anesthesiologist and Founder and Co-Director of the Patient Simulation Center at VA Palo Alto Health Care System, as well as Associate Dean for Immersive & Simulation-based Learning and Professor of Anesthesiology, Perioperative & Pain Medicine at Stanford School of Medicine. His interests include human performance and patient safety, applying the organizational safety theory to healthcare, and simulation research.

William R. McIvor, MD, is a Professor of Anesthesiology at the University of Pittsburgh School of Medicine and the director of medical student education and simulation education for the department of anesthesiology. His interests include screen-based (computer, VR, and AR) simulation and using simulation to demonstrate competencies and milestone achievements. He has received several departmental and institutional awards for his teaching and is a Fellow of the American Society of Anesthesiologists (FASA).

Arna Banerjee, MD, is a Professor of Anesthesiology, Surgery, and Medical Education in the Department of Anesthesiology and Critical Care Medicine at Vanderbilt University Medical Center. She is the Assistant Dean for Simulation in Medical Education and directs the Center for Experiential Learning and Assessment at VUMC. She also serves as an Assistant Vice Chair for Faculty Development Programs in the Anesthesiology Faculty Affairs Office and is the Executive Medical Director for Critical Care (Adult) VUMC.

Jason Slagle, PhD, is a Research Associate Professor in the Center for Research and Innovation in Systems Safety (CRISS) at VUMC. Dr. Slagle applies human factors principles and methods to address issues in a wide variety of actual and simulated clinical domains. His research has focused on factors that affect the performance of clinicians (e.g., task distribution and workload, and coordination), unexpected clinical events, and the design of decision support tools and health information technology.

Shilo Anders, PhD, is a Research Associate Professor in the Department of Anesthesiology, Biomedical Informatics, and Computer Science at Vanderbilt University. Dr. Anders’ research interest is to apply human factors engineering as an approach to improve system design, individual and team performance, and patient safety and quality in healthcare in simulation and real-world environments.