Heuristics in Managing Complex Clinical Decision Tasks in Experts’ Decision Making

Abstract

Background

Clinical decision support is a tool to help experts make optimal and efficient decisions. However, little is known about the high level of abstractions in the thinking process for the experts.

Objective

The objective of the study is to understand how clinicians manage complexity while dealing with complex clinical decision tasks.

Method

After approval from the Institutional Review Board (IRB), three clinical experts were interviewed the transcripts from these interviews were analyzed.

Results

We found five broad categories of strategies by experts for managing complex clinical decision tasks: decision conflict, mental projection, decision trade-offs, managing uncertainty and generating rule of thumb.

Conclusion

Complexity is created by decision conflicts, mental projection, limited options and treatment uncertainty. Experts cope with complexity in a variety of ways, including using efficient and fast decision strategies to simplify complex decision tasks, mentally simulating outcomes and focusing on only the most relevant information.

Application

Understanding complex decision making processes can help design allocation based on the complexity of task for clinical decision support design.

I. Introduction

Clinical medicine has been greatly improved by advances in science and information technology (IT). The use of health IT in the medical domain has especially transformed the culture as clinical practice moved from paper to electronic use of health information. This transformation helped to shape the use of electronic health record (EHR) systems and the design of clinical decision support systems (CDS) [1]. However, clinicians have minimally adopted CDS systems [2]. The failure to design systems that fully match workflow and the processes of human cognition is one reason for this slow adoption and low user satisfaction with EHRs and CDSs. Experts have complex cognitive knowledge structures that have taken many years to develop, resulting in the ability to reason at high levels of abstraction. Decision support that can provide high-level, context-specific and efficient recommendations at the point of care will improve adoption by clinicians as well as more fully harness the benefits of information technology for the improvement of care. With an understanding of how to manage complexity, decision support designers will be able to provide system-level, big picture perspectives that can mitigate tunnel vision and support informed decision making for clinical experts. Therefore, it is imperative that basic decision support researchers develop a broader-based and valid foundation for the study of decision making as it occurs in natural settings [3]. However, to understand the complex clinical decision tasks it is important to gain knowledge about the clinical domain as well as the medical decision making process from a heuristics standpoint. In this study, the experts represent three distinct clinical domains.

Medical decision making has been the subject of research since the pioneering work by Egon Brunswik on social judgment theory [4]. Recently, dual process models of memory have explained cognitive processes in the medical domain by differentiating system 1 (automatic and rapid processes of an associative network) from system 2 (slower, more deliberate reasoning and mental simulation) [5, 6]. Metacognitive processes integrate the different modes of learning and memory [7]. The clinician uses system 2 to generalize processes by which a correct medical decision has been made. Through meta-cognition, the clinician forms robust pattern recognition or mental simulation in such a way so that he or she can make accurate medical decisions efficiently and accurately [8]. Metacognition forces the system 1 cognition state to be analyzed by system 2. This analysis is driven by the experiences of the clinicians where they have prebuilt heuristics or a mental model from previous similar cases. Understanding these heuristics can help us to comprehend the complex reasoning process from the experts’ perspective.

Humans use heuristics that allow them to process information with less effortful manner for reaching better decisions in an efficient way [9]. Simple heuristics can be more accurate for complex problems and is one of the major discoveries of the last decades [10]. Thus, simple heuristics achieve this accuracy by successfully exploiting the evolved mental abilities and environmental structures [11]. Heuristics improve decisions because they are derived from ecological rationality and not from classical definitions of rationality in terms of coherence or internal consistency of choices. Ecological rationality stems from the fact that we achieve intelligence in this world by using simple heuristics in the proper context. It conforms to the fact that our intelligent adaptive behavior emerges from the interaction between the mind and the world. In this study, we explore some of the heuristics or short cut mental models that clinicians use for reaching a decision point in complex clinical cases. Despite the advancements in the CDS domain, researchers argue that health IT systems, especially CDS, are designed without considering these aspects that underlie clinical decision tasks such as medical decisions [12–15]. Previous research has been conducted to investigate the factors associated with case complexity. However, little is known about the complexity of specific clinical decision tasks [16]. By understanding the factors that contribute to the complexity of decision tasks, researchers will have an improved understanding of clinician’s coping strategies when faced with complex clinical decisions. The overall purpose of this study is to understand different strategies or heuristics that clinical experts adopt for making complex clinical decisions. We expect that better understanding of these strategies and heuristics is critical for guiding the design of advanced CDS systems. This present report presents preliminary results of a small pilot study conducted to test and fine-tune the study procedures and data analysis.

II. Methods

A. Overview of Design

An in-depth qualitative approach was used. The Institutional Review Board of University of Utah approved the study and all participants waived to be consented.

B. Participants

Three volunteer clinical experts were interviewed. Each of the experts had more than 5 years of experience in their fields after general residency. The participants included one infectious disease expert, one geriatric medicine expert and one palliative care expert. The experts were recruited randomly and contacted by email from both at the Salt Lake City Veteran’s Administration Salt Medical Center as well as the University of Utah hospitals. All the participants were male and active in practice.

C. Procedure

At the start of the interview, the investigator asked the participants to recall a recent complex case. They were asked to recall a recent case that they perceived to be complex by nature and remembered in detail. The clinical experts were asked only to describe the cases that they found challenging in terms of diagnostic or treatment uncertainty. Interviews were conducted at the university or participants’ offices and took one hour. After each interview, the investigator gathered demographic information about each expert.

D. Critical Decision Method

The critical decision method (CDM) was used to conduct the interviews [17]. In a CDM interview, the researcher elicits important information about how participants view the task, their goals, expectations and the decision making processes. The goal is to uncover sense-making processes for a specific challenging incident method in the form of story-telling.

Interviews were audio-recorded and transcribed. All personal and identifiable information regarding patients, participants and colleagues was removed from the transcript.

E. Analysis

The analysis was iterative, involving the three investigators who analyzed the data in multiple cycles until they reached consensus. First, the three investigators initially independently coded each transcript (Table II) [18] for decision points, complexity factors, decision cues, goals and action based on the critical decision method [17]. The categories were reconciled through group consensus. Each category was then discussed among the reviewers for identifying themes and meaning.

TABLE II.

Criteria used to code the transcripts

Criteria	Description
Decision Points	The point of making a decision while evaluating multiple options and choices
Complexity Factors	Factors that contribute to making the case complex from the decision making perspective
Decision Cues	All types of implicit and verbal information including signs, symptoms and context that contributed to making a decision
Goals	The participants described goals for the patient at each decision point
Action	Any kind of actions taken including treatment, looking at lab results, consultation with another expert, etc.

III. RESULTS

The following five key categories of coping strategies with complex decision tasks were identified during data analysis.

A. Decision Conflict

Expert clinicians face decision conflicts during most of the complex case resolution. Experts try to find patterns in patients’ cases amongst this complexity [19]. Conflicts also arise when the clinicians deal with socio-technical problems with patients’ family. Moreover, disagreement with other physicians about patient management can lead to decision conflicts. Experts try to find patterns in patients’ cases. However, when the patient is very unique due to the diagnosis or patients’ demographics, experts face decision conflict due to the lack of established guidelines. The decision conflict causes anxiety and frustration. For example:

“After awhile I kind of looked back at the pattern of infections and it seemed like she was getting admitted about the same amount, two or three times a year, and was developing resistance to the antibiotics that were being used for suppression. And she’s supposed to follow up in my clinic soon and then I’ll have to decide whether or not to continue suppression or just take her back off of the prophylaxis and see what happens. “

“Well, so the first thought is, Do you use prophylactic suppression or not prophylactic suppression? And so the thought process would be maybe that you would or would not. If you use suppression you might have fewer infections but they might be harder to treat. So there’s potentially this conflict between possibly having fewer infections. There isn’t a literature that says one way or the other about that.”

“Unfortunately, a complicating factor was we had a niece show up with DPOA papers, who said, Take my mom off the ventilator right now because she’s a DNR! We had a little bit of discussion…and this kind of sidetracked the whole care for a little while.”

B. Mental Projection

Experts project themselves into other practitioners’ points of view and try to mentally simulate what course of action or treatment of option other clinicians would consider. Experts are worried that other physicians might not consider treatment options that they are considering or be able to infer their line of reasoning. As a result, they work through possible ways that other clinicians could view the treatment plan. For example:

“…..So there’s an antibiotic that’s commonly used in the Emergency Room that you really don’t use in older patients with poor renal clearance. And so it does limit somewhat the choices of antibiotics that are possible to use and it also kind of increases the chance that someone would use an inappropriate antibiotic for an older person because they’re not really aware of the consequences. They’re not as aware of the medicines that we don’t use for older people to treat infections usually.”

“They’re focused on keeping the patient alive at all costs and don’t recognize that some of the things that they do can sort of destroy the endgame. So you can make somebody so sick that you can’t do a bone marrow transplant or they can be so sick that you should do a bone marrow transplant to this patient? It’s highly debatable. Having multiple uncontrolled fungal infections going in to transplant is never a good idea, especially if you’ve failed the first induction. It’s a bad prognostic indicator.”

“But we assumed that they would have been the case anyway because the Cipro was already only intermediately sensitive. So it probably didn’t change anyone’s management but it may have influenced them to think, Oh, now we need a different drug for suppression because the Cipro isn’t working anymore for suppression. I think they’re more likely to follow the pattern that’s been out there before. So, I think they were probably more likely to say, Well, she was being suppressed with cipro before and now we’ve lost cipro. We have to suppress with something else.”

C. Decision Trade-offs

Experts have to constantly adjudicate both long-term and short-term goals, mentally simulating the possible interactions and impact of different options. As a result, they are thinking of choosing the treatment option that resolves the immediate issue while at the same time trying to ensure the best outcome in the long term. We found that decision trade-offs included judgments regarding life expectancy versus quality of life or curative effects versus side-effect profiles. Some of the tradeoffs involve value judgments. In this process, the experts might have to let go certain options for the long-term benefit for the patient. For example:

“And all of that with the possibility of missing it or getting antagonistic activity. In that particular case, at least for a while until he stabilized somewhat, we felt that the risk of missing was high enough to justify the risk that it was antagonistic. So we’d rather hit with one of them than completely miss, even if the activity was lower. So in this particular case we gave both amphotericin and voriconazole, although I’ve never done it since.”

“And so there’s a timing piece to this too because she’s in her nineties. I mean, if they were in their seventies and you said, Oh, well…you’re going to have recurrent urinary tract infections for 15 or 20 years. But in their case the timeline you’re probably looking at is a couple of years, three or four years.”

“We’d actually put her on BiPAP just to buy some time, hoping that the antibiotics would kick in, hoping that something would come up.”

D. Managing Uncertainty

Experts must manage high levels of uncertainty when dealing with very complex problems. We analyzed significant reports of lack of information, including lack of options, evidence, patient’s past medical data, clinical guidelines, and even laboratory results. Moreover, conflicts among medical teams about patient management, uncertain pathogen data as well as a lack of information and knowledge about a diagnosis can cause a high level of stress among experts. For example:

“I mean, he had no platelets, he had no white blood cells. So again there’s nothing written in the book about what to do so we picked sort of the old submarine under the polar icecap option. So we tried to cool it off with the antibiotics and drain it and hope that we could come back to surgery later. We made what I tend to think was the wrong decision at the time.”

“ It may have been the first case of that particular fungus but weird things do occur. As far as giving broad-spectrum antifungal therapy, you give broad-spectrum therapy and hope for the best. You can call the foremost fungal expert in the world and there’s only so much that we know.”

“And his thought was it’s probably just pneumonia, it may just be resistant, we’ll just put her on these big guns and this should take care of it. We’ll know 24 hours after the antibiotics get infused, we’ll know better. So when I was giving her three days I actually thought I was being more conservative. I figured 24 to 48 hours and we’ll have this turned around.”

E. Rule of Thumb or Heuristics

Experts in clinical practice have significant time constraints, including multiple interruptions and simultaneous demands, resulting in limited cognitive attention reserves. We found from data analysis that they often construct a short cut mental model of a complex case in order to spare attention resources. For example:

“I think in terms of how to treat a patient. One of the thoughts I had was it sort of made me use the rule of thumb. I guess, is that you might think about using an antibiotic where there’s an intermediate level of resistance for suppression. So that was kind of the rule I use. That’s what I learned and that was sort of the teaching point to me.”

“If I were to look at it in terms of rules, I guess the first one is there any guiding information? The answer is no. The next question really is can you afford to completely miss with an agent? I guess maybe that works. Can you afford to completely miss with the use of one antifungal? And the answer was no at the time, at least until he had gotten stabilized some.”

“Don’t get lulled in by appearances. The patient came in with what looked like what was going to be a pretty straightforward pneumonia. The antibiotics should have worked, that we did initially. I mean, we were on the protocol and she didn’t respond like we expected. So the important thing we did jump on it early. I did get a pulmonary consult early and when we looked back we hadn’t missed anything or hadn’t delayed. But I think, that was one of the learning cases to me early in my career. Things are not always what they seem. So there was complexity there but I didn’t realize it until I was already into it.”

IV. DISCUSSION

In this pilot study, cognitive task analysis tools identified specific decision-making features that clinical experts might apply to manage decision task complexity. The decision task complexity that experts described here can be referred to as subjective task complexity. Subjective task complexity is a property of task and task performer characteristics supported by researchers in the information-seeking domain [20–22]. The task performer will perceive the task as complex if the task outruns the mental capacity of the task performer [23]. Our exploratory study was able to find the specific decision-making strategies that the experts’ adopt to deal with cases that are complex.

Expert clinicians are in demand. This demand associated with the complexity in the clinical workplace requires extensive mental effort on the part of the expert to find, aggregate, and process clinical information in a timely manner from a variety of sources [3]. Health IT and CDS could play an important role to support the cognitive functions of clinicians. However, CDS must provide the cognitive support at high levels of reasoning and integration of multiple sources. The decision support tools can support several functions including minimizing cognitive demand [24], supporting reasoning and decision making [25] and offloading tasks [26]. Decision support designers should leverage from our preliminary findings of experts’ decision-making process. There are several key cognitive areas that our exploratory work suggests deeper analysis.

A. Assistance with Decision Conflict

Experts are facing decision conflicts by getting overwhelmed with the data that is not helpful for them. They have to integrate complex information from different sources, sifting and sorting through irrelevant information, making judgments about the quality of the information as well as content. Decision support tools should help to reduce the decision conflict by highlighting the relevant tasks. Then, clinicians could more rapidly target the specific source of conflict and the associated needed information [27]. For example, if the decision conflict arises due to less clear evidence, then an analytic approach of showing all available information in a graphical format with probabilities can help clinicians by reducing conflict. Future research on the relationship among decision complexity, task identification and information seeking behavior can help experts to focus on the specific attributes that they perceive to be complex and conflicting.

B. Supporting Mental Projections and Simulation

Drug resistance and the changing dynamics of the pathogen pose a great threat for the patient due to limited treatment options [28]. Treatment patterns are changing and vary across settings. As a result, clinicians must try to imagine what pattern resistance takes, how the disease might mutate and what other physicians would do in terms of treating the patient in the future.

Mental projection means a theory of mind that is an instance of a “common mechanism” for representing actions by the self and others. There are two ways to perceive what other practitioners think. One is the “imagine-other” perspective that involves an empathic imagination of other clinicians’ situation and an understanding of their desires and needs. The other one is the “imagine-self” perspective. Its role is in evoking empathic concern indirectly while doubting the other clinician with impatience and judgment. This perspective taking ability needs to be recognized by the medical community. Clinicians might fail to take the first step toward understanding one another’s points of view if they are too heavily influenced by their own egocentric point of view[29].

Our research shows that experts use the “imagine-self” perspective of the theory of mind when they are estimating the abilities and skills of other clinicians. Thus, they use their perception as an anchor while projecting other practitioners’ actions through mental simulation.

C. Supporting Decision Trade-offs

Experts are looking into for certain cues that allow them to process information in a less effortful manner. This selection of cues actually corresponds to the “fast and frugal” heuristic model, which employs minimum time, knowledge and computation to make adaptive choices in a real environment. Fast and frugal heuristics limit their search for information using easily pre-determined stopping rules and making choices with easily computable decision rules [30]. For example, in our study the decision trade-offs of using antibiotics was largely focused on the factor of life expectancy and the age of the patient. However, the experts could have considered more than 10 different cues. The experts tend to give the most priority to one or two cues that make the most sense for them instead of weighing different cues. Therefore, decision support should be designed based on those certain cues in the context of the decision making process. However, more research can help us to understand which cues experts perceive to be important.

D. Support to Manage Uncertainty

Experts face uncertainty due to a lack of clear and concise information. Experts may be uncertain due to a missing laboratory test result, lack of diagnosis, knowledge, or even treatment options. Yet, decision support that can show similar patients in the same decision space of uncertainty can reduce that uncertainty. This population-based decision support can provide the experts a big picture of the patients who were treated and the outcome of these patients compared to the population. Population-based big data analytics can show intervention and outcomes in a graphical format that can help clinicians deal with the patient that they are treating. Future research on how population-based visual analytics or big data analytics can help in decision making process is a much needed aspect for reducing decision uncertainty in the clinical domain.

E. Support for Rules of Thumb or Heuristics

Experts examine fewer cues and store only pertinent information in their memory that they can use when faced with similar problems. This process can be explained by lexicographic heuristics that requires decision makers to decide which cue will be the most important and then select the alternative with the best value [31]. In the case of a tie for the most important cue, experts focus on the next most important cue until choosing a single alternative. The other variants of the lexicographic heuristics include lexicographic semi-order heuristics [32], priority heuristics [30], take the best heuristic [11] and the single variable heuristics [2]. Even when decision makers are searching through multiple cues, they consider one cue at a time and thus are able to reduce the amount of information that needs to be kept in the working memory. Decision support designers can benefit by understanding the cues that experts perceive to be important and use to make decisions based on these cues. Future research on which cues are ignored and how effort-reduction occurs in complex decision tasks can shed light on focusing decision support logic on those certain cues.

This exploratory study suggests key messages (Table III) that we should consider as well as the need to find more specific information for a better design for decision support. Current research on CDS design does not incorporate the high level of thinking process of clinicians [8, 33–35]. The preliminary findings from this pilot study suggest more studies with more participants are needed with expertise from different clinical domains. Thus, we will be able to find specific cues and decision points that can be incorporated into design task allocation for better decision support design and adaptation. Future studies using work domain analysis with cognitive task analysis can shed light on the expert’s decision-making process as well as the environmental factors that influences the process.

TABLE III.

KEY MESSAGES

Providing simulation tools for better decision support can integrate patient’s data with scientific evidence

Decision support that can provide similar patients’ outcomes and interventions by other providers can reduce decision uncertainty and decision conflicts

Information extraction from patients’ charts for customized holistic view of patients situations in terms of flexibilities of views can help with decision trade-offs and decision conflicts

V. LIMITATIONS

We acknowledge that our most important limitation is the small number of participants; therefore the results of our study are exploratory in nature and should be considered with caution. To address this limitation, a full study with a larger sample size is underway. Despite the small sample size, the themes that emerged seem to be consistent among the three experts and can serve as a foundation for our future work. The critical decision method (CDM) is based on case recall techniques that might be limited by recall bias. In terms of sense making, this approach does not accurately explain all possible options that the experts were considering as experts are quite likely unconsciously dealing with options. Also, since the first author conducted both data collections, there is the potential to influence the way the interview was conducted. To guard against this bias, we constructed the questionnaire based on the critical decision method instrument.

VI. CONCLUSION

In this pilot study, the preliminary results suggested that complexity is created by decision conflicts, mental projection, limited options and treatment uncertainty. Experts cope with complexity in a variety of ways, including using efficient and fast decision strategies to simplify complex decision tasks, mentally simulating outcomes and focusing on the most relevant information. As a result, experts look for certain discriminative cues. Incorporating these cues into the design of CDS can improve these systems, fostering better adoption and outcomes. Future research with more participants will provide a better understanding of experts’ coping strategies with complex decision tasks.

TABLE I.

PHASES IN CRITICAL DECISION METHOD

Phases	Description
Incident Identification	The participant defines the scope of the incident and is refined by the interviewer, if necessary. For example, in this study the details about patients’ secondary information and demographic information were noted but not probed in detail.
Timeline Verification	The investigator repeats a timeline of events to elicit accurate information about decisions information. This step is aimed at getting a clear, refined and verified overview of the incident structure, identifying key events and segments.
Deepening	Using the timeline, the interviewer probes into more details about decision points, judgments and the decision making process. This phase makes sure that the implicit information and cognitive processes are evoked.
“What If” Queries	In this final phase, the interviewer asks hypothetical questions to identify implicit cues relevant to understanding the decision making processes. For example, the interviewer asks, “What knowledge or tool could have helped in a situation like this?”