Big-Data Based Decision-Support Systems to Improve Clinicians’ Cognition

Abstract

Complex clinical decision-making could be facilitated by using population health data to inform clinicians. In two previous studies, we interviewed 16 infectious disease experts to understand complex clinical reasoning. For this study, we focused on answers from the experts on how clinical reasoning can be supported by population-based Big-Data. We found cognitive strategies such as trajectory tracking, perspective taking, and metacognition has the potential to improve clinicians’ cognition to deal with complex problems. These cognitive strategies could be supported by population health data, and all have important implications for the design of Big-Data based decision-support tools that could be embedded in electronic health records. Our findings provide directions for task allocation and design of decision-support applications for health care industry development of Big data based decision-support systems.

I. INTRODUCTION

Recent estimates indicate that more than 75 million people in the United States have two or more concurrent chronic conditions, and as a result many patients do not fall under existing guidelines.¹ Therefore, therapeutic decisions are often made in situations of substantial uncertainty, potentially resulting in inappropriate testing and treatment.

Clinical decision-support tools embedded in electronic health record (EHR) systems hold the potential to support complex clinical reasoning, thus improving patient safety.² Encouraged by the HITECH act, the widespread adoption of EHRs has created the potential to leverage the electronic clinical data of entire populations to solve problems in individual patients.³ Thus, secondary use of EHR data to elicit practice-based information when evidence-based information does not exist is the inherent vision for the learning health system model.⁴ A decision-support system leveraging population data to show similar patients for dealing with treatment variations and uncertainty may provide cognitive support to clinicians.

Population decision-support could benefit clinical practice within the specialty of infectious diseases (ID) by allowing the tracking of disease burden, providing outbreak detection, and forecasting resource needs.⁵ To develop such decision-support tools, it is important to understand the complex clinical reasoning process of clinical experts. Such understanding will allow for more intuitive design and better task allocation within the interface. Prior work has shown the importance of understanding clinical reasoning within a medical domain when developing clinical decision-support tools, but less research has been conducted to understand the cognitive design of population decision-support in the ID domain.

In this study, we propose to fill this knowledge gap by examining how population decision support can help ID experts deal with complex clinical reasoning, specifically how it could augment the cognitive strategies ID experts use when performing complex decision tasks. Such understanding is critical to guide the design of advanced population decision-support systems embedded within EHR systems.

II. METHOD

A. Overivew of Design

In a previous study, we conducted semistructured interviews with 12 ID clinicians using cognitive task analysis (CTA) to examine the cognitive strategies that clinicians use to deal with complex tasks. We also conducted an observation study in which we interviewed 4 ID clinicians. We asked the participants specific questions as part of both studies regarding how access to population decision support to deal with complex decision tasks could help clinicians. Both studies were conducted at the Salt Lake City Veteran’s Administration Medical Center and University of Utah Hospital. The Institutional Review Board (IRB) of the University of Utah approved the study, and all participant provided verbal consent.

B. Procedure

For the cognitive task analysis study, the investigator (DR) asked the participant to describe a case that he or she remembered in detail and that was perceived to be very complex and challenging in terms of diagnosis uncertainty and treatment unpredictability. Diagnosis uncertainty and treatment unpredictability both increase cognitive complexity for the clinician. Then, specific probes were used to understand the value of Big data in healthcare to solve complex decision tasks. For example, we asked specific questions about different features, functionalities and analytics that can help clinicians using the population decision-support. We also asked how different information from population database can add value to reduce clinical uncertainty at point-of-care. These probes helped to get more detailed information about how Big-Data based decision-support systems can help clinicians to improve care. For the observation study, we asked clinicians same questions. The interviews were audio-recorded and transcribed. All names and other patient identifiers were removed from the transcript.

C. Data Analysis

Three researchers (DR, MJ and JC) with clinical backgrounds were involved in the data analysis process. We separated the data from the transcripts related to the research questions for this study. Then, the research team conducted the data analysis with well-accepted practices for content analysis of qualitative data.^{6, 7} The analysis was iterative, involving three researchers with clinical backgrounds and one researcher with a visual analytics background. The team members reviewed initial categories, merged similar codes, and reached agreement by consensus. This group approach of finding consensus through discussion is common in qualitative research; this approach is also rigorous and encourages rich conceptual analysis and interpretation.⁸ We used specific criteria of clinical reasoning such as decision points, decision cues, goals and features of Big-Data based decision support systems that can help clinical reasoning.

The set of codes that emerged from the initial analysis was systematically examined, and similar codes were merged based on code frequency and consensus. Theses codes were defined by group consensus and merged into different categories. Themes and coding categories were open to revision during group analysis and modified sequentially in subsequent sweeps. The final step of the data analysis involved the identification of relationships across major categories and yielded broad, interconnected themes. An integrated summary of major findings in terms of themes was then generated. Any major differences in opinion were resolved by group consensus. Categories not relevant to the research question were deleted with group consensus. The team used the Atlas ti software for data analysis.

III. RESULTS

The following three categories of themes emerged as cognitive strategies used by experts when dealing with complex cases in which population decision support could help.

A. Trajectory Tracking

Trajectory tracking is the macrocognitive process of imagining how unexpected events may affect practice and plans.⁹ Trajectory tracking is about getting ahead of the curve and projecting possible scenarios to be able to handle situations.¹⁰ In medicine, this process requires a functional perspective of how the disease will progress and what is expected based on what is observed. Clinical experts use this cognitive process of planning and replanning to prepare for unexpected events. This process provides a holistic overview of the patient’s situation. For example,

Well, I don’t know. If you ask me, his chance of healing with a local surgery, I would say that they’re poor. Antibiotics alone, I would have said that his chances are poor. What does poor mean? I don’t know. But they’re not good. He’s going to end up with a BKA or an AKA, I said. Or, he might end up needing surgery soon.

B. Perspective Taking

The experts used group conformity to reduce the social pressure associated with antibiotic prescribing, primarily by engaging in perspective taking or simulating other minds. In this process, they imagine other practitioners’ decision logic for a certain medical intervention or outcome. For example,

The way in which population-based information, or sort of identifying similar patient cases and treatment and management strategies and outcomes, would be helpful is if the management strategies were similar. If 9 times out of 10 a patient who’s chronically colonized comes in and he gets Meropenem from other providers, by inertia it feels a little bit like, shouldn’t I be with the 90% and treat this guy? Whereas, it would very much strengthen, as I described, this feeling of being on your own. If you had other cases where people held off you’d say that look people do this, it’s okay; it’s okay to wait.

C. Metacognition: Thinking About Thinking

Metacognition is a process of thinking about and controlling one’s own thinking.^{11, 12} Dual Process Theory (DPT) has associated clinical reasoning with System 1 (nonanalytical) and System 2 (analytical) cognitive processes.¹³ Metacognition represents another cognitive measure clinicians use to consciously regulate if a correct diagnosis is difficult to make. As a result, the System 2 analytical process helps a clinician assess the differential diagnosis until a clinical decision is made. Thus, experts use metacognition, which represents a high level of cognitive reasoning accumulated from years of practice and knowledge. Metacognition monitors the clinical reasoning for validation and rejection through its regulatory or controlling function. For example,

It’s a good question. I thought about it for some time. One thing, I think, is to make sure you’ve explored all the possibilities. For example, when a patient looks like this from the past up to the present, when a course looks like this, it may be that I’m blind, that I’m anchored, that I might be fixating on one or two, a set of possibilities from my experience but missing what it may be so I have no sense of the base rate instead of anchored.

IV. DISUCSSION

Our results support findings from previous studies on complex clinical reasoning.^{14 15 16 17} Most complex cases do not fall under simple clinical guidelines due to a lack of evidence related to the unique situation.¹⁸ Population decision support embedded in the EHR has the potential to aid decision-making by providing information about similar cases when guidelines or other evidence-based resources do not apply. In this study, the cognitive strategies represent expert ID clinicians’ subjective perception of how population decision support could aid in complex clinical reasoning. Designers and industry partners may use these cognitive strategies for design allocation. The cognitive strategies found in this research have similarities yet distinct in terms of their uniqueness. The projection into the future about the patient’s situation is a common thread between each strategies. However, each strategy denotes the clinician’s specific abilities to improve overall situational awareness. For example, trajectory tracking may be crucial for taking care of the patient’s needs that may arise in the future. On the other hand, perspective taking can help with improving care coordination. Also, metacognition helps with reducing self-cognitive biases. In the following paragraphs, we describe some useful ways to support these cognitive strategies for intuitive health care Big-Data based decision-support design.

A. Supporting Trajectory Tracking

Trajectory tracking helps clinical experts guard against or forestall future threats to the patient; it is a type of future-oriented sensemaking. Previous research on predicting future threats based on trajectory tracking also found this strategy useful for sensemaking purposes.¹⁹ One way to support this process is by providing visualizations of similar patients’ treatments and outcomes, which will allow for better sensemaking.²⁰ Such tools can simulate situations and alert clinicians to potential threats. For example, a simple graph progression chart of “Surgery now” versus “Medication management” based on similar patients with outcome simulation can facilitate clinical decision-making as well as prevent adverse events. Future research is needed to examine different visualization techniques to use population data to display the courses of similar patients in ways that support trajectory tracking.

B. Assisting With Perspective Taking

In the ID domain, there is significant pressure to reduce overprescribing of antibiotics due to the increasing resistance pattern of microbes.²¹ ID experts endeavor to cope with this pressure by looking for group conformity or peer consultations.²² However, due to the unique nature of medically complex patients, it may not be possible to find peers who have seen similar cases. Therefore, population decision support can provide the attending ID expert with matched similar cases. The ability to find an expert who has seen similar cases or a demonstration of grouped cohorts may make the differences among patients treated with different regimens more distinguishable, which may in turn help the expert deal with the perspective-taking cognitive strategy. Future research on visualization techniques to show the degree of similarity may help clinicians comprehend the similarities among patients and may also provide data validation.

C. Assisting Metacognition

Clinical experts reason through cognitive tasks, switching between System 1 (nonanalytical) and System 2 (analytical) processing.¹³ When a patient’s case is similar to cases previously seen by the clinician, System 1 becomes active. However, if certain cues are missed during the diagnosis process, then anchoring bias may occur. Anchoring bias, which refers to the tendency to fixate on the first impression, can cause a clinician to initially miss the correct diagnosis.²³ Therefore, the design of population decision support should include interventions that trigger System 2 (analytical thinking) in order to support metacognitive thinking. For example, using predictive modeling embedded into population decision support can trigger consideration of scenarios that clinicians might not have thought about. This approach has shown promise in the areas of predicting ICU mortality, cardiovascular risk, and neonatal sepsis.^24–27 The construction, validation, and evaluation of the prediction models embedded in population decision support is needed as such tools have the potential to assist clinicians by providing them a high-level view of possible scenarios and probabilities associated with a case, thereby reducing anchoring bias.²⁸ Future research on creating a better predictive model using population data may improve the overall sensitivity and specificity of the model for clinical use.

V. LIMITATIONS

The cognitive strategies found in this study represent the ID domain. The findings may not be directly applicable to the specific problems faced by other clinical specialties within medicine. However, as infection is a prevalent problem faced by most domains in healthcare, our results can be used for broader impact in most areas of medicine. Lastly, the first author conducted all the interviews and interviewer bias is possible. Therefore, we used a structured qualitative approach, critical decision method, to guard against this bias.

VI. CONCLUSION

In this study, we conducted cognitive task analysis to understand how population information can help with complex clinical reasoning in the ID domain. Our results suggest that experts use trajectory tracking, perspective taking, and metacognition to solve complex clinical problems. Population decision support can act as a cognitive extension for clinicians to assist in the cognitive strategies they cmploy to deal with complex clinical problems. Incorporating these cognitive strategies in the design of future population decision support can help decision-support designers in the industry with better task and design allocation.