Supporting novice clinicians cognitive strategies: System design perspective

Abstract

Infections occur among all clinical domains. The changing nature of microbes, viruses and infections poses a great threat to the overall well-being in medicine. Clinicians in the infectious disease (ID) domain deal with diagnostic as well as treatment uncertainty in their everyday practice. Our current health information technology (HIT) systems do not consider the level of clinician expertise into the system design process. Thus, information is presented to both novice and expert ID clinicians in identical ways. The purpose of this study was to identify the cognitive strategies novice ID clinicians use in managing complex cases to make better recommendations for system design. In the process, we interviewed 14 ID experts and asked them to give us a detailed description of how novice clinicians would have dealt with complex cases. From the interview transcripts, we identified four major themes that expert clinicians suggested about novices’ cognitive strategies including: A) dealing with uncertainty, B) lack of higher macrocognition, C) oversimplification of problems through heuristics and D) dealing with peer pressure. Current and future innovative decision support tools embedded in the electronic health record that can match these cognitive strategies may hold the key to cognitively supporting novice clinicians. The results of this study may open up avenues for future research and suggest design directions for better healthcare systems.

I. Introduction

About 15 million out of the 57 million (>25%) annual deaths worldwide are estimated to be associated with infectious diseases. Emerging diseases, microbial adaption, breakdown of public health measures, climate, weather change and international travel all make the domain challenging and increase diagnostic and treatment uncertainty for the infectious disease (ID) clinician [1, 2]. The current health information technology (HIT) systems, such as the electronic health record (EHR) and clinical decisions support systems (CDSS), have great potential to support clinicians and ultimately improve quality of care [3–5]. However, despite the wide adoption of HIT systems, frustration and errors exist due to the lack of user-centered design and cognitive support among other factors [4, 6].

The differences between expert and non-expert clinical skills and judgment have been studied extensively in the last 30 years [7–9]. It is evident that clinical experts possess superior judgment skills to solve complex clinical problems. Years of training, experience, skill, and fine-tuned intuition distinguish the expert clinician from the novice. However, the current system design in healthcare does not take into account the differences in cognitive abilities between expert and novice clinicians. As a result, information is presented the same way regardless of the clinician’s level of expertise. Consequently, neither expert nor novice clinicians are optimally supported when using the current systems. There is a need to understand the cognitive strategies, skills and knowledge requirements for clinicians at different levels of expertise.

Numerous studies have looked at the cognitive strategies that expert and novice clinicians possess in the ID domain. The work of Kruger and Dunning suggested that those with the least skill may be the most at risk of inaccurately assessing their abilities [10]. Therefore, direct interviews with novices are unlikely to yield valid insight into their cognitive strategies with complex clinical reasoning. However, clinical experts spend significant time with their trainees and can appreciate the strengths and weaknesses of novices. It is plausible that the impressions of clinical experts could yield valid information. In this study, we interviewed clinical experts to elicit their perceptions about how the novice solves complex problems and how current and future technology decision support tools may help. We postulate that by understanding the novices’ cognitive abilities from the expert’s point of view, better decision support tools could be embedded in our health IT systems.

The objective of this study was to understand cognitive strategies used by novice clinicians to solve complex clinical problems in the ID domain for better health information technology design.

II. Methods

A. Settings

We conducted semi-structured, in-depth, qualitative interviews. The interviews were conducted at the University of Utah and the Salt Lake City VA Medical Center and were approved by the Institutional Review Board.

B. Participants

We interviewed 14 infectious diseases experts, as defined by at least 5 years of recent clinical experience after their formal training. The clinical experts were recruited by email and participation was voluntary.

C. Procedure

We conducted the interview process based on the critical decision method (CDM) [11]. The CDM interview consists of four phases: incident identification, timeline verification, deepening and what-if queries. In incident identification, the goal is to understand the context of the problem. Timeline verification provides an overview of the detailed incident structure. In deepening, the interviewer asks probing questions to understand the pivotal information cues, decision points and goals. The “what-if” queries help to simulate certain situations to elicit more in-depth responses.

We first asked ID experts to recall a recent complex case they remembered in detail. Then, we asked them to describe how a novice clinician would try to solve the case as well as the factors that would make the case challenging for the novice clinician. In this study, we defined novice clinicians as residents or fellows in training who were under the supervision of attending ID clinicians. Finally, we probed about types of decision support tools that can help novice clinicians with clinical decision-making. The interviews were audio-recorded and transcribed. All patient data were removed from the transcript before data analysis.

D. Data Analysis

The analysis was iterative and based on group consensus. Three researchers with clinical backgrounds met on several occasions for coding and recoding the transcript. First, they coded the transcript independently and then met for group consensus. The criteria used for coding are described in Table I. Codes were modified, deleted and merged based on group consensus. Eventually, the codes were merged into categories. Finally, the researchers discussed and identified broader themes and meanings from the categories. We used the Atlas Ti 7.0 software for the qualitative data analysis [12].

TABLE I.

CRITERIA USED FOR CODING TRANSCRIPTS

Criteria	Descriptions
Decision points	The valuable piece of information that leads to successful decision-making
Goals	The participants’ description of how novice clinicians set goals and prioritize
Complexity- contributing factors	The different challenges and obstacles novice clinicians face with complex cases and scenarios
Decision cues	The different pieces of information that novice clinicians focus on while making their decisions

III. Results

The data analysis revealed four key themes that expert ID clinicians perceived are cognitive strategies characteristic of novice clinicians in situations involving complex clinical reasoning: A) dealing with uncertainty, B) lack of higher macrocognition, C) oversimplification of problems through heuristics and D) dealing with peer pressure.

A. Dealing with uncertainty

Dealing with uncertainty in the ID domain is a common phenomenon due to emerging infections, viruses and ever-changing pathogens. It is even more challenging for less experienced clinicians when a patient’s response to treatment or the diagnosis is difficult to predict. For example,

“I guess maybe they will try to understand that you just can’t treat this with three days, five days, one week, that it needs a longer course. Maybe that’s where we put our foot down and say, ‘No, no, this can’t be anything less than six weeks and we have to think about suppression if we can in terms of it.’ But, this guy had so many other complications that it was very hard to project the total therapy timeline.”

B. Lack of higher macrocognition

Macrocognition refers to the ability to perform complex cognitive work in novel situations and engage in knowledge-based performance [13]. The key macrocognitive functions include sense making, planning, adaptation, problem detection, and coordination [8]. Novice clinicians tend not to connect a clinical situation with the high-level complete clinical picture. This high-level picture refers to understanding the patient’s situation from a broader context. For example,

“So my resident was thinking more along the lines of TB, fungal, you know, other chronic infections but he didn’t think about Brucella or Coxiella because he didn’t actually ask him about animal exposure while he was down in a different country. So if you don’t ask the exposure questions you’re not going to get it.”

C. Oversimplification of problems through heuristics

Heuristics refers to the short-cut mental model for processing information cues when making decisions [14]. Heuristics help cognition by conserving attention. However, in this process, important information can get ignored resulting in different types of cognitive biases. As a result, clinicians risk making errors. Novice clinicians tend to follow heuristics more often and thus, oversimplify complex problems.

“I always tell them, generate your differential diagnosis before ordering antibiotics. And then prioritize based on two things. Likelihood of that diagnosis and gravity of that diagnosis. So, testing and treatment are a combination of likelihood and then you weigh it for the gravity of missing that diagnosis. But, most of the times they end up simplifying the problem by ordering empirical therapy and wait for the lab. They tend to take the easy route most of the time.”

D. Dealing with peer pressure

Novice clinicians deal with the stress of performing in front of their peers, supervisors and patients. As new clinicians, they experience the peer pressure to perform well of dealing with the complexity of medicine. Dealing with the peers and creating a good image is important for the novice clinicians during the initial phase of their career.

“The other big problem is that there’s not a calibration of disease with intervention. So, there’s a tendency, then, to, over treat. And it’s because they’re afraid. What is interesting is that it’s not even liability, it’s not even getting sued so much. It is that they feel it’s safer to do things to their patients so that they won’t be held responsible if something goes wrong.”

IV. Discussion

In this study, we elicited the cognitive strategies that novice clinicians use to deal with complex cases from the interviews of expert clinicians. Our results agree with the findings from the cognitive sciences literature concerning naturalistic decision-making, heuristics, uncertainty management and affective computing [15–19]. The cognitive strategies used by novice clinicians may be supported by some currently available tools, as discussed in the following paragraphs:

A. Supporting uncertainty

It is a key challenge to manage uncertainty. In medicine, uncertainty may arise due to information overload, failure to understand patient trajectory, or in the event of high cognitive complexity [20],[21]. Visual analytics combined with advanced techniques in data mining and interactive tools may provide users with valuable information that supports complex reasoning and highlights patterns drawn from large databases. For example, the green button initiative within EHR systems can help clinicians leverage information about similar patients at the point of care [22]. A prominent problem in medicine is complex and unique patients who can present with symptoms that do not match clinical guidelines that increase decision uncertainty for novice clinicians. In those situations, it is difficult for novice clinicians to make the appropriate decisions. Therefore, data from secondary sources (e.g., clinical database, EHR systems) providing previous decision and treatment outcomes may reduce some of the confusion and improve confidence at the point of care. Future research may evaluate different ways to represent uncertainty (graphically versus linguistically) and show the aggregated patient data from secondary sources for reducing uncertainty. For example, showing information as a plot graph from a timeline perspective may be better than just showing numerical values.

B. Assisting with macrocognition

For the novice clinician, the complex case and novel situation place an excessive load on their cognitive abilities. Also, the limited cognitive abilities to deal with information cues from many different resources may seem to be increased uncertainty. However, planning and setting up goals to reduce task loads may help to deal with uncertainty by using macrocognition or high level of reasoning. As a result, they tend to focus on mental simulation to help assess the hypothesis, action or plan under consideration [23]. In this process, cognitive simulation helps novice clinicians recognize additional information needed and factor in cues not previously considered.

To aid this process, simulation tools supporting “what-if” analysis using a machine-learning algorithm embedded in the EHR or CDSS might help. Currently, there are risk prediction models that have useful applications. For example, understanding and identifying high-risk patients can help with hospital readmissions and reduce overall healthcare expenses [24]. However, though they assess the group risk accurately, these models frequently fail to adequately predict clinical outcomes [25]. Prediction models can perform the what-if analysis that may help to increase the confidence of clinicians in their decisions. More research is needed in this area to make these models aligned with patient care and to validate them in real clinical settings.

C. Supporting decision heuristics

Clinicians operate under constraints of limited time, information, resources and cognitive attention. Models of decision-making are helpful in engineering decisions but not necessarily in medical decisions. For example, physicians typically reject logistic regression models to make a diagnosis, yet they accept fast and frugal tree based reasoning whose sequential structure matches their mental model and intuitive thinking [26]. The goal of fast and frugal heuristics is not to satisfy logical requirements of consistency or to find a mathematical solution to an optimization problem, but to execute successful decisions as measured by criteria such as speed, frugality, accuracy and robustness [18, 27]. More research is needed to understand the utility of decision heuristics that can support novice clinicians based on their mental model to make the fastest and most efficient decisions.

D. Reducing peer pressure

Anxiety to perform, liability and status among peers can increase the overall psychological stress for clinicians. Particularly in the ID domain, avoiding the overuse of antibiotics by itself adds a lot of stress for novice clinicians. Inappropriate prescribing of antibiotics has been associated with mounting rates of antibiotic resistance worldwide and thus, is a major concern in all domains of medicine [28–31]. The notion that doing less in medicine sometimes can mean more has been an important discussion for the ID community. Future research is needed to align the field of affective computing with medicine. The goal of affective computing is to develop computational systems that can recognize and respond to the affective or emotional states of the user [32]. Affective computing may help to detect stressed emotional states for team members to initiate support from collaborators. For example, wearable technologies can sense heart rate and perspiration for understanding if someone is stressed or not [33]. Creating future decision-support tools where data from the wearable technologies can be used to alert the team leader about the stressed team members may help to support the emotional states of team members.

V. LIMITATIONS

We have conducted a data analysis on the cognitive strategies of novice clinicians based on the opinions of expert clinicians that may have introduced recall bias. Never the less, this methodology provides an independent and self-bias free evaluation from the experts about the novices’ ability to deal with complex cases in the ID domain. One author conducted all the interviews and may have introduced bias. For this reason, we have conducted all the interviews using the CDM procedure. There may be generalizability issues, as the experts were all from the ID domain and from the same geographical location.

VI. Conclusion

Current health IT system design does not consider the different levels of clinician expertise when designing systems. Information is presented to both expert and non-expert clinicians in the same way. Based on interviews with expert clinicians, we elicited the cognitive strategies that novice clinicians use in the ID domain to deal with complex cases. These strategies may guide future decision support researchers and inform EHR designers in developing tools that can support clinical cognition to improve patient safety.