What contributes to diagnostic error or delay? A qualitative exploration across diverse acute care settings in the US

Abstract

Objectives:

Diagnostic error and delay is a prevalent and impactful problem. This study was part of a mixed methods approach to understand the organizational, clinician, and patient factors contributing to diagnostic error and delay among acutely ill patients within a health system, as well as recommendations for the development of tailored, targeted, feasible, and effective interventions.

Methods:

We did a multisite qualitative study using focus group methodology to explore the perspectives of key clinician stakeholders. We used a conceptual framework that characterized diagnostic error and delay as occurring within one of three stages of the patient’s diagnostic journey –critical information gathering, synthesis of key information, and decision making and communication. We developed our moderator guide based on the sociotechnical frameworks previously described by Holden and Singh for understanding non-cognitive factors that lead to diagnostic error and delay. De-identified focus group transcripts were coded in triplicate and to consensus over a series of meetings. A final coded dataset was then uploaded into NVivo software. The data was then analyzed to generate overarching themes and categories.

Results:

We recruited a total of 64 participants across 4 sites from emergency departments, hospital floor, and intensive care unit settings into 11 focus groups. Clinicians perceive that diverse organizational, communication and coordination, individual clinician, and patient factors interact to impede the process of making timely and accurate diagnoses.

Conclusions:

This study highlights the complex sociotechnical system within which individual clinicians operate and the contributions of systems, processes, and institutional factors to diagnostic error and delay.

Introduction

Diagnostic errors or delays (DEODs) remain an understudied threat to patient safety occurring in up to 20% of patient-clinician encounters.^1–6 A DEOD has been defined by the National Academy of Medicine (NAM) as a failure to establish an accurate explanation of the patient’s health problem or to communicate that explanation to the patient and within the health record.⁷ Singh classified DEODs as “wrong, missed, or delayed diagnoses” and more recently as “missed opportunities” to make a diagnosis.⁸ Much of our previous and even current understanding of DEOD is based on autopsy findings, malpractice litigation, surveys, and electronic medical record reviews.^9–18 The World Health Organization (WHO) and the Agency for Healthcare Research and Quality (AHRQ) have prioritized improved understanding of DEOD.² Recent calls to action have sought to improve the detection, use of big data, reporting, and effective prevention of DEOD.^19–22 However, few effective interventions exist to reduce and avert DEOD despite its negative outcomes.^23–25

Much of the research about DEOD conceptualizes it as a problem resulting from cognitive errors made by clinicians, focusing on the heuristics and cognitive biases that lead to errors.^26–31 However in recent decades there has been increased acknowledgement that in a complex medical environment, the interplay of multiple non-cognitive factors and systems issues are relevant and may determine the likelihood of a DEOD occurring.^9,32–34 The Systems Engineering Initiative for Patient Safety (SEIPS) provides a robust framework to examine DEODs by depicting the relevance of people interacting with elements of the “work system” and the importance of processes in complex adaptive systems.^35–37 The Safer Dx Framework, designed to tackle the issue of DEOD specifically, approaches the problem in a similar way but with the focus on the patient and his/her interactions with elements of the diagnostic process.¹

Qualitative research methods are ideally suited for the evaluation of complex problems such as DEOD as well as to facilitate the translation of research knowledge into practice.³⁸ Some qualitative studies examining DEOD in ambulatory care settings have been done.^39–42 Most of those studies were conducted in primary care settings and demonstrated that organizational and interactional factors (such as between patients and providers, providers and systems and patients and systems) contributed to DEOD as frequently as cognitive errors and communication problems. These studies like others based in an academic setting often focused on physician perspectives and did not seek the perceptions of other healthcare team members or clinicians.^39,40,43,44

Qualitative studies in diverse acute care hospital-based settings encompassing the ED, hospital floor, and intensive care unit and those involving multiple members of the healthcare team are lacking. Given the importance of interdisciplinary collaboration to facilitate accurate and timely diagnoses in acute care environments, we identified the need for a qualitative study of DEOD in acute care settings that sought the perspectives of a variety of clinician stakeholders from both academic and community hospitals.⁴⁵ Furthermore, we believed this was important to support the development of effective and sustainable interventions to prevent DEOD.

Methods

The Mayo Clinic Institutional Review Board approved this study, which we report in accordance with the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines.⁴⁶ Our core research team consisted of 3 diverse and experienced qualitative researchers: a medical sociologist (AK) and two physician researchers (AB, AL). This qualitative study is the first step in a mixed methods approach to understanding DEOD in acute care settings.

Participants and Recruitment

All data was collected from focus groups (FG) conducted between April and September 2019 in 4 hospitals in the Midwest and Southeast of the US. All hospitals belonged to the same large healthcare system, but varied in size, including 2 community-based and 2 academic tertiary care centers. After seeking the opinions of experts from the larger study team, we identified target stakeholder groups and used internal institutional contacts and site-based distribution lists to recruit participants using random sampling—via email—to discipline and site-based focus groups. The distribution lists included 880 individual email addresses. Our research coordinator, who did not take part in the data collection and analysis process, coordinated the entire recruitment process. We do not have information about how many emails were opened. All participants received $100 for their time.

Data Collection

AK facilitated all FGs in person, using a semi-structured moderator guide (Appendix 1), in conference rooms at the various hospitals. AB and/or AL also attended FGs, taking notes. The moderator guide was co-developed by our team and based on the SEIPS model and sociotechnical work system.³⁵ We also used a conceptual framework of the patient’s diagnostic journey based on the NAM diagnostic process outlined in “Improving Diagnosis in Healthcare”.⁷ As well as exploring the causes of DEOD we asked participants to make recommendations for reducing DEOD. Oral consent was obtained from all participants. Questions were intentionally open-ended to encourage participants to share opinions. All FG discussions lasted approximately one hour, were audio-recorded and transcribed verbatim before de-identification for analysis. We discontinued data collection once data saturation was achieved and no new themes were emerging during FGs.⁴⁷

Data Analysis

Our team independently read and open-coded an initial set of 3 randomly selected transcripts. We combined insights from this effort with field notes to generate a preliminary codebook. We (AK, AB, AL,) then used this codebook to code all transcripts independently and in triplicate. We met weekly to discuss coding, resolve disagreements and reach consensus on each transcript. We also considered and labeled emerging themes and discussed their relationships and organization, consistent with a general inductive approach.⁴⁸ We used NVivo software for analysis and data management. We discussed our findings with the larger study team as a means of triangulating our findings and ensuring that these findings accurately and completely represented the contributors to DEOD.

Results

Participant Demographics

We had 64 participants in 11 FGs. For each FG we tried to obtain 4 to 10 participants per group. Although we had initially planned for FGs of no less than 4 participants and always had a commitment from at least 5, for two FGs we only had 3 participants. Reasons for non-participation are unknown, but presumably relate to scheduling. Nine of the eleven FGs restricted participants to a single role at a single setting to avoid issues with power relations affecting participants’ comfort with engaging in the discussion. No one withdrew from the study. Details of the composition and location of all FGs groups are presented in Table 1.

Table 1:

Focus Group location/description and participant type and numbers

Focus Group Label	Location and Description	Participants(n)
FG 1	Tertiary Care 1	ICU Consultant Physicians (5)
FG 2	Tertiary Care 1	ICU NP/PAs (8)
FG 3	Tertiary Care 1	Floor/Hospitalist NP/PAs (5)
FG 4	Tertiary Care 1	Floor Bedside Nurses (6)
FG 5	Tertiary Care 1	ED Trainee Physicians/Residents (3)
FG 6	Tertiary Care 1	Transition and Transfer allied health staff* (9)
FG 7	Community Based 1	Charge Nurses ICU/Floor and RRT team (5)
FG 8	Community Based 1	Hospitalist Consultants (7)
FG 9	Tertiary Care 2	ED/ ICU Consultants (9)
FG 10	Tertiary Care 2	NP/PA mixed specialties (3)
FG 11	Community Based 2	Residents and Consultants Family Medicine (4)
Total participated/approached		64/880

^*focus group included interdisciplinary roles such as nurses and respiratory therapists

General Impressions of Diagnostic Error and Delay

Across all FGs, participants identified diverse but consistent contributors to DEOD. In addition to purely cognitive biases on the part of clinicians; organizational and system issues, challenges with interpersonal communication and the coordination of tasks, and individual characteristics and behaviors of patients and clinicians were all mentioned frequently. The contributors were often relevant across all stages of the diagnostic process, conceptualized as 3 stages: critical information gathering, information interpretation and synthesis, and decision making and communication.⁷ A summary of the themes we identified are presented in Tables 2, 3, 4 and 5 and represented in Figure 1.

Table 2:

Organizational Factors Sub-themes and illustrative Quotes

Infrastructure and Capacity	“So once you get so big, things just don’t work well anymore. It just starts to fall apart. What is that size? I don’t know, but I can see where stuff would actually get responded (to) and you would get your information and everything back if things could be a well-oiled machine, but I think the machine’s just not so well-oiled here in the hospital cause it’s too big.” (FG3)
Insufficient Time	“Sometimes on our floor, we can get like five, six patients, new admits like in an hour. And when the teams have that many new ones to focus on, sometimes it’s hard to get the tests ordered to get them done to figure out what’s going on, and that can be kind of a problem.” (FG6)
Labs	“[the patient had] her metabolic panel drawn at 2:25am and it didn’t get resulted until 7:30am in the morning, and it had a glucose of 22 or 20 or 22… I think we ended up getting a point of care [glucose] but by this time, it was 7:00 in the morning cause somebody did their 6:00 a.m. check on her and she was not responsive.” (FG2)
EMR and Technology	“And we’re a little limited to do what X is talking about, using [our EMR], reviewing past charting to look at that patient’s level of alertness to see if they had been somnolent or easily aroused. In the new system, you can click and see the last charted value, and you can also see the last values from 24 hours as well as the initial charted value. But if you wanted to trend that patient on something that’s not a measurable vital sign, there’s not a way within [our EMR] to create a trend for those categories. So …adventitious lung sounds [for]example, I would have to manually go through, column by column, of respiratory assessments to see, When did the coarseness start, and all of that. … So we’re a little limited from the system in doing our own quick analysis of what the trend is.” (FG2) “…and in [our previous EMR], it was a lot better- it was a lot easier to navigate. It was a lot easier to scroll through labs and look at something and trend something; in [our current EMR], not so much. So here she[colleague] is, her third day on actually recognizing that someone’s in renal failure, even though they had been in renal failure with the person before, and it’s just because of the labs system- the way [our current EMR], is organized. It [is] just a lot harder to recognize.” (FG 3)
Protocols and procedures	“I do wonder with a lot of those protocols, too, if they don’t cause a little bit of premature narrowing of our diagnosis. If we see someone who fits all the things that we know make them our sepsis criteria, I think sometimes people stop looking for what might actually be the underlying cause and they just say, Yeah, they’ve got a UTI and they need these things; they definitely just have sepsis, and then we find out later that they’ve got who knows what else that was really the cause of their symptoms or cause of their vital and laboratory abnormalities.” (FG11)
Multiple organizational factors	“I feel error is more likely to happen is when we have three admissions coming at the same time, and how are we supposed to do our best to manage all three patients at the same time if- especially if they’re all really sick. Then you kind of anchor to what the ED might have said, Oh this is sepsis, so you don’t have time to think, Is it anything else? You just have to treat it as sepsis until you have some more time to reassess and things.” (FG1)

Table 3:

Interactional Factors Sub-themes and Illustrative Quotes

Communication	“the most common thing I’ve experienced as far as there being an error or delay that has caused significant harm to patients is breakdown in communication …the example I think of is our lab error. So lab will get a result back that says, This patient’s hemoglobin was 2.5, but we didn’t believe it, so we didn’t tell you about it, and we re-ran the sample instead, and oh, … their hemoglobin’s really 2.5. That one, to me, feels like it’s not that uncommon that you get a call back and say, Oh we didn’t believe it. [and repeated it several times].” (FG3) “…communication between different specialties, I think that is also very crucial. If we don’t communicate appropriately, sometimes we miss things.” (FG9) “…occasionally, this happens, we see a report, and then the report gets changed without notice. And if you discharge patients without re-verifying the report, that’s disastrous.” (FG8)
Coordination within institution and outside	“A lot of communication has to happen to make sure every department’s ready to go. In order for certain tests—nuclear medicine, —Are they delayed because a certain medication wasn’t given? Are they delayed because the patient ate and wasn’t supposed to? Cardiac MRI—is there a physician on that actually read it or do it.” (FG7) “Yeah. I had an argument the other day on a patient, And it was like literally back and forth three or four times and then it was both of them [other consulting teams] that came together, ultimately…And they [other consulting teams] don’t wanna talk to each other. They want us to be the- as the medicine team, they want us to be the go-between, like the gopher or the- we feel like, kind of like glorified secretary middlemen sometimes and that can really delay things a lot.” (FG9) “I think timed studies, sometimes, become a problem. Sometimes, they’re done. Sometimes, they’re not. And I find I have a timed order that an hour and a half has gone by, and it still has not been drawn, and you have to call the nurse, or you have to call the lab, and that is kind of a nuisance. Sometimes, it is done and sometimes it’s not, just like it’s always like a mix-up 50/50 whether or not it’s actually gonna be done or not.” (FG9) “we get a lot of patients that come from outside and we have no records…that’s where there’s a lot of delay…what’s been done and what hasn’t been done.” (FG 9)
Handovers	“And like the times that I’ve gotten in trouble, [is]where the patient’s turned out to be sicker than that or has something different[than handover suggested]” (FG3) “I mean one of the things that we are talking about is that a lot of things do get missed when people hand over the service from one person to another.” (FG3) “I think with the handoff thing from a- from a nursing perspective, and probably also from a provider perspective, I’ve seen where one person’s kind of prejudiced towards the patient, not necessarily racial but like, This person is seeking, or this person is this or that, and then that piece gets handed from person to person [so] the true problem doesn’t get investigated” (FG10)
Differential Role and Power and Teamwork	“I know that there’s been a few times where there have been patients that just don’t look right, the floor nurses notice this, and they will let the general surgeon know like, This patient- like they’re becoming hypotensive, like their abdomen’s really firm, they don’t feel good- [the surgeon responds] They’re fine. Stop paging me. … And then it turns out that they have a perfed bowel and they need an ostomy and multiple surgeries and stuff like that. So it gets really frustrating when we- as nurses, we notice these things and the providers ignore us”(FG7) “I’ve often experienced, especially here, honestly, especially in this particular ICU where the attending or the consultant physician will only speak with the fellow, and that leaves you out of a lot of things, and then also that isolates you. It puts you in a corner…- it’s just a big circle to spiral down where you don’t feel like you can speak up or things like that.” (FG2)

Table 4:

Individual Clinician Factors Sub-themes and Illustrative Quotes

CLINICIAN
Implicit Bias	“I think that there is a lot of bias with the Muslim community even more so than some of the other religious communities .And so I think that from a provider’s standpoint, they may be less inclined to delve into some of those issues and to ask some of the questions that are pertinent because… they have that negative connotation of Muslims … they don’t ask as many things.” (FG10) “and the patient gets labeled [by prejudiced provider]as whatever, whether it’s like somebody with fibromyalgia who is seeking pain medicine or like sickle cell or something and has that true need but gets labeled a complainer or crazy or just looking for medicine, and so then nobody like really investigates what’s this underlying thing that’s actually bothering this person.” (FG10)
Fatigue/Exhaustion	“Like last night, I was just done at the end of my shift and really had to force myself to slow down and make sure I was looking at all my results and all my imaging to ensure that I wouldn’t miss something. But I can, very easily, see how things would slide under your radar when you’re tired…” (FG5)
Cognitive Bias –anchoring, general	“I think, with some of the diagnostic stuff, just getting the history from a patient can take you down a rabbit hole, it can be hard …to not have tunnel vision and just go along with what the patient says, and then each person kind of keeps going along with like- it went from the patient’s opinion to, this is the diagnosis, and these other things haven’t even been considered…” (FG10)
Training/Background/Experience	“But then you compare that to a nurse who’s six months in or fresh off of orientation who feels that they haven’t listened to enough normals to really identify the abnormal. And so are they crying wolf, themselves? Are they overreacting?… I’ve had nurses who have less than six months of experience on our unit say to me, Oh I’m not worried about that A-fib RVR cause it’s only 120. I’m not gonna bother the provider with that 3:00 a.m. Well, 120 RVR can quickly go to SVT.” (FG4) “We have significant concerns about the safety of our patients after midnight with a single resident reading. They’re slow and they make mistakes, and the over-reads coming in the morning and calling patients back.” (FG9)
Ego	“if your ego gets to where you’re trying to handle problems that really would be best handled by a specialist that, in general, is a possible area for diagnostic error or delay.” (FG11) “I would say sometimes ego can affect our provider even accept what you’re telling them. Sometimes, nurses are the ones sitting by the bedside, or probably worked with this patient maybe 4 days in a row for 12 hours, so you know your patient. But you’re trying to communicate with the doctors, and you’re saying, With their vital signs the way they’re doing, I think this is what is going on. But it just sound like[the provider thinks], You can’t tell me what to do, but you’re like, I think this patient is- has been here for five days- is constipated,… he’s been having nausea, and maybe we need to check something. But they’re[providers] like…they’re kind of pushing you backwards, so you can tell that it’s like ego and like, I know more than you do…” (FG4)
Work Ethic/Responsibility	“We have proceduralists that don’t show up on time. …for example, we’re gonna do… some type of neuroradiology procedure, like … check and see if somebody’s aneurysm could be clipped instead of coiled, and your radiologist doesn’t show up until 8:15. I mean the whole team has been sitting there. The patient’s been waiting since 7:45, and people just kind of stroll in when they feel like it…”(FG6) “One of our older doctors, like he sees about 34 patients a shift [in ED] compared to like the younger ones [doctors] that see maybe 12 to 14… he’s amazing. But if they[younger physicians] modeled after this guy [There would not be any waiting for a diagnosis in ED] Where the other ones[doctors], if- we feel like they sit on ‘em intentionally[and slow down diagnosis in ED” (FG6)
Multiple factors	“the provider’s historical perspective on how they’ve done it, So this has always been that way, or like you’re talking about the behavioral patterns of these patients and like they- they stereotype them, or the person who’s giving the information, how much do they trust that person who’s delivering it? What do they- how- what are their thoughts on, Oh this person has been here a while. I trust them. I’m just gonna listen, do what they wanna do because they’ve been right before. So it’s the perceptions of the patient from the provider’s point of view, the perceptions of the information deliverer, from the provider point of view, and then the pressures that they’re getting from outside sources. If it’s a resident, are they getting pressures from the consultant? Are they on the night shift and they’ve been in surgery for ten hours that day and they’re tired? Or are they- this is their tenth admit in the past seven hours, and they’ve been hit by a patient? So it’s the external variables…”(FG2)

Table 5:

Individual Patient Factors Sub-themes and Illustrative Quotes

PATIENT
Knowledge about his/her own condition/health literacy	“…it’s the way the physicians talk to the patient because sometimes they don’t understand that this patient [doesn’t] understand all our medical terms. So most of the time, you have to break it down …how bad it is or how not bad it is. So two years ago, they found a stone on my husband’s neck, a salivary gland, but it was simple, probably just small enough, and they didn’t even talk to him about the CT scan. They [said] We’ll send an ENT to touch base with you. But it’s a guy that doesn’t know anything about medical [things]. He’s an IT guy. An African-American-guy- they hate surgery. Surgery is like the last thing, except if [they’re] dying. That’s when you’re gonna operate . So they called him, and it’s like, Oh, well you have a referral for ENT. Can we see you? But at the back of his mind, he’s like, This is not a big deal. I’m not showing up. You’re just gonna wanna talk to me again.” (FG3)
Cognitive challenges that prohibit clear history	“….patients who are cognitively impaired. Those are very difficult to get a clear history and to pinpoint exactly what is going on. So that would be a patient that has dementia, for example. They may not be aware of it, but you can’t get much information from them. Like this patient’s not moving the arm; and if they can’t tell you how they were yesterday, then that can sometimes create a challenge in terms of diagnostic delays.” (FG9)
Narrative skills-good historian	“So, if the patient is a poor historian obviously, you will miss things, and it happens many times. On the fourth or fifth day, we find out that, Oh patient’s taking this medication, but three people- pharmacy asked, nursing asked, provider asked. The patient never admitted, I’m taking this medication. And suddenly, in the morning, patient said, Oh I forgot to tell you…so I think being a poor historian also kind of doesn’t help, I mean it is a cause of delays.” (FG8) “And I think, like we mentioned before, too, like - people will say various symptoms. They’ll say this symptom to this group and this symptom to this one. Or they won’t mention this because, Oh I didn’t think that was important. Well that really was a big important part of that diagnosis” (FG6)
Language	“But I think language is a huge deal. I think language barriers and care are a huge deal. I think even working through interpreters is just not the same, and I think that’s a big hindrance a lot of times [to accurate and timely diagnosis]. I don’t have a great fix for it other than directly speaking the language.” (FG1 )
Culture	“I think there’s some cultures that are very timid about telling you their symptoms. That’s always painful. … It just takes a lot longer. There are some cultures you walk in the room, and you get their whole life story in 30 seconds, and then there are others that you have to ask repeated questions to get to actually what you need to know [to make the diagnosis].” (FG9) “One provider explained: “I can tell a family is reserved because they are in fear of that bias …They’ve heard that, historically, they might get treated differently because of their race …so sometimes, it’ll take longer to develop that trust with them and that understanding” (FG10)
Medical complexity or atypical presentation	“The medical complexity of some of the patients that we see here…probably leads to some diagnostic error, delay, …and that’s just a product of how sick they are.” (FG5) “diagnostic reasoning isn’t necessarily a black and white process, right? I mean - not everybody drops into a clean bucket, and we spend a lot of time communicating back and forth diagnostic uncertainty and the things that we think [are] going on. I mean, frequently, when the patient comes in critically ill, we’ll empirically cover several different diagnoses until we let the dust settle and the patient differentiates themselves.” (FG1)
Family- interfering	“Difficult families, too, get in the way…especially when two or three of ‘em think different things” (FG11) “family sometimes can be a barrier, to getting things done. Again they’re bargaining or they’re questioning why we’re doing things. And then you have to go back and explain things…(causing a) delay” (FG3)
Behavioral issues	“I was thinking that like on my floor, personally, we have a lot of behavioral patients. And so a lot of times, they misinform the providers with things or nursing with things that maybe aren’t even true or just they are false or just different things.”(FG4) “I think they’re [clinicians] likely to fall back on a, Well they’ve done this. This is their behavior. This is what- This is their “norm,” and so I think a lot of things can like get missed in that circumstance.”(FG4)
Acuity of condition	“…one thing that would prevent a problem/diagnosis from getting identified is a multitude of problems and the hierarchy of the problems. And where day two, this is considered a low-level problem, but it was never addressed. So now on day four, it leapfrogged a couple other ones, so it’s now a higher problem. Like if we didn’t address hypovolemia on day two, and we just let it slide, and we just said, Oh the patient’s gotta drink cause their gut’s working; but on day four, well now, they got put into A-fib. So that smaller problem exacerbated and propelled itself to a larger problem…So I guess it would be the acuity of the problem list can interfere with addressing a diagnosis.” (FG8)

Figure 1:

Framework of Contributors to Diagnostic Error and Delay as they relate to the Diagnostic Process

Organizational and System Factors (Table 2)

Frequently referenced organizational and system contributors to DEOD included the infrastructural capacity of the organization (including the availability of diagnostic tests, equipment, services, allied staff and technician personnel) and associated demands on clinicians and their time. One ED trainee noted:

“I think the things that contribute the most to delay and error … are not individual characteristics of the practitioner, but it’s the environment within which we’re working” (FG5).

An advanced practice provider reflecting on her large workload said:

“I think our lack of recognition [of DEOD] … I would say 90% of the time, is because we have too many patients” (FG3).

The usability and usefulness of the electronic medical record (EMR) and electronic environment for accessing and tracking data to detect clinical deterioration or progress were often cited as problematic. Participants described the “deluge of data” and “duplication of data” (FG8) and the need for “dumpster diving” (FG1) to find data as hindering the diagnostic process. When discussing the technology and tools in the institution, an ICU physician remarked:

“So we’re a little limited from the (EMR) system in doing our own quick analysis of what the trend is” (FG1).

Included in the definition of diagnostic delay are lapses in communication with patients about their diagnosis. Several participants commented that this was a concern, specifically that they were so busy that they were unable to deliver diagnostic updates to patients.

“I do feel like there’s a delay in the ability to communicate your plan with a patient. I was like a roadrunner last night” (FG3).

The reliability and efficiency of laboratory and radiology services were also challenging. Deficiencies in these areas were especially salient causes of diagnostic delay during the information gathering and interpretation phases. When it came to acting on diagnostic decisions; cumbersome processes, protocols, and occasional bureaucracies sometimes stood in the way. Protocols, while sometimes useful, could introduce cognitive burden if they did not work smoothly, leading to errors. When standardized across diverse settings they frequently did not fit with the real world, local needs, work processes, and procedures thus causing frustration. One provider stated:

“the order set is almost entirely useless” (FG3).

Interpersonal and Interactional Factors (Table 3)

Broadly speaking, interpersonal, and interactional factors served as the sociological and personnel-mediated counterpart to the more tangible built environment and infrastructural factors of the organization and its systems. Our data demonstrated how these factors served to negatively influence the communication, coordination, and completion of diagnosis-related tasks. During the information gathering phase, for example, poor handoffs causing “further degradation of the information” would result in missing critical information.

Getting accurate information from outside facilities was especially challenging. One physician, noting difficulties with patients transferring from nursing homes and outside facilities, stated:

“We’ve missed hip fractures and stuff like that because people are sent over for shortness of breath and nobody mentions that they’ve [fallen]. So that’s where we have a lot of misses, actually, when we get half the story” (FG9).

Communication problems also occurred within the same institution. A provider commented:

“I think we all know that handoff, whether it’s between shifts- so whether it’s providers, whether it’s from OR to ICU …or between services… is always at high risk for loss of information” (FG2).

During the synthesis and interpretation phase, participants also reported examples of inadequate teamwork and power differentials between roles that inhibited the consideration of all perspectives and the inclusion of all information. Participants noted that although healthcare was a “team sport” some healthcare team members felt “left out” and team leaders did not have a “receptive attitude” that was open to suggestions from all team members. These comments were particularly prominent among bedside nurses and advanced practice providers.

Interpersonal and interactional factors were also important to the administration of diagnostic decisions, including the communication of these decisions to care team members and the coordination of diagnostic tasks among diverse services. Lack of familiarity with team members contributed to communication obstacles. Lack of feedback about updated or modified diagnostic results was also a concern among all clinicians and reflected complexity related to systems as well as people.

Individual Clinician Characteristics and Behaviors (Table 4)

Cognitive biases such as anchoring, recall bias, and even implicit bias are typically associated with individuals and cited as a cause of DEOD. One physician related:

“Sometimes we’re influenced by the case that recently happened or that had a big impact on us for some reason” (FG11).

A hospitalist determined that DEOD was:

“…anchoring to one particular thing, one particular diagnosis and then not able to switch gears and think differently” (FG8).

However we also identified other individual clinician traits and factors that were implicated in DEOD, characteristics such as personal and professional background, training, knowledge, and experience. Among healthcare team leaders such as physicians, ego was also cited by non-leaders as a potential issue. Fatigue related to long shifts and large caseloads were also reported to negatively impact accurate and timely diagnosis. One clinician said:

“I forget to order a.m. labs because I’m overwhelmed with my workload, and I think that does play a part and can impact the care we provide because when you have that many really sick patients, it’s hard to juggle, and you’re doing 20 different things at one time” (FG3).

Individual factors were most influential during information synthesis and interpretation. Work ethic and having a sense of responsibility for a patient’s welfare were also mentioned as important variables for DEOD.A hospitalist noted:

“if we don’t own the patient, then there’s more chance that errors could happen more” (FG8).

Individual Patient Characteristics and Behaviors (Table 5)

Patient factors were most influential during the information gathering phase of diagnosis. Patient factors included low health literacy that impeded knowledge of health condition. A consultant physician remarked:

“Sometimes, they don’t tell you everything they could, maybe intentionally or maybe unintentionally” (FG11).

Cognitive challenges were also a contributing factor in preventing clinicians from constructing a clear historical narrative. An ICU provider said:

“…and then you find out two days later, they have really bad dementia, and you just believed everything they said, when really it was all untrue”(FG2).

Patient physical attributes and behavior such as being untruthful also contributed to DEOD. One charge nurse explained: “The biggest thing I can think of is we’ve had patients who are like too large to fit into like scanners here” (FG7).

Language barriers even if language services were available were cited as influential. Cultural differences and trust issues that inhibited communication could also lead to DEOD unintentionally. Additionally; medical complexity, medical acuity, and having a rare diagnosis or atypical presentation could also lead to delays. As one provider articulated: “human don’t always present like a textbook” (FG2).

Inter-contributor Relationships

On several occasions multiple factors were identified as converging and exacerbating the risk of diagnostic error and delay. For example, individual background experience, and lack of time causing anchoring to semi-established diagnoses, ultimately leading to DEOD. (Tables 3 and 4)

See supplemental data for recommendations and quotes.

Discussion

We engaged a variety of key clinical stakeholders from diverse hospitals and acute care settings such as the ED, hospital floor, and ICU to identify factors contributing to DEOD. Broadly, these encompass organizational, interactional, and interpersonal factors, as well as characteristics of clinicians and patients. The factors that contribute to DEOD interact in complex ways and span the diagnostic process across stages of information gathering, interpretation and synthesis, and decision making and communication. When asked to propose strategies to addressing DEOD, clinical stakeholders recommend addressing organizational and interactional factors within the sociotechnical system preferentially.

Sometimes participants explicitly attributed DEOD to single specific contributors, but frequently the synergy of a variety of factors seemed to be more salient. Other studies have also found this.^49–51 This suggests several parallel approaches may be needed to measurably and effectively reduce rates of DEOD. Importantly, although diagnostic error and delay are often bundled together as one problem, contributing factors may differ between what is likely to cause and error and what is likely to cause a delay. Our participants perceived that errors were more frequently secondary to individual factors whereas delays were more likely to be secondary to systemic and coordination issues.

Qualitative work in other settings among physicians in primary care and hospital medicine also found that the contributions of interactional and organizational factors to DEOD exceeded those secondary to cognitive factors.^43,44 Some contributors to DEOD noted by participants have been highlighted in previous studies but in different settings. These included, challenges with the EMR, lack of time, experience, knowledge, fatigue of clinicians, as well as communication, and coordination issues.^{28,30–33,50,52–55} Getting timely and updated laboratory and radiology results was more common than mis-labelling or test inaccuracies.^56,57 We found the EMR was a recurrent source of vexation among all stakeholders in all settings. The presentation of data should be improved, how orders are entered, and the development of automated DEOD algorithms be developed to assist clinicians.^{58,59,60–62}When protocols are developed they need to match the processes and procedures of the hospital setting so standardization across an enterprise or health system may not be helpful.⁶³ The issues of suboptimal communication, benefits of good teamwork, clinician workload and lack of time affect all aspects of healthcare and their role in DEOD was cited by many participants in this study too.^64,65

However, participants also perceived other factors not previously described were also associated with DEOD. These included clinician ego, hierarchy in the clinical team and lack of ownership of patient’s needs. A qualitative study by Thomas et al did highlight that barriers to diagnosis included sociocultural and institutional norms that suggested that making a diagnosis was the purview of physicians, however a well-functioning interdisciplinary team can facilitate a timely and accurate diagnosis and should be leveraged for this purpose.^66,67

Patient complexity and atypical presentation of disease are well-recognized contributors to DEOD but other patient factors such as behavioral, cognitive, health literacy and trust issues were perceived by our participants as being important.^68,69 These factors will be harder to address but increasing diversity among the clinician workforce may offset concerns about potential bias and trust. Eliminating institutional barriers to access and maximizing use of language services should mitigate some of the issues we noted.⁷⁰ Piccardi’s study in a primary care setting highlighted that vulnerable social groups are more likely to experience DEOD as our data suggest.⁷¹ How much Checklists can help to support diagnosis and attenuate DEOD secondary to cognitive errors such as premature closure is not clear.^72,73 Other solutions suggested in the literature include decision support systems, improved communication, feedback, and even self-reporting of errors, earlier specialty referrals, diagnostic management teams, and diagnostic time-outs, although among acutely ill patients these may be more challenging to implement.^42,74–81 These overlap with some of the recommendations made by our participants.

This work is unique as it seeks the perspectives of multiple types of clinicians and in different types of hospitals across multiple acute care settings. Furthermore, this study describes contributors to DEOD from the macro (organizational), meso (interactional and interpersonal), and micro (individual) levels—24 unique contributors in all. Key strengths of the study include its grounding in a guiding framework, , and use of best practices in qualitative research to explore and richly describe a complex topic.⁴⁶ We incorporated multiple levels of triangulation with our approach by holding focus groups in geographically and structurally different hospitals and acute care settings, and recruiting a wide variety of clinicians and stakeholders. Furthermore our core research team brought different expertise, backgrounds, and experience to our study and sought regular input from the clinicians on the broader study group. Our study also has several limitations. First, all focus groups were conducted within one healthcare system, so our insights may not be generalizable to different institutional cultures, and infrastructures. Furthermore we focused on DEOD in acute care settings so some factors we identified may not apply in non-acute settings Additionally, a small subset of invited clinicians agreed to participate in this study and email recruitment may risk selection bias. These individuals and their perspectives may not be representative. However, we attempted to mitigate this bias using multiple triangulations strategies-investigator, data and environmental triangulation to enhance scientific rigor and validity.^82,83

Although occasionally participants did mention over-diagnosis as an issue, this was not a prominent concern. However we recognize that over-testing and the ordering of unnecessary investigations can also do harm and as with any medical decision risks and benefits must be carefully contemplated.⁸⁴Participants suggested some general recommendations that make intuitive sense and have been noted before. However mechanisms to deploy them into practice remain unclear. Due to what we perceived as the limited recommendations data we did not include this in our main manuscript results.. The work we conducted did not include any robust assessment of cost factors in the challenge of eradicating DEOD. As with current attempts to improve diagnosis by screening, cost analyses deserve serious consideration during solution design, development, and implementation.^85,86

Based on our findings, future efforts to address DEOD in acute care settings should focus on the entire sociotechnical system in which care is delivered. Better measures of DEOD are needed to design practice and research efforts that can evaluate the effects of interventions which target the contributors we describe. Related to this, future quantitative research approaches will be beneficial in helping to identify the factors that matter most and that should be targeted and this will be the focus of the next phase of our mixed methods approach. Our data suggests that the interaction and combination of factors are far more likely to contribute to DEOD than any individual contributor, making the task of tackling DEOD challenging. We must also consider approaches to including patients in our understanding of DEOD.⁸⁷

Conclusion

DEOD is a pervasive and important problem. Addressing DEOD requires understanding of its contributors and how these relate to each other. We did qualitative research (as the first step in a mixed methods effort) to advance this understanding. Although many factors contribute, we found error to be primarily a cognitive problem of clinicians. Conversely, we found delay to be a mostly coordination problem of multidisciplinary teams. Both error and delay are influenced by organizational, interactional, and individual factors; but these factors are more salient for problems related to delay. Weighting the relative and separate contributions of both error and delay to negative outcomes will be important in guiding decisions about the preventive interventions that should be prioritized.