Errors in clinical diagnosis: a narrative review

Abstract

Diagnostic errors are often caused by cognitive biases and sometimes by other cognitive errors, which are driven by factors specific to clinicians, patients, diseases, and health care systems. An experienced clinician diagnoses routine cases intuitively, effortlessly, and automatically through non-analytic reasoning and uses deliberate, cognitively effortful analytic reasoning to diagnose atypical or complicated clinical cases. However, diagnostic errors can never be completely avoided. To minimize the frequency of diagnostic errors, it is advisable to rely on multiple sources of information including the clinician’s personal experience, expert opinion, principals of statistics, evidence-based data, and well-designed algorithms and guidelines, if available. It is also important to frequently engage in thoughtful, reflective, and metacognitive practices that can serve to strengthen the clinician’s diagnostic skills, with a consequent reduction in the risk of diagnostic error. The purpose of this narrative review was to highlight certain factors that influence the genesis of diagnostic errors. Understanding the dynamic, adaptive, and complex interactions among these factors may assist clinicians, managers of health care systems, and public health policy makers in formulating strategies and guidelines aimed at reducing the incidence and prevalence of the phenomenon of clinical diagnostic error, which poses a public health hazard.

Introduction

A diagnosis can be viewed as a designation that distinctively characterizes a particular disease or condition in terms of etiopathogenesis, risk factors, treatment, and prognosis, and that is based on the affected patient’s signs and symptoms and on information obtained from the results of physical examination, diagnostic tests, and the health history. ¹ Formulation of a timely, accurate, and reliable diagnosis that can be properly explained to the patient is essential for the patient’s health care outcomes and well-being.^2–4 Data on diagnoses are essential for conducting meaningful clinical research and for developing and implementing efficient and effective public health policies.

The diagnostic process comprises a series of sequential steps that take place over time and involve communication with patients, consultation with a variety of health care professionals when necessary, and the use of appropriate tests, relevant technologies, and analytic and non-analytic reasoning; together, these enable the clinician to construct a diagnosis ⁵ (Figure 1). Patients with an uncertain diagnosis, particularly diagnoses for which the likelihood of being correct is low or unknown, should be promptly followed-up to identify possible diagnostic errors, thereby improving patient safety. ⁶

Figure 1.

Factors influencing the diagnostic process. Complex interactions among clinician-, patient-, disease-, system-, and environment-specific factors, some of which are characterized by elements of uncertainty, are key determinants of clinical decision-making. The greater the uncertainty, the greater the likelihood of generating risky or inappropriate clinical decisions, and this is complicated by dealing with difficult or demanding patients. Adapted from Croskerry (2018). ²⁶

The outcome of the diagnostic process is influenced by complex interactions between (1) clinician-specific factors, (2) patient-specific factors, (3) disease- or condition-specific features, and (4) the particular characteristics of the health care system in which the patient was diagnosed (Figure 2).^6–8 The nature of these complex interactions, which encompass many inherent elements of uncertainty and ambiguity, renders the diagnostic process susceptible to errors, inconsistencies, and overdiagnoses.^7,9 Therefore, taking into consideration the intricate, dynamic, adaptive, uncertain nature of the diagnostic process, the goal of achieving and sustaining zero diagnostic errors over time is unrealistic. ¹⁰

Figure 2.

The diagnostic process. The information obtained from the clinical history, physical examination, diagnostic tests, referrals, and consultations is analyzed and interpreted. This information is then used to formulate a working diagnosis through clinical judgment and reasoning. After the diagnosis has been finalized, it is explained to the patient and treatment options are discussed. The outcome of the treatment choice must be followed-up, and if needed, the initial diagnosis and treatment should be revised. ⁴ Clinical diagnosis is a dynamic, adaptive, multifactorial process. Diagnostic errors can be brought about by any faulty operative factor driving this process. To minimize the adverse health consequences of diagnostic errors, it is essential to follow-up patients in a timely manner, particularly those with an uncertain initial diagnosis, and promptly revise the diagnosis if needed.^6,15

The acquisition of clinical expertise is driven by ongoing domain-based practice, increased clinical experience, improved clinical knowledge and skills, and the development of domain-specific intuitive capacities. These allow clinicians to formulate accurate diagnoses via rapid non-analytic holistic associations. Intuitive cognitive processes are particularly useful for diagnosing complex or ill-structured clinical conditions with uncertain informational elements. ¹¹

To retain, maintain, and further develop a high-level of expertise, clinicians must be intrinsically motivated and prepared to engage in long hours of domain-specific clinical activities, immediate and important feedback practices in diagnostic performance, self-assessment and monitoring, and metacognitive activities.^11,12 These mechanisms of self-regulation may further improve diagnostic performance and consequently reduce the risk of diagnostic error.^10,13

Recent reports suggest that a sizeable number of clinicians occasionally commit diagnostic errors, which may be because they do not have the necessary psychological coping and resilience capacities to deal with related emotional and physical demands and the ambiguous and uncertain characteristic features of clinical practice.^12,14

There are insufficient evidence-based data about the incidence and rates of morbidity and mortality associated with diagnostic errors, with available information representing an educated estimation.^3–5 First, this is because there are presently no standardized and internationally accepted criteria for defining diagnostic error. Second, cases of diagnostic error are not always recognized as such; even if correctly identified, diagnostic errors are typically neither documented nor accessibly recorded in designated registers. Third, the existing information is mainly based on results of retrospective studies and not on prospective studies using consistent criteria.^5,7 For these reasons, there is a pressing need to improve disclosure and formal documentation of diagnostic errors with a view to learn from and prevent future events. ³

The purpose of this narrative review was to discuss certain factors that may play a role in the occurrence of diagnostic error and how situations of clinical uncertainty stemming from inadequate available information, limited professional knowledge, cognitive biases, and misinterpretation of patient-specific health needs, can impact the process of diagnosis. Better understanding of these complex interactions may assist clinicians, health care system managers, and public health policy makers in formulating strategies to reduce the incidence and prevalence of clinical diagnostic errors, which pose a public health hazard. The information for this narrative review was obtained in searches of MEDLINE and PubMed using the search terms judgment, decision-making, cognitive biases, diagnostic errors, medical uncertainties, analytical reasoning, and non-analytical reasoning. We also searched the references of articles that were deemed pertinent. Academic papers published in English language only were scrutinized in the writing of this narrative review.

The clinical reasoning process

The conceptualization of a diagnosis is a complex mental affair involving the operation of executive function-driven, high-order cognitive processes such as problem-solving, judgment, and decision-making. Use of both intuitive, rapid, automatic, effortless non-analytic as well as deliberate, attention-demanding, slower analytical cognitive systems is also required.^9,11,14–16 Intuitive cognitive pathways in the diagnostic process are susceptible to cognitive bias and other errors of reasoning, and when recognized by the clinician through reflective and metacognitive processes, these are corrected by deliberate analytic reasoning. In general, both intuitive and analytic reasoning are used concurrently during the diagnostic process.^9,11,15,16

Cognitive misconceptions that have a role in the generation of diagnostic errors may be brought about by inadequate knowledge and by failure of cognitive information processing elements such as data collection, data interpretation and comprehension, data integration, and data-driven compliance. Inadequate meta-cognitive monitoring and reflective activities may also play a part as well as poor situational awareness and/or inadequate expert clinical judgment and reasoning. Proper functioning of these cognitive determinants is essential in successful diagnostic processing.^15,17

Clinical judgment and decision-making deal with many uncertainties and ambiguities including patient-specific health factors, clinician-related factors, equivocal clinical data and laboratory test results, and the probabilistic nature of relevant evidence-based data. ⁹ Arriving at a definitive diagnosis requires clinicians to correlate and integrate information gathered from their own clinical experience; studying the current literature; attending professional meetings, conferences, and congresses; and professional discussions with colleagues. ⁹

Clinical expertise refers to clinical knowledge and wisdom assembled and integrated over time in memory through acquiring a high-level of understanding in treating an increasing number of patients with diverse clinical conditions.^9,11 An expert clinician is able to effectively focus attention on and evaluate details of the patient’s clinical problem to both intuitively and deliberately generate a number of relevant differential diagnoses and potential strategies to solve the identified clinical situation. If required, the expert clinician can successfully modify the diagnosis and the course of treatment on an ongoing basis, as new information becomes available. However, owing to the complexity of the diagnostic process, even an expert who is an authority in a particular clinical domain is not immune to committing diagnostic errors, albeit infrequently.^9,16 Compared with an expert, a novice clinician will likely make more diagnostic errors. ⁹

Formulating a particular diagnosis depends to a great extent on pattern recognition, the clinician’s familiarity with similar clinical scenarios, and the cognitive ease with which similar previously managed clinical cases are retrieved from memory and come to mind. Frequent conscious, critical reflection on daily clinical experience and on the cognitive processes that govern diagnostic procedures may boost the incorporation and retention of newly acquired pattern recognition and knowledge, thereby augmenting the mental bank of memory-anchored prototypes. These cognitive activities should improve the diagnostic capacity and skills of clinicians and reduce the risk for diagnostic errors.^{9,12,15,16,18,19}

In the context of the diagnostic process, the judgment and decision-making that novice clinicians make use of are usually deliberate, mentally taxing, and relatively slow rather than routine and nearly automatic. With clinical practice, however, a novice evolves into an experienced clinician who can intuitively and almost automatically formulate an accurate diagnosis and solve clinical problems more quickly, more frequently, and with less mental effort and focused attention. Nevertheless, complex, difficult, or atypical clinical cases require deliberate, focused, time-consuming analytic reasoning—even from clinical experts—with an increased risk of committing clinical errors.^7,9,16,17

Metacognition is a thinking strategy by which, through introspection, self-description, and analysis, one’s own thought operations are recognized and understood, thereby providing opportunities to moderate cognitive and affective processes. Mastering metacognitive skills enables clinicians to self-monitor their clinical reasoning, judgment, and decision-making in the diagnostic process, identify possible logical fallacies and cognitive biases, and to evaluate and constructively criticize their own diagnostic performance. Thus, if necessary, a diagnosis can be revised, reducing the risk of diagnostic error. ¹²

The term expertise, or expert judgment, refers to several qualities, including the ability to evolve and master new developments and knowledge within the clinical domain of daily practice, to respond successfully to changing clinical circumstances, and to efficiently and effectively coordinate analytic and non-analytic reasoning. These qualities will enable an expert clinician to explicitly reorganize the complexities of ill-defined, unusual, atypical, or clinically challenging cases. Diagnosing intricate, uncertain clinical cases requires the use of deliberate, analytic, and effortful cognitive processes and can be aided by decision support resources.^{12,15,20–22}

In the context of the diagnostic process, adaptive expertise refers to the capacity to demonstrate cognitive flexibility, and the flexible use of existing routine expert knowledge enables the formulation of innovative rules and principles for diagnosing difficult or atypical clinical cases.^23–25 It has been argued that the use of “adaptive expertise” by diagnosticians may reduce the rate of occurrence of diagnostic error. ²⁵

Negative affect (emotional stress, anxiety, loss of motivation, anger, fear, burnout, and other similar emotions) as well as physical ailments such as illness or fatigue, have the capacity to impair the cognitive functions of judgment and decision-making, with consequent effects on the quality of the diagnostic process and an increased risk of diagnostic error.^15,16,25 However, positive affect may support cognitive functioning, with consequent efficient and effective clinical judgment in decision-making and a reduced risk of diagnostic error.^9,16

Most clinical training programs in various health disciplines do not have designated, well-structured modules dealing with diagnostic reasoning, judgment, decision-making, and preventing diagnostic error. Diagnostic skills are primarily imparted by trainers in an apprenticeship system via tacit learning and are passed along in the domain of clinical practice.^16,26 This is likely because intuitive, non-analytic reasoning, judgment, and decision-making, which are imperative for an effective diagnostic process, are driven subconsciously and are acquired through experience and not through lecture hall teachings. Nevertheless, simulation exercises are useful tools to educate both undergraduates and qualified clinicians in the clinical reasoning, judgment, and decision-making pathways used during the diagnostic process, thereby improving their diagnostic skills.^7,26

Clinical diagnostic errors

Inappropriately delayed, wrong, or missed medical diagnosis, collectively termed diagnostic error, has a negative impact on treatment outcomes, patient well-being and the effectiveness, efficiency, and reputation of health care services. ²⁷ Diagnostic errors are a main cause of malpractice suits against health care systems.^10,17,19,28 It is believed that nearly all patients have either experienced or will be exposed in their lifetime to a diagnostic error, which may occur in any clinical health care setting.^2,5

It should be noted that classifying diagnostic errors into delayed, wrong, or missed is an oversimplification, first, because these different classes overlap considerably and it is therefore commonly difficult to distinguish between them; second, the concept of “inappropriately delayed” is undefined and vague; and lastly, as the process of reaching a final diagnosis evolves over time, there are stages in this process when a current working diagnosis will be provisional but not necessarily erroneous at the time. ²

It is estimated that the rate of occurrence of diagnostic errors that are associated with adverse outcomes ranges between 10% and 15% and that approximately 75% of these are related to cognitive errors committed by clinicians involved in the diagnostic process.^7,17,28,29 Despite this, the phenomenon of diagnostic error is underappreciated and has received little attention from public health authorities and academic teaching institutions.^3,4

Clinicians who are constantly exposed to work-related stressors, such as difficult or demanding patients, ethical dilemmas, pressing schedules, and excessive workloads, and who are required to make difficult professional judgments and operative decisions under time constraints, are particularly at risk of committing diagnostic errors.^7,14 In the context of the diagnostic process, time pressure refers to the psychological stressor caused by a real or perceived perception of not having sufficient time to formulate an accurate diagnosis, and this may increase the risk of cognitive errors and diagnostic errors.^12,30

Although most diagnostic errors are a result of complex interactions related to health care system factors and the clinician’s intrinsic cognitive factors,^6,17,31 it is believed that many diagnostic errors are caused by prevalent cognitive biases.^12,32 For example, there is an intuitive tendency to overestimate the risk of occurrence of unlikely events if they are overweighed; to search for and retain in memory new information that confirms prior beliefs and supports pre-existing concepts; and to terminate the decision-making process of a diagnosis (“premature closure”) before it has been fully completed. Other systematic errors in thinking that may affect the diagnostic process include constructing a diagnosis on the basis of unsubstantiated concepts, weak clinical and statistical evidence, and subjective first impressions; underestimating the role of chance; focusing on obvious and ignoring less-obvious evidence, focusing on minor non-essential details rather than on principal ones; and not considering plausible diagnostic alternatives.^9,12,32–34

One way to reduce the impact of clinician-specific cognitive bias is to use evidence-based information, established guidelines, and a probabilistic approach. ¹⁰ Although the available evidence-based data gathered from randomized studies and meta-analyses provide useful scientific guidance related to the diagnosis and management of a “randomized, average” patient, this information does not take into consideration many patient-specific characteristics such as risk factors and comorbidities, which are important elements influencing clinical judgment and decision-making. Thus, relying solely on the “generalized best” evidence-based information without taking into account the contingent and ambiguous inherent nature of clinical practice, results of observational studies, clinicians’ personal experience, expert clinical opinion, and the patient’s unique personal characteristics, may lead to diagnostic errors.^9,35

To avoid diagnostic errors, the clinician should have adequate domain-specific structural knowledge and clinical experience, possess the necessary mental tools to identify and counteract relevant biases and flaws in reasoning, and master both analytical and intuitive cognitive processes that support effective clinical judgment and decision-making. This should facilitate the formulation of an accurate definitive diagnosis, after considering and reviewing alternative options. ⁷

Interventions to reduce the burden of diagnostic errors

To reduce the frequency of diagnostic error, first and most importantly, clinicians and managers of health care systems must be cognizant of the prevalence of this phenomenon among all clinical domains and its adverse impact.^28,32,34 Such heightened awareness may increase the motivation of all stakeholders to implement preventive and corrective measures. ³²

Both clinicians and health care organizations should increase and improve the use of health information technology to enable access to digital images, up-to-date relevant information, expert and second opinions, and clinical guidelines and algorithms. Clinical decision support systems provide easy access to prior records and data (if they have already been captured in the system), thereby facilitating weighing of diagnostic probabilities, differential diagnosis, and diagnostic primacy.^{7,19,21,22,31} This will make the diagnostic process more efficient and effective and probably reduce the occurrence of diagnostic errors.

Interventions driven by the health care system with a view to reduce the likelihood of diagnostic errors include the following. First, establishing mechanisms to routinely detect the occurrence of diagnostic errors will enable and support reporting and learning from diagnostic errors by identifying failures in operational functioning.^22,31 Second, it is important to reduce clinical work overload, clerical responsibilities and administrative duties, and address other work-related stressors. ¹⁴ Third, performance evaluation systems should be introduced that recognize the values and importance of the quality of clinical judgment, clinical decision-making, and professional expertise, rather than merely focusing on criteria that dictate a preference for quantity over quality in health care services. Lastly, structural, managerial, and cultural modifications should be instituted in the workplace to support camaraderie, foster communication and teamwork, and promote autonomy, competence, relatedness, and professional development among clinicians.^14,19

Mindfully mediated reappraisal of clinical practice uncertainties, ambiguities, and other work-related stressors (time constraints, work overload, lack of managerial support) and of cognitive-affective maladaptive responses (e.g., cognitive biases, impatience, overconfidence, lack of confidence) may promote self-understanding of the mental processes that can lead to diagnostic errors, including emotional exhaustion, mental weariness, fatigue, and a lack of adequate knowledge or competence. Together with metacognitive practices, this process may facilitate the formulation of cognitive strategies to counteract the mental pressures that facilitate the generation of diagnostic errors.^7,16,19,32

The way forward

Because diagnostic errors have substantial adverse consequences on public health and safety, patients’ physical and mental well-being, and financial resources of the health care system, public health authorities must formulate strategies and launch initiatives to reduce this burden. ³¹ However, there are insufficient evidence-based data about the epidemiological, clinical, and laboratory features associated with diagnostic errors and those factors that influence their occurrence. ³¹ Therefore, certain relevant elements of cognitive science and critical thinking, as well as available basic knowledge about the phenomenon of diagnostic error, should be included in the curricula of undergraduate and postgraduate medical, dental, and nursing education. These should be featured as topics in continuing education courses and relevant health care-related conferences.^25,29,32 It is hoped that such educational measures will increase awareness, knowledge, and evidence-based research related to diagnostic error and promote collaborative measures to reduce its prevalence and incidence.^19,32,36

Further research is necessary to determine with confidence the incidence and prevalence of diagnostic error and its associated risk factors, which may improve clinical judgment and decision-making in relation to the diagnostic process. It is also important to determine the best ways to manage clinical uncertainties so as to prevent diagnostic errors. More research is also needed regarding the influence of different personality traits among clinicians related to committing diagnostic errors, the best training and educational methods for improving clinical reasoning, and the most effective strategies and interventions to achieve optimal clinical results and reduce patient harm owing to diagnostic errors.^31,32

Conclusion

There are no universally agreed and binding criteria for diagnostic error. Because the criteria used are often variable, vague, and clinically problematic to apply, it is difficult to measure and evaluate the occurrence of diagnostic error with any accuracy or consistency or to consequently determine the true epidemiological features of this phenomenon. Therefore, it might be prudent to recognize diagnostic errors as a distinct health-care related problem, which may promote consensus-building regarding defining the criteria for diagnostic error and may facilitate the recording and accumulation of evidence-based epidemiological data to enable planning and implementing interventional policies aimed at addressing this public health concern.