Burden of Serious Harms from Diagnostic Error in the United States

Abstract

Background:

Diagnostic errors cause substantial preventable harms worldwide, but rigorous estimates for total burden are lacking. We previously estimated diagnostic error and serious harm rates for key dangerous diseases in major disease categories and validated plausible ranges using clinical experts.

Objective:

We sought to estimate the annual US burden of serious misdiagnosis-related harms (permanent morbidity, mortality) by combining prior results with rigorous estimates of disease incidence.

Methods:

Cross-sectional analysis of US-based nationally representative observational data. We estimated annual incident vascular events and infections from 21.5 million (M) sampled US hospital discharges (2012-2014). Annual new cancers were taken from US-based registries (2014). Years were selected for coding consistency with prior literature. Disease-specific incidences for 15 major vascular events, infections, and cancers (“Big Three” categories) were multiplied by literature-based rates to derive diagnostic errors and serious harms. We calculated uncertainty estimates using Monte Carlo simulations. Validity checks included sensitivity analyses and comparison to prior published estimates.

Results:

Annual US incidence was 6.0M vascular events, 6.2M infections, and 1.5M cancers. Per “Big Three” dangerous disease case, weighted mean error and serious harm rates were 11.1% and 4.4%, respectively. Extrapolating to all diseases (including non-“Big Three” dangerous disease categories), we estimated total serious harms annually in the US to be 795,000 (plausible range 598,000-1,023,000). Sensitivity analyses using more conservative assumptions estimated 549,000 serious harms. Results were compatible with setting-specific serious harm estimates from inpatient, emergency department, and ambulatory care. The 15 dangerous diseases accounted for 50.7% of total serious harms and the top 5 (stroke, sepsis, pneumonia, venous thromboembolism, lung cancer) accounted for 38.7%.

Conclusion:

An estimated 795,000 Americans become permanently disabled or die annually across care settings because dangerous diseases are misdiagnosed. Just 15 diseases account for about half of all serious harms, so the problem may be more tractable than previously imagined.

Introduction

Diagnostic error is a major source of preventable harms worldwide across clinical settings,^1-6 but epidemiologically valid estimates of overall misdiagnosis-related morbidity and mortality are lacking. The US National Academy of Medicine describes improving diagnosis in healthcare as a “moral, professional, and public health imperative.”⁷ In its 2015 report, the National Academy concluded that “most people will experience at least one diagnostic error in their lifetime, sometimes with devastating consequences.” However, the report also noted that, “the available research estimates [are] not adequate to extrapolate a specific estimate or range of the incidence of diagnostic errors in clinical practice today.”⁷ This concern is reflected in the wide variation of US estimates for total annual diagnostic errors (12 million to >100 million) and serious misdiagnosis-related harms (40,000 to 4 million).⁸ No studies have yet used nationally representative data sets to measure aggregate US diagnostic errors or harms.

Given wide variation in prior estimates of total diagnostic errors and harms,^8,9 we pursued a novel disease-based approach to constructing a national estimate that would span ambulatory clinic, emergency department, and inpatient care. The disease-based approach leveraged three major disease categories—vascular events, infections, and cancers (the “Big Three”)—found in both malpractice claims and clinical studies of diagnostic error to account for three-quarters of serious harms.⁹ To estimate the total US burden of medical misdiagnosis, we multiplied national estimates of disease incidence (including those initially misdiagnosed) by the disease-specific proportion of patients with that disease experiencing errors and harms. We did this for 15 key diseases causing the most harms, then extrapolated to the grand total across all diseases. To assess the robustness of our final estimates, we used sensitivity analyses to measure the impact of methodological choices and tested validity via comparison to prior literature and expert review.

Methods

This was a three-part research study in which the first two published components^8,9 form the basis of the current analysis, which represents the third and final component (Supplement 1-A1). The main goal of this three-phase research project was to estimate the total number of serious misdiagnosis-related harms (i.e., permanent disability or death) occurring annually in the US across all care settings (ambulatory clinic, emergency department, and inpatient). As reported previously,^8,9 each study phase was designed to answer a key question from a specific data source that would support the final estimate: (1) what dangerous diseases account for the majority of serious misdiagnosis-related harms? (using 10 years of data from a large, nationwide malpractice database representing ~30% of all US claims, then comparing the proportion of “Big Three” diseases to clinical practice-based [non-claims] studies⁹); (2) how common are diagnostic errors potentially causing harm among these dangerous diseases? (using estimates of error and harm rates from high-quality clinical studies,^3,8 further validated by experts); and, for this final component, (3) what is the overall epidemiologic incidence of diagnostic errors and harms among these dangerous diseases? (using nationally representative databases to measure dangerous disease incidence and multiply these by error rates). This final analysis also extrapolates to all (including non-“Big Three”) diagnostic errors and serious misdiagnosis-related harms by using the previously reported⁹ attributable fraction of “Big Three” diseases in clinical practice. We constructed our scientific approach such that the final grand total estimates for errors and harms in the US are based on clinical literature and US population incidence, not malpractice claims. This is because (a) no error or harm rates were taken from claims-based studies, (b) the extrapolation from “Big Three” disease estimates to the grand total were based on the proportion of “Big Three” diseases causing errors and harms from clinical studies (described in Outcome Measures), and (c) any impact of having used malpractice claims to construct the original disease list or weights are mathematically unrelated to the grand totals (Supplement 1-A2). We summarize key aspects of prior study methods^8,9 as needed for readers to follow this final component.

Diagnostic Error, Misdiagnosis-Related Harm, and Harm Severity Definitions

As reported previously,^8,9 we used published definitions for diagnostic error⁷ and misdiagnosis-related harms.^11,12 In this study, we considered only false negative diagnoses (i.e., initially missed or delayed) and associated harms.^3,8 Harms from inappropriate use or overuse of diagnostic tests,^10,11 or from overdiagnosis (i.e., overtreatment of correctly diagnosed conditions that, left undiagnosed, would be unlikely to impact patient health)^12,13 were not considered. Harm severity was categorized according to a recognized insurance industry standard for measuring severity of injury in malpractice claims.^14,15 Serious (high-severity) misdiagnosis-related harms were defined as scale scores 6-9 representing serious permanent morbidity or mortality (Box 1).⁹

Box 1. NAIC scale with specific exemplars used as anchors by CRICO in coding malpractice claim severity.

NAIC 6 — Permanent significant (e.g., deafness, loss of single limb, loss of eye, loss of one kidney or lung; cancers where there is a large tumor possibly with lymph node involvement – this includes cancers that are stage III and stage IV such as breast cancer with total mastectomy, lung cancer with pneumonectomy, or a small cell lung cancer that is inoperable because it has already spread too far).

NAIC 7 — Permanent major (e.g., paraplegia, blindness, loss of two limbs, brain damage)

NAIC 8 — Permanent grave (e.g., quadriplegia, severe brain damage, lifelong care, or fatal prognosis; cancer cases with distant metastasis and / or a prognosis of less than 6 months)

NAIC 9 — Death (including fetal and neonatal death)

Abbreviations: CRICO – Controlled Risk Insurance Company, Ltd.; NAIC – National Association of Insurance Commissioners

Although technically proportions, we use the more common terminology “rates” to describe diagnostic errors and misdiagnosis-related harms for ease of readability. The diagnostic error rate is the proportion of patients with a target disease who were not diagnosed in accurate and timely fashion; the misdiagnosis-related harm rate is the proportion of patients with a target disease who were not diagnosed in accurate and timely fashion and suffered serious harms from the target disease.

Current Study Design & Data Sources

This cross-sectional study multiplied literature-based estimates of diagnostic errors and harms (reported previously by our team^3,8) by nationally representative epidemiologic data on disease incidence (reported here for the first time) to estimate total misdiagnosis-related harms. Multiplying disease incidence by the disease-specific proportion of patients experiencing errors and harms will result in total estimates across care settings (ambulatory clinics, emergency department, and inpatient). False negative diagnostic error and harm rates for 15 key diseases (1-stroke, 2-venous thromboembolism, 3-arterial thromboembolism, 4-aortic aneurysm/dissection, 5-myocardial infarction, 6-sepsis, 7-pneumonia, 8-meningitis/encephalitis, 9-spinal abscess, 10-endocarditis, 11-lung cancer, 12-breast cancer, 13-colorectal cancer, 14-melanoma, 15-prostate cancer) were summarized from clinical studies and vetted by experts.⁸ Our team published a follow-on systematic review³ updating error rates for vascular events and infections. For the present study, we used updated rates only for diseases for which we found high-quality studies that could be subjected to formal meta-analysis³ (diseases #1,2,4,5,6 above). For updated rates, we reapproached relevant experts if revised rates had a more than 1% absolute difference and the previous point estimate fell outside the new estimate’s confidence interval. Only stroke met these criteria; we reapproached two emergency physicians and two stroke neurologists to assess the face validity of the revised rates. As reported previously, for unnamed “other” diseases within each “Big Three” category (i.e., where it was not possible to find literature-derived rates), we substituted the average rate for that category.⁸ To ensure that estimates in this final national analysis were optimized and comparable, we repeated the same statistical procedures as before⁸ but using the revised error rates.

As reported previously,^3,8 diagnostic error rates were all based on studies of missed or delayed diagnoses (i.e., false negatives) among patients with true disease and were abstracted from the highest quality clinical studies we could find. All studies used for these calculations had to have clinical source populations, so no malpractice or autopsy studies were included. In some cases, studies were from countries outside the US (Australia, Canada, New Zealand, UK, and several European nations).^3,8 We discarded lower-quality studies when more rigorous studies (e.g., systematic reviews, population-based sampling, large sample sizes, rigorous case ascertainment) were available. Error rates for vascular events and infections were predominantly derived from studies in emergency department or inpatient settings, while error rates for cancers were predominantly registry based.^3,8 Disease-specific misdiagnosis-related harm rates were derived by multiplying high-quality data on disease-agnostic (non-disease specific) harms per diagnostic error (from well-respected clinical studies) by disease-specific harm-severity weights (from malpractice claims)⁸ (Supplement 1-A2).

We derived population-based data on disease incidence from public use data sets employing nationally representative sampling or census methods. This represents the number at risk for diagnostic error across all clinical settings. All age groups were included. The annual incidence of specific conditions within the “Big Three” disease categories (i.e., vascular events, infections, and cancers) was measured using discharge data from two sources: (1) the National Inpatient Sample (NIS) (2012-2014), Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality¹⁶ and (2) North American Association of Central Cancer Registries (NAACCR)¹⁷ curated by the American Cancer Society (ACS) (2014).¹⁸ The year 2014 was chosen as the last full year in which national data were coded using the International Classification of Diseases 9^th Revision, Clinical Modification (ICD-9-CM), prior to the 2015 transition to the International Classification of Diseases 10^th Revision, Clinical Modification (ICD-10-CM), for coding consistency with the previously published components of the study.^8,9

Disease Incidence for Vascular Events & Infections

The conservative assumption was made that incident cases of dangerous (life or limb-threatening) vascular events and infections in the US would eventually involve a hospitalization, even if the patient was initially misdiagnosed in an ambulatory care setting. Outpatient (e.g., primary care, emergency department) visit diagnoses were not included separately in the disease incidence calculations because they would risk inflating disease incidence estimates through double counting. For example, if “myocardial infarction” cases that were correctly diagnosed in outpatient care (and then later confirmed as an inpatient) had been included in the analysis, the same incident cases would be counted twice. Out-of-hospital deaths from these conditions were not considered, as cause-of-death listings on death certificates are known to be inaccurate for some conditions (e.g., myocardial infarction¹⁹).

HCUP NIS data were used to measure US inpatient hospital stays, counting discharge or in-hospital-death diagnoses coded in either the principal or first-listed secondary diagnosis positions, as these diagnoses are often of equal, competing weight.²⁰ We chose this approach for the primary analysis because (1) using second-position codes can increase sensitivity without sacrificing specificity²¹ and (2) “secondary” diseases are also incident disease cases with the potential to be misdiagnosed, independent from the “primary” disease (e.g., a co-morbid stroke in a patient with endocarditis might also be missed and this additional missed opportunity could also harm the patient).

Disease- and “Big Three” category-level code groupings were the same as those used in prior project phases^8,9 and double-checked for coherence with NIS analysis (Supplement 1-A3). These were derived from HCUP’s Clinical Classification Software, which groups ICD-9-CM codes into clinically meaningful categories. We used NIS data (2012-2014) to estimate the annual number of hospital discharges nationwide by disease and category. A three-year average was chosen to improve stability of incidence measures for rare conditions (e.g., spinal abscess). We followed standard procedures for NIS data to derive nationally representative estimates (Supplement 1-A4).²²

Disease Incidence Data for Cancers

Inpatient hospital stays would not be a good proxy for incident cancer cases, since cancers are treated in outpatient settings and patients are usually only hospitalized for complications. Instead, national incidence counts by cancer site (i.e., body location) were obtained from the 2014 ACS Report.¹⁸ As stated in the report, counts were based primarily on incidence data collected by the NAACCR, which represents 89% of the US population. ACS also used other unidentified sources to generate their final counts, but, because both NAACCR and ACS treat these registry-based estimates as a census (i.e., no sampling-related uncertainty), we did the same. Some ACS categories were grouped to match the prior disease classification from earlier study phases (e.g., colon and rectum cancer grouped as “colorectal”).^8,9

Outcome Measures

The main outcome measures were estimates of total annual diagnostic errors (false negatives) and serious misdiagnosis-related harms (permanent morbidity or mortality) in the US for 2014, across all clinical settings. Outcomes were calculated for the “Big Three” disease categories, including 15 specific diseases (i.e., the previously-identified⁹ top 5 vascular events, infections, and cancers), “other” (non-top 5) diseases within each category, and corresponding category totals.

In turn, these “Big Three” results were used to calculate a grand total (including non-“Big Three” dangerous diseases) using the clinical proportion of diagnostic errors (58.5%) and serious harms (75.8%) attributable to “Big Three” diseases.⁹ These proportions derive exclusively from research studies based in clinical practice (i.e., not malpractice claims studies) (see prior citation,⁹ p237). Mathematically, the grand total of diagnostic errors was calculated by dividing the “Big Three” total number of diagnostic errors by 0.585. Similarly, the grand total of serious misdiagnosis-related harms was calculated by dividing the “Big Three” total number of serious misdiagnosis-related harms by 0.758.

Using the proportion of deaths among serious harms across clinical settings (~46.7%^6,23), we estimated total deaths (total serious harms x proportion of deaths among serious harms = total deaths). By subtraction, we estimated total disabilities (total serious harms – total deaths = total disabilities).

Uncertainty estimates were calculated using a probabilistic sampling approach based on Monte Carlo simulations²⁴ (full statistical R v4.2.2 code is provided in Supplement 2). In this manuscript, many ranges are denoted “probabilistic plausible ranges” (PPRs), rather than 95% confidence intervals (CIs). This is because they rely on some diagnostic error rates (n=5 cancers) that use literature-derived (and expert-validated) plausible ranges (PRs) rather than statistically derived 95% CIs, reflecting uncertainty beyond mere sampling error⁸ (Supplement 1-A5). We used PRs for the top five cancers because different studies defined diagnostic delays of different lengths—defining shorter delays as errors created an upper PR bound, while defining longer delays created a lower PR bound.⁸

Sensitivity Analyses & Validity Checks

We used five separate approaches to assess the robustness of our final results: (1) sensitivity analyses using different data assumptions ([a] one-way analyses to assess the impact of uncertainty in model parameters by using the lower and higher uncertainty bounds rather than the point estimate and [b] the impact of analyzing disease incidence for vascular events and infections using only principal NIS diagnoses) (Supplements 1-B1,1-B2); (2) assessing the risk of misestimating deaths by undercounting (incident cases resulting in pre-hospital death) or overcounting (patients admitted more than once in a given year, yet who could only die once) (Supplements 1-B3,1-B4); (3) comparison to independent hospital and autopsy estimates (Supplements 1-C1,1-C2); (4) triangulation of data derived from studies of diagnostic errors and harms across clinical settings (inpatient, emergency department, ambulatory clinics) (Supplement 1-C3); and (5) an iterative process of expert review by 24 clinical domain experts (following the same method used in our prior publication to validate estimates of error and harm rates⁸), which served as a final check on the face validity of our disease-specific incidence and total harm estimates.

Statistical Analysis & Reporting

We used sample sizes, totals, means, medians, CIs, interquartile ranges [IQR], and PPRs to describe populations and outcomes, as appropriate. NIS analysis was conducted using the PROC SURVEYMEANS procedure in SAS v9.3 (Cary, NC). All other statistical calculations were performed using R v4.2.2 (Vienna, Austria). This manuscript follows EQUATOR (STROBE)²⁵ reporting guidelines for observational studies.

Role of the Funding Source

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

IRB Approval

No human subjects participated in this study, and no IRB approval was needed.

Results

Quality of Data Sources for Error and Harm Rates (published previously^3,8)

For 14 of 15 diseases (besides arterial thromboembolism, where we aggregated four retrospective case series), condition-specific diagnostic error rates were derived from high-quality clinical literature. This included clinical studies with strong designs (large prospective clinical trials or studies using population-based sampling or registries) or meta-analyses of high-quality clinical studies. There were 47 source studies (vascular events [n=28], infections [n=10], cancers [n=9]) representing 942,916 patients (median study sample n=397 [IQR 176-1,914]; median per-disease sample n=2,343 [IQR 398-10,351]). Each study operationalized definitions slightly differently (e.g., nature of diagnostic reference standard), but all definitions for errors/harms were consistent with published definitions described in our Methods.

US Population-Based Incidence of Vascular Events, Infections, & Cancers

The total NIS sample from 2012-2014 included 21.5M hospitalizations (for all conditions, not just vascular events or infections), representing a weighted national estimate of 107.4M total discharges (mean annual 35.8M). In 2014, the sample was taken from 4,411 different hospitals across 45 states. The mean weighted annual number of incident vascular events was 6.0M (95% CI 5.9-6.0). Patients had a median age of 67.5 years (IQR 57.2-78.2, range 0-90); 44.8% were female; and 70.0% were non-Hispanic white. The mean weighted annual number of incident infections was 6.2M (95% CI 6.1-6.3). Patients had a median age of 63.7 years (IQR 52.8-79.8, range 0-90); 51.3% were female; and 68.6% were non-Hispanic white. The number of incident cancer cases in 2014 was 1.5M. Patients had a median age of just over 65 years (<20, 0.9%; 20-49, 11.8%; 50-64, 33.2%; 65-74 28.5%; ≥75 25.7%); 50.7% were female; and 80.0% were non-Hispanic white. The estimated total annual incidence of all “Big Three” diseases was 13.7M (43.5% vascular events, 45.2% infections, 11.3% cancers) (Table 1).

Table 1.

Annual US incidence of dangerous diseases, diagnostic errors, & serious misdiagnosis-related harms

Big Three Disease	Disease Incidencea N, in thousands (95% CI)	Diagnostic Error Ratea % (95% CI, PR, PPRb)	Diagnostic Errors N, in thousands (PPR)	Serious Misdiagnosis-Related Harm Ratea % (PPR)	Serious Harms N, in thousands (PPR)
Vascular
Stroke	952 (937-967)	17.5% (CI: 9.5-27.3)	166 (90-260)	9.8% (5.3-15.5)	94 (51-148)
Venous Thromboembolism	320 (315-324)	20.4% (CI: 17.0-23.9)	65 (54-77)	10.9% (8.9-13.1)	35 (28-42)
Arterial Thromboembolism	173 (170-176)	23.9% (CI: 18.9-29.5)	41 (33-51)	12.7% (9.9-16.0)	22 (17-28)
Aortic Aneurysm & Dissection	96 (93-99)	35.6% (CI: 21.0-51.7)	34 (20-50)	22.1% (13.0-32.5)	21 (12-31)
Myocardial Infarction	1,242 (1,219-1,266)	1.5% (CI: 1.0-2.2)	19 (13-27)	0.8% (0.5-1.2)	10 (7-15)
Top 5 Vascular Events Subtotal	2,783 (2,754-2,811)	11.7% (PPR: 8.8-15.1)	326 (245-421)	6.5% (4.9-8.5)	182 (136-237)
Other Vascular Events	3,173 (3,131-3,215)	11.7% (PPR: 8.8-15.1) c	372 (279-480)	1.4% (1.1-1.9)	46 (34-59)
TOTAL VASCULAR EVENTS	5,956 (5,905-6,006)	11.7% (PPR: 8.8-15.1)	697 (524-900)	3.8% (2.9-5.0)	228 (170-296)
Infection
Sepsis	1,345 (1,325-1,365)	9.9% (CI: 2.8-20.6)	134 (38-278)	5.9% (1.7-12.3)	79 (23-165)
Pneumonia	1,469 (1,452-1,486)	9.5% (CI: 2.3-14.3)	140 (34-210)	4.6% (1.1-7.0)	68 (16-103)
Meningitis & Encephalitis	47 (46-48)	25.6% (CI: 20.8-30.8)	12 (10-15)	14.7% (11.8-18.1)	7 (6-9)
Spinal Abscess	14 (13-14)	62.1% (CI: 54.6-69.2)	8 (7-9)	36.7% (31.5-42.2)	5 (4-6)
Endocarditis	34 (33-35)	25.5% (CI: 21.7-29.6)	9 (7-10)	13.8% (11.5-16.4)	5 (4-6)
Top 5 Infections Subtotal	2,909 (2,882-2,936)	10.4% (PPR: 4.6-14.9)	303 (133-435)	5.6% (2.5-8.3)	164 (73-242)
Other Infections	3,286 (3,249-3,323)	10.4% (PPR: 4.6-14.9) c	342 (151-491)	3.3% (1.4-4.7)	107 (47-154)
TOTAL INFECTIONS	6,195 (6,150-6,241)	10.4% (PPR: 4.6-14.9)	645 (284-925)	4.4% (1.9-6.4)	271 (120-395)
Cancer
Lung Cancer	224 (d)	22.5% (PR: 11.3-37.8)	50 (25-85)	14.2% (7.1-24.1)	32 (16-54)
Breast Cancer	235 (d)	8.9% (PR: 8.5-26.3)	21 (20-62)	4.5% (4.2-13.4)	11 (10-31)
Colorectal Cancer	137 (d)	9.6% (PR: 8.4-47.7)	13 (12-65)	5.6% (4.9-28.1)	8 (7-38)
Melanoma	76 (d)	13.6% (PR: 6.8-25.0)	10 (5-19)	5.7% (2.8-10.6)	4 (2-8)
Prostate Cancer	233 (d)	2.4% (PR: 1.7-13.8)	6 (4-32)	1.3% (0.9-7.4)	3 (2-17)
Top 5 Cancers Subtotal	905 (d)	11.1% (PPR: 10.1-20.9)	100 (92-189)	6.3% (5.6-12.0)	57 (51-108)
Other Cancers	640 (d)	11.1% (PPR: 10.1-20.9) c	71 (65-134)	7.4% (6.7-14.2)	47 (43-91)
TOTAL CANCERS	1,545 (d)	11.1% (PPR: 10.1-20.9)	171 (156-323)	6.8% (6.1-12.8)	105 (94-198)
Additional Totals
TOTAL BIG 3 (Top 5 Only)	6,597 (6,558-6,636)	11.1% (PPR: 8.3-13.8)	729 (549-913)	6.1% (4.6-7.8)	403 (305-511)
TOTAL BIG 3 (Top 5 + Other)	13,697 (13,628-13,765)	11.1% (PPR: 8.2-13.8)	1,514 (1,122-1,889)	4.4% (3.3-5.7)	603 (454-776)
GRAND TOTAL e	N/Ae	N/Ae	2,588 (1,918-3,230)	N/Ae	795 (598-1,023)

^aDisease incidence as measured here is an estimate of total “true disease” cases (rather than only “correctly diagnosed” cases). Diagnostic error and serious misdiagnosis-related harm rates were published previously^3,8 (reference #3: stroke, venous thromboembolism, aortic aneurysm & dissection, myocardial infarction, sepsis; reference #8: all other individual diseases). These rates derive from studies of “true disease” cases. The “diagnostic error rate” and “serious misdiagnosis-related harm rate” are both given with respect to the overall dangerous disease incidence. For example, for stroke (shown in the first content row of Table 1): (a) diagnostic errors are derived as ~952,000 [column #2] x 17.5% [column #3] = ~166,000 [column #4]; (b) serious misdiagnosis-related harms are derived as ~952,000 [column #2] x 9.8% [column #5] = ~94,000 [column #6].

^bShown are either 95% confidence intervals (CI), plausible ranges (PR), or probabilistic plausible ranges (PPR). True statistical confidence intervals (95% CIs) were used when data allowed their calculation without expert input. PRs were used when there was heterogeneity in the findings across disease-specific studies of similar quality or when two different error rates were defined within a single study based on different lengths of diagnostic delay; PRs were thus defined and determined based partially on input from relevant domain experts, as described in the Methods, so reflect more than just sampling-related variability. PPRs derive from Monte Carlo analysis, which included a mix of diagnostic error rates that used 95% CIs and those that used PRs. Because simulations used some PRs (n=5 cancers), all Monte Carlo results are reported as PPRs.

^cBecause we could not estimate error rates for the residual, unnamed non-top 5 “other” diseases within each “Big Three” category, we used the mean error rate for the top 5 diseases (e.g., for unnamed “other” vascular events, we used the mean diagnostic error rate for stroke, venous thromboembolism, arterial thromboembolism, aortic aneurysm and dissection, and myocardial infarction). We used disease incidence-weighted means (e.g., the error rate for myocardial infarction had proportionally more impact on the final mean than the error rate for aortic aneurysm and dissection, because there are ~13-fold more incident cases of myocardial infarction). PPRs derive from Monte Carlo analysis.

^dBecause North American Association of Central Cancer Registries (NAACCR) and American Cancer Society (ACS) treat estimates as a complete census of cases (i.e., no sampling-related uncertainty), no 95% CIs are represented.

^eThe “Grand Total” is calculated from the “Big Three” to all dangerous diseases causing serious misdiagnosis-related harms, based on the proportion of errors (58.5%) and serious harms (75.8%) attributable to the “Big Three” in previously published clinical literature.⁸ Thus, no estimates are provided for “disease incidence,” “diagnostic error rate,” or “serious harm rate” columns. PPRs derive from Monte Carlo analysis.

Abbreviations: CI – confidence interval; PPR – probabilistic plausible range; PR – plausible range

Overall Incidence of Diagnostic Errors & Serious Harms

Table 1 shows annual estimated disease incidence, diagnostic errors, and serious misdiagnosis-related harms by disease and by category (and denotes whether uncertainty for each parameter is represented by CI, PR, or PPR). Serious misdiagnosis-related harms are summarized in Figures 1 and 2. Across the “Big Three” categories, there were 1.51M (PPR 1.12-1.89) missed diagnoses and 603,000 (PPR 454,000-776,000) serious harms; mean diagnostic error and serious harm rates per true disease case for any “Big Three” disease (including “other” subcategories) were 11.1% and 4.4%, respectively. The 15 individually analyzed “Big Three” diseases together accounted for 403,000 serious harms (50.7% of the grand total); mean diagnostic error and serious harm rates per true disease case for the 15 specific diseases (excluding “other” subcategories of the “Big Three”) were 11.1% and 6.1%, respectively. Among these, five conditions linked to the largest numbers of serious harms (stroke, sepsis, pneumonia, venous thromboembolism, and lung cancer) together accounted for 308,000 serious harms (38.7% of the grand total). Across all dangerous diseases (including non-“Big Three”), the grand total estimate was 2.59M (PPR 1.92-3.23) missed diagnoses and 795,000 (PPR 598,000-1,023,000) serious harms (broken down as 371,000 total deaths and 424,000 total disabilities).

Figure 1. Annual population incidence of serious misdiagnosis-related harms from vascular events, infections, cancers, and all non-“Big Three” others.

The estimated grand total annual US incidence for serious harms (combining “Big Three” harms with other non-“Big Three” harms) is 795,000 (PPR 598,000-1,023,000). Whiskers denote probabilistic plausible ranges (PPRs) from the Monte Carlo analysis.

Figure 2. Fraction of serious misdiagnosis-related harms in the US attributed to the top diseases by category.

The treemap diagram proportionally represents hierarchical categories and specific diseases causing serious harms when the diagnosis is incorrect. As we reported previously, based solely on clinical studies, “Big Three” diseases account for 75.8% of all serious harms.⁹ The current analysis shows these are broken down as 34.0% infections, 28.6% vascular events, and 13.2% cancers. Taken together, the top 5 vascular events, infections, and cancers account for 50.7% of all serious harms; the five most frequently harmful conditions across “Big Three” categories account for 38.7% of all serious harms. Abbreviations: AA/AD – aortic aneurysm/aortic dissection; ATE – arterial thromboembolism; CRC – colorectal cancer; EC – endocarditis; ME – meningitis/encephalitis; MI – myocardial infarction; ML – melanoma; PC – prostate cancer; SA – spinal abscess; VTE – venous thromboembolism

Sensitivity Analyses & Validity Checks

The population-level serious harm totals were most sensitive to harm rates for the highest-incidence infections (“other” infections, sepsis, pneumonia) and stroke, but even if each of these harm rates were placed at the lower plausible bound of harms for that specific disease, the grand total of serious harms across all diseases would still be over 500,000 (Supplement 1-B1). Using only principal diagnosis NIS codes, which assumes a lower disease incidence and reduces any residual risks of double counting, gave lower estimates by about 30% (grand totals 1.78M missed diagnoses and 549,000 serious harms, Supplement 1-B2). The impact of methodological assumptions on undercounting (Supplement 1-B3) and overcounting (Supplement 1-B4) were both estimated at <8% and likely offsetting.

Validity checks assessed current results based on similarity to (or coherence with) values derived independently using setting-specific (e.g., hospital-based) medical literature. Estimated misdiagnosis-attributable death rates were 14.1% (n=~371,000 of 2.6M US deaths in 2014) for the primary analysis and 9.8% (n=~256,000 of 2.6M US deaths in 2014) for the principal-only analysis (Supplement 1-C1). By comparison, the literature-derived rate of misdiagnosis-attributable deaths based on hospital autopsies (8.4%, 95% CI 5.2-13.1) and inpatient diagnostic adverse events (~7.4%) were lower, as expected (Supplements 1-C1, 1-C2). Our disease-based estimate of total serious misdiagnosis-related harms (across clinical settings) of ~795,000 (PPR 598,000-1,023,000) was comparable to independent literature-derived values using a setting-based (rather than disease-based) approach, which assessed ~855,000 (490,000-1,659,000) serious misdiagnosis-related harms (Supplement 1-C3, Table S4). Estimates of inpatient misdiagnosis-related deaths derived from our disease-based approach (upper limit ~105,000) fall within the uncertainty bounds of those derived independently from previously published medical literature on hospital autopsies (~82,000 [51K-128K]) and hospital-based adverse events (~72,000 [51,000-113,000]) (Supplement 1-C3). Per-visit serious harm rates by either method were estimated at 0.08% (Supplement 1-C3). After iterative review and feedback (described previously⁸), final estimates for disease-specific incidence, error/harm rates, and total serious harms were deemed face valid by 24 clinical domain experts.

Discussion

This manuscript provides the first robust, national annual US estimate for serious misdiagnosis-related harms (nearly 800,000 combined deaths [~371,000] or permanent disabilities [~424,000]) across care settings (ambulatory clinic, emergency department, and inpatient). Even with the most conservative assumptions about disease incidence or disease-specific harms, we estimated the number affected to be over 500,000. The number of affected patients is large, and this makes diagnostic error a pressing public health concern. Our results also suggest that meaningful progress could be made by addressing just a few dangerous diseases that are relatively common—reducing diagnostic errors by ~50% for the 15 named dangerous diseases could potentially prevent ~200,000 serious misdiagnosis-related harms while reducing diagnostic errors by ~50% for the five most harmful diseases (stroke, sepsis, pneumonia, venous thromboembolism, and lung cancer) could prevent ~150,000.

Sensitivity analyses and validity checks show serious misdiagnosis-related harm results are robust. The impact of methods-induced undercounting and overcounting were relatively small and likely cancel one another. The credibility of our current estimate is bolstered by convergent construct validity with two alternative methods of estimation using the rate of misdiagnosis-attributable deaths based on hospital autopsies and inpatient diagnostic adverse events. Care setting-based estimates using independent, disease-agnostic data from two large systematic reviews (inpatient¹ and emergency department³) also corroborate our findings.

Our results suggest that diagnostic error is probably the single largest source of deaths across all care settings (~371,000) linked to medical error. This number may exceed estimated deaths from all other patient safety concerns combined, regardless of which prior estimate of total deaths due to medical error (range 12,500 to 250,000²⁶) is considered. This seems plausible because prior estimates systematically undercount diagnostic errors and diagnostic errors more often cause serious harms than other errors.²⁷

How many misdiagnosis-associated disabilities or deaths are preventable and how much (or little) longevity might potentially be reclaimed for affected patients is uncertain. Preventability is inconsistently judged by different raters, and some remain skeptical that error prevention can meaningfully increase longevity with a good quality of life.²⁸ Nevertheless, there are numerous anecdotes of otherwise healthy young patients in whom a half-century or more of quality life years are likely to have been saved through prompt diagnosis.²⁹ For some of the most harmful diseases in our list, correct initial diagnosis has been associated with substantial reductions in morbidity or mortality (e.g., ischemic stroke [~5-fold],³ aneurysmal subarachnoid hemorrhage [~5-fold],³⁰ ruptured abdominal aortic aneurysm [~2-fold]³¹). Finally, large variation in diagnostic error and harm rates across demographic groups, diseases, clinical settings, and individual institutions point to strong prospects of preventability for at least some harms.^3,32

Although the study estimated total diagnostic errors (2.59M), this reflects only errors in patients with dangerous diseases, not all diagnostic errors. Total annual diagnostic errors in the US likely number in the tens of millions, but the total is likely highly contingent on the threshold for defining a diagnostic error.⁸ This is different, however, than serious harms (death and permanent disability), which are more objectively defined, so less subject to this particular type of methodological heterogeneity.⁸

The large absolute numbers of patients harmed should not be mistaken for an inordinately high per-incident-case or per-visit risk. According to these results, a patient with a life-or-limb-threatening disease has a ~11% chance of being missed; because of the substantial risk of harm when a dangerous disease is missed, that same patient also has a ~4% overall chance of dying or becoming permanently disabled pursuant to a misdiagnosis. Admittedly, both are higher than what medical experts generally think of as an “acceptable” miss rate for dangerous diseases (e.g., <0.5-1%).^33-35 However, given over 1 billion healthcare visits per year in the US,⁸ a patient visiting a doctor for any reason (i.e., who may or may not have a dangerous underlying disease) likely has a <0.1% chance of suffering serious misdiagnosis-related harms. Thus, patients should not panic or lose faith in the healthcare system.

Although the present study focused on US-based estimates, some of our disease-specific error rates were based on data from other high-income countries outside the US,^3,8 and there is good reason to believe that diagnostic errors and misdiagnosis-related harms represent a global problem. There is meta-analytic evidence that hospital-based diagnostic error and harm rates are comparable across North America and Europe, but higher in other countries.¹ Measured error and harm rates in primary care^4,6,36 and emergency departments³ are similar in the US, UK, and Western Europe. In 2015, OECD nations averaged 6.5 doctor consultations per person year³⁷ and had ~1.3 billion persons³⁸—if per-visit serious harm rates are comparable to the US, this would translate to roughly 7 million serious misdiagnosis-related harms in OECD nations (including the US). Less is known about the scope and nature of diagnostic errors in developing nations. However, access to basic diagnostic testing resources are very limited in many low- and middle-income countries,^39,40 and diagnostic delays for life-threatening diseases can be substantial,^41-43 so the global burden for ~7.9 billion persons is likely several-fold higher.

Disease distributions for serious misdiagnosis-related harms differ across clinical settings and age groups. Missed vascular events and infections dominate in hospitals and emergency departments, while missed cancers dominate in primary care.^3,9 In adult care, vascular events are typical, while in pediatric care, infections are typical.³ Thus, diseases that should be the focus of interventions to improve diagnostic performance would ideally be tailored to the specific clinical context.

This study focused on missed diagnoses (false negatives) of dangerous diseases. While it is desirable to prevent false negatives, practical realities may constrain our ability to do so. Implications for improving diagnosis must consider these results in the broader diagnostic context which includes overuse of diagnostic tests, false positive (mis-)diagnoses, incidental findings, and overdiagnosis,¹³ because these are also associated with substantial harms^10,44,45 and increased healthcare costs.⁴⁶ Reducing missed diagnoses by increasing sensitivity at the expense of specificity (i.e., trading false negatives for false positives by shifting clinical decision thresholds around ordering tests or interpreting test results) should not be considered “improving diagnosis.”^47,48 Instead, diagnostic innovations that increase both sensitivity and specificity at a given test threshold are needed,⁴⁷ as recently shown in a pilot tele-consult program for dizziness and stroke in the emergency department.⁴⁹ Economic modeling may be an important means to estimate the full future impact of solutions designed to improve diagnosis, before they are implemented.⁴⁷

Limitations

Our approach relies on literature-derived estimates being roughly representative of US national diagnostic error and serious harm rates, which cannot be directly verified. Although some estimates based on older studies might not generalize to current practice, limited available evidence suggests diagnostic errors are either stable or rising over time in the US.^3,50 Population-based incidence estimates for vascular events and infections using the NIS are based on administrative codes that could not be independently clinically verified by our team, but annual disease-specific incidence values were deemed face valid by relevant specialists. Our approach is limited by drawing together data from several sources, each with its own uncertainty, so our final estimates are necessarily less precise than would be desirable. This estimate does not account for the sometimes profound effects of non-disabling suffering due to diagnostic delays of non-lethal illnesses, including prolonged diagnostic odysseys,⁵¹ chronic side effects and risks of treatments administered for diseases patients do not actually have (false positives),^52,53 and the substantial health effects and economic consequences of overtesting^10,44 and overdiagnosis.⁴⁵ Nevertheless, our national extrapolations are based on current best evidence regarding error/harm rates, triangulate well with data from other sources, and are face valid to disease-specific domain experts.

Conclusions

Across clinical settings (ambulatory clinics, emergency department, and inpatient), we estimate that 800,000 Americans die or are permanently disabled by diagnostic error each year, making it the single largest source of serious harms from medical mistakes. We believe this is the best estimate currently possible, and, in an area of patient safety where estimates vary widely, results presented here offer an important scientific advance for the field. Although not all these harms are necessarily preventable, our findings add urgency to what the US National Academy of Medicine has already labeled a “moral, professional, and public health imperative.” Policymakers have recently taken notice,⁵⁴ but diagnostic error-related research still remains substantially underfunded relative to its public health impact⁴⁸—to make progress, this must change. Research and quality improvement programs should include a strong focus on prompt diagnosis of vascular events, infections, and cancers, with an emphasis on the top 15 dangerous diseases identified in this study, which together likely account for half of all serious misdiagnosis-related harms. Prospective, interventional studies are needed to confirm the real-world preventability of these harms.

KEY MESSAGES SUMMARY.

What is already known on this topic –

Diagnostic errors are known to be common, costly, and often catastrophic in their health outcomes for patients. Nevertheless, current estimates of the aggregate burden of serious harms resulting from medical misdiagnosis vary widely.

What this study adds –

This study provides the first national estimate of permanent morbidity and mortality resulting from diagnostic errors across all clinical settings, including both hospital-based and clinic-based care (0.6 million to 1.0 million each year in the US alone). It does so via an approach that extrapolates from disease-based estimates for the most common dangerous conditions that often cause serious harms when missed—vascular events, infections, and cancers.

How this study might affect research, practice, or policy –

Because the overall burden of serious misdiagnosis-related harms is quite large, improving diagnosis of dangerous diseases most often responsible—stroke, sepsis, pneumonia, venous thromboembolism, and lung cancer—constitutes an urgent public health imperative.