Key Points
Question
Of emergency department visits attributable to imminent ruptured abdominal aortic aneurysm, acute myocardial infarction, stroke, aortic dissection, and subarachnoid hemorrhage, what proportions end in discharge home without diagnosis?
Findings
In this cohort study of Medicare claims, the proportion of missed opportunities to diagnose these conditions in the emergency department ranged from 2.3% (ruptured abdominal aortic aneurysm) to 4.5% (aortic dissection). We found no evidence for improvement across the 2007 to 2014 study time frame.
Meaning
Among Medicare patients, opportunities to diagnose these conditions in the emergency department are missed infrequently, but further improvement may prove difficult.
Abstract
Importance
The Institute of Medicine described diagnostic error as the next frontier in patient safety and highlighted a critical need for better measurement tools.
Objectives
To estimate the proportions of emergency department (ED) visits attributable to symptoms of imminent ruptured abdominal aortic aneurysm (AAA), acute myocardial infarction (AMI), stroke, aortic dissection, and subarachnoid hemorrhage (SAH) that end in discharge without diagnosis; to evaluate longitudinal trends; and to identify patient characteristics independently associated with missed diagnostic opportunities.
Design, Setting, and Participants
This was a retrospective cohort study of all Medicare claims for 2006 to 2014. The setting was hospital EDs in the United States. Participants included all fee-for-service Medicare patients admitted to the hospital during 2007 to 2014 for the conditions of interest. Hospice enrollees and patients with recent skilled nursing facility stays were excluded.
Main Outcomes and Measures
The proportion of potential diagnostic opportunities missed in the ED was estimated using the difference between observed and expected ED discharges within 45 days of the index hospital admissions as the numerator, basing expected discharges on ED use by the same patients in earlier months. The denominator was estimated as the number of recognized emergencies (index hospital admissions) plus unrecognized emergencies (excess discharges).
Results
There were 1 561 940 patients, including 17 963 hospitalized for ruptured AAA, 304 980 for AMI, 1 181 648 for stroke, 19 675 for aortic dissection, and 37 674 for SAH. The mean (SD) age was 77.9 (10.3) years; 8.9% were younger than 65 years, and 54.1% were female. The proportions of diagnostic opportunities missed in the ED were as follows: ruptured AAA (3.4%; 95% CI, 2.9%-4.0%), AMI (2.3%; 95% CI, 2.1%-2.4%), stroke (4.1%; 95% CI, 4.0%-4.2%), aortic dissection (4.5%; 95% CI, 3.9%-5.1%), and SAH (3.5%; 95% CI, 3.1%-3.9%). Longitudinal trends were either nonsignificant (AMI and aortic dissection) or increasing (ruptured AAA, stroke, and SAH). Patient characteristics associated with unrecognized emergencies included age younger than 65 years, dual eligibility for Medicare and Medicaid coverage, female sex, and each of the following chronic conditions: end-stage renal disease, dementia, depression, diabetes, cerebrovascular disease, hypertension, coronary artery disease, and chronic obstructive pulmonary disease.
Conclusions and Relevance
Among Medicare patients, opportunities to diagnose ruptured AAA, AMI, stroke, aortic dissection, and SAH are missed in less than 1 in 20 ED presentations. Further improvement may prove difficult.
This cohort study estimates the proportions of emergency department visits attributable to symptoms of imminent ruptured abdominal aortic aneurysm, acute myocardial infarction, stroke, aortic dissection, and subarachnoid hemorrhage that end in discharge without diagnosis.
Introduction
Accurate and timely diagnosis is a cornerstone of high-quality care. A report by the Institute of Medicine (IOM), Improving Diagnosis in Health Care, calls diagnostic error an underappreciated source of patient harm and “a blind spot” in the patient safety movement.” The report cites the inherent difficulty in measuring the incidence of diagnostic error as an impediment to improvement and states that developing measurement tools should be an urgent research priority. As motivational examples, the report presents several vignettes of patients who are seen at the emergency department (ED) with symptoms of acute, life-threatening cardiovascular emergencies.
The IOM committee suggests that to quantify diagnostic error “one would need an estimate of the number of opportunities to make a diagnosis each year (denominator) and the number of times the diagnosis (health problem) is not made in an accurate and timely manner or is not communicated to the patient.”(p7) Although the word error has the linguistic implication of fault, the committee distinguished between failure of diagnostic process and the outcome of that process, choosing an outcome-based definition of error as patient centric. They note that one does not necessarily imply the other. For example, a radiograph misread by a radiologist but correctly overread by the ordering physician would be a near miss but not a diagnostic error. Conversely, if a patient is seen with an aortic dissection and is discharged, then a diagnostic error occurred, no matter how nonspecific the clinical presentation.
Defined this way, the frequency of diagnostic error can only be measured retrospectively. If the goal is to identify specific instances of diagnostic error, the presence of the disease during the encounters in question would need to be adjudicated after medical record review. However, to estimate the proportion of diagnostic opportunities missed, we suggest that a statistical approach may offer a more accurate, reproducible, and feasible alternative.
In this study, we consider the ED diagnosis of the following 5 life-threatening emergencies: ruptured abdominal aortic aneurysm (AAA), acute myocardial infarction (AMI), stroke, aortic dissection, and subarachnoid hemorrhage (SAH). Adapting the IOM’s formulation, we measure the proportion of patients discharged home among those seen at the ED with symptoms retrospectively attributable to the underlying acute cardiovascular pathology.
Methods
Study Population and Data Sources
We defined the study population as all fee-for-service (FFS) Medicare patients newly diagnosed as having 1 of the 5 conditions of interest during a hospitalization that began in the ED during calendar years 2007 to 2014. The index conditions were defined according to the coded principal discharge diagnosis (eTable 1 in the Supplement). To ensure that the pathology was acute, that all preceding ED visits were captured, and that patients were not knowingly discharged home despite the acute condition, index cases were excluded for the following reasons: (1) lack of continuous FFS Medicare enrollment in the 12 months preceding the index hospital admission, (2) previous enrollment in hospice, (3) a claim for a skilled nursing facility stay within the preceding 30 days, (4) an ED visit within the preceding 365 days where the index condition was listed as a diagnosis, and (5) the absence of ED-specific charges (Medicare Provider Analysis Review File [MedPAR]) or a matching outpatient ED claim (outpatient file) indicating admission occurred via an ED. The data sources were Medicare standard analytic files (100%) for the calendar years 2006 to 2014 (allowing for a 1-year look-back period).
The study was approved by RAND’s institutional review board. Informed consent was not applicable.
Conceptual Overview
We performed separate analyses for each of the 5 conditions. Starting with the index hospital admission, we look back in time to identify all ED visits in the preceding year. We assume that the acute pathology giving rise to the index condition (eg, rupture of a coronary plaque preceding AMI) begins in the hours, days, or weeks before the index hospital admission. At some point after the acute pathology appears, patients are seen at the ED for symptoms that are in some way related (albeit sometimes nonspecific) and either are admitted to the hospital (or transferred) and diagnosed or are discharged with an unrecognized emergency. Those who are discharged return to the ED one or more times and are eventually admitted.
We treat the presence of the underlying acute pathology at ED discharge as unobservable in the data. Patients may coincidentally be seen at the ED for problems unrelated to the cardiovascular emergency at any time before the index hospital admission. We specify that the underlying pathology can start no more than 45 days before the index hospital admission (based on preliminary inspection of the data) and demonstrate in a sensitivity analysis that results are robust to choices of shorter or longer durations. We estimated the number of unrecognized emergencies in each population as the number of “excess ED discharges” within those 45 days. Excess discharges, defined as the difference between the number observed and the number expected to occur, are based on the mean number of daily ED discharges for the same patients earlier in the year. Logistic regression was used to adjust for a slightly increasing rate with each passing day owing to age and a survivorship bias (eg, an aggressive cancer is less likely to appear earlier in the year because we impose the constraint that all patients must survive to the index hospital admission).
The primary study outcome (for each condition) is formulated as the false-negative rate—the proportion of ED visits where the diagnosis was not made (ie, the patient was discharged)—among all visits where the acute pathology was present. The numerator (unrecognized emergencies) is estimated as the number of excess ED discharges. Because each index patient has only one ED visit leading to correct diagnosis, we estimated the denominator as the sum of index hospital admissions (recognized emergencies) plus excess discharges (unrecognized emergencies).
Methodological Detail
We aggregated data across all available years. To evaluate longitudinal trends, we also performed yearly estimates per index hospital admission date. To calculate the expected number of discharges in the 45 days preceding the index hospital admission, we expanded the data to 365 observations for each patient. We fit a logistic regression model of whether an individual had an ED discharge on any given day as a function of the number of days to the index hospital admission, using observations that occurred 365 to 45 days before the index visit to fit the model. We calculated expected discharges as the aggregate of predicted probabilities for the 45 days previously excluded. Because each patient’s observations were correlated in time, only point estimates were used; we obtained 95% CIs for the overall result via the bootstrap method.
To avoid counting official or unofficial transfers miscoded as discharges, ED discharges were excluded if they occurred on the same day as the index hospital admission (any) or if they occurred 1 day prior and the discharge diagnosis was for the index condition itself. Observation stays were treated as discharges if patients were discharged home at the end of observation and were treated as admissions if the patient was subsequently admitted to the hospital. As a sensitivity analysis, we show how results are affected by excluding ED visits that ended in observation and then discharge. We also show how results are affected by casewise exclusion of patients hospitalized for any other reason within 30 days of the index admission, to mitigate possible confounding (see the Discussion section).
As an exploratory analysis meant to illustrate expanded applications of our approach, we measured multivariable associations between patient characteristics or socioeconomic factors and unrecognized emergencies. To do so, we fit patient-level logistic models, with ED discharge during the 45-day preadmission window (one or more) as the dependent variable, as a function of age, sex, Medicare and Medicaid dual eligibility, race/ethnicity, and a history of any of the following conditions (chosen a priori on clinical grounds, defined by Chronic Conditions Data Warehouse indicators provided by the Centers for Medicare & Medicaid Services): end-stage renal disease, dementia, depression, diabetes, hypertension, cerebrovascular disease, or ischemic heart disease. We also included the number of ED discharges earlier in the year (ie, the 320 days preceding the 45-day window) as a covariate, to account for possible correlation between patient characteristics and baseline ED use.
Similarly, we identified ED discharge diagnoses occurring with greater than expected frequency during the 45 days before index hospital admissions compared with the preceding 320 days. We report incidence rate ratios (IRRs) and exact 95% CIs for the 20 diagnoses that occur in greatest excess and the 20 diagnoses with the highest IRRs.
Statistical Analysis
Analyses were performed with SAS Enterprise Guide (version 7.1; SAS Institute Inc), hosted by the Centers for Medicare & Medicaid Services Virtual Research Data Center, and in Stata (version 13.1; StataCorp LP). We created 95% CIs for our main estimates by clustered bootstrap at the level of the patient, sampling individuals with replacement over 1000 replications, fitting the logistic model, and performing the post-fit calculations for each replication, using the 2.5th and 97.5th percentiles of these results for the 95% CI. To evaluate longitudinal trends, we fit a linear regression (with robust SEs) of the logarithm of each year’s estimate as a function of calendar year, and we report the mean annual percentage change with the exponentiated coefficient.
Results
We identified 1 873 207 Medicare FFS beneficiaries with 1 of the 5 conditions of interest. Of these, 152 484 were excluded because index hospital admission via the ED could not be established, 146 621 because they had previously been enrolled in hospice (n = 19 852) and/or had a skilled nursing facility claim within the prior 30 days (n = 131 220), and 12 162 because they had a previous ED discharge for the index diagnosis. For the 5 conditions in aggregate, the mean (SD) age was 77.9 (10.6) years. The proportion younger than 65 years ranged from 3.1% for ruptured AAA to 14.5% for SAH (8.9% overall) (Table 1). Mortality during the index hospital admission ranged from 5.2% for stroke to 48.2% for ruptured AAA.
Table 1. Characteristics of Patients, by Index Conditiona.
| Variable | Ruptured Abdominal Aortic Aneurysm (n = 17 963) | Acute Myocardial Infarction (n = 304 980) | Stroke (n = 1 181 648) | Aortic Dissection (n = 19 675) | Subarachnoid Hemorrhage (n = 37 674) |
|---|---|---|---|---|---|
| Age, mean (SD), y | 79.0 (8.4) | 75.1 (10.7) | 78.7 (10.2) | 76.5 (10.6) | 74.6 (11.6) |
| Age <65 y, No. (%) | 556 (3.1) | 38 481 (12.6) | 92 875 (7.9) | 1947 (9.9) | 5463 (14.5) |
| Female sex, No. (%) | 5763 (32.1) | 133 162 (43.7) | 671 266 (56.8) | 10 280 (52.2) | 24 677 (65.5) |
| Medicare and Medicaid dual eligibility, No. (%) | 2578 (14.4) | 59 993 (19.7) | 278 897 (23.6) | 3755 (19.1) | 9357 (24.8) |
| Race/ethnicity, No. (%) | |||||
| White, not Hispanic | 16 293 (90.7) | 263 225 (86.3) | 953 830 (80.7) | 15 709 (79.8) | 29 155 (77.4) |
| Black | 861 (4.8) | 19 218 (6.3) | 138 527 (11.7) | 2532 (12.9) | 4149 (11.0) |
| Asian/Pacific Islander | 239 (1.3) | 4700 (1.5) | 21 123 (1.8) | 508 (2.6) | 1162 (3.1) |
| Hispanic | 397 (2.2) | 14 210 (4.7) | 55 958 (4.7) | 708 (3.6) | 2630 (7.0) |
| Other/unknown | 173 (1.0) | 3627 (1.2) | 12 210 (1.0) | 218 (1.1) | 578 (1.5) |
| Chronic conditions, No. (%) | |||||
| End-stage renal disease | 523 (2.9) | 7412 (2.4) | 28 518 (2.4) | 790 (4.0) | 1037 (2.8) |
| Dementia | 2415 (13.4) | 37 163 (12.2) | 270 389 (22.9) | 2738 (13.9) | 6434 (17.1) |
| Depression | 14 005 (78.0) | 221 298 (72.6) | 790 427 (66.9) | 13 494 (68.6) | 24 864 (66.0) |
| Diabetes | 4893 (27.2) | 115 717 (37.9) | 526 417 (44.5) | 5594 (28.4) | 13 073 (34.7) |
| Stroke/transient ischemic attack | 3487 (19.4) | 50 694 (16.6) | 384 752 (32.6) | 4077 (20.7) | 9636 (25.6) |
| Hypertension | 14 729 (82.0) | 245 875 (80.6) | 1 059 969 (89.7) | 17 537 (89.1) | 30 881 (82.0) |
| Coronary artery disease | 10 791 (60.1) | 178 439 (58.5) | 753 747 (63.8) | 12 268 (62.4) | 20 005 (53.1) |
| Chronic obstructive pulmonary disease | 7922 (44.1) | 87 987 (28.9) | 393 489 (33.3) | 7496 (38.1) | 11 641 (30.9) |
| Hospital mortality, No. (%) | |||||
| Died during index hospital admission | 8666 (48.2) | 33 612 (11.0) | 61 398 (5.2) | 4617 (23.5) | 10 862 (28.8) |
Each column represents a distinct study population. For each population, the number and proportion of patients with each characteristic are shown. In-hospital mortality rates are given in the last row.
Among the 1 561 940 individuals remaining after exclusions, there were 902 159 ED discharges to home 1 to 365 days before the index hospital admission. Histograms showing the frequency of ED discharges as a function of the number of elapsed days before the index hospital admission are shown in Figure 1 and in the eFigure in the Supplement.
Figure 1. Emergency Department (ED) Discharges by Time Before Index Visit for Acute Myocardial Infarction.
The timing of ED discharges among the 304 980 patients ultimately diagnosed as having acute myocardial infarction. The height of each vertical bar represents the observed number of daily ED discharges, and the regression line represents the number expected. The logistic regression was fitted using the unshaded portion of the graph (ie, days 365-46). The number of unrecognized diagnostic opportunities is represented as the net difference between the number of ED discharges observed (vertical bars) and the number expected (regression line) during the 45 days preceding the index hospital admission (shaded area). eFigure 1 in the Supplement provides a similar illustration of the other 4 conditions.
Table 2 lists the main study results. Estimates for the proportion of ED diagnostic opportunities that were missed are as follows: ruptured AAA (3.4%; 95% CI, 2.9%-4.0%), AMI (2.3%; 95% CI, 2.1% -2.4%), stroke (4.1%; 95% CI, 4.0%-4.2%), aortic dissection (4.5%; 95% CI, 3.9%-5.1%), and SAH (3.5%; 95% CI, 3.1%-3.9%).
Table 2. Results Summary (Aggregate Data for Index Hospital Admissions, 2007 to 2014)a.
| Variable | Index Patients | Observed ED Discharges | Expected ED Discharges | Potential Diagnostic Opportunities Missed, % (95% CI) |
|---|---|---|---|---|
| Ruptured abdominal aortic aneurysm | 17 963 | 1603 | 970 | 3.4 (2.9-4.0) |
| Acute myocardial infarction | 304 980 | 25 622 | 18 483 | 2.3 (2.1-2.4) |
| Stroke | 1 181 648 | 142 743 | 92 348 | 4.1 (4.0-4.2) |
| Aortic dissection | 19 675 | 2482 | 1565 | 4.5 (3.9-5.1) |
| Subarachnoid hemorrhage | 37 674 | 4245 | 2871 | 3.5 (3.1-3.9) |
Summarized are the principal study results, pooling index hospital admissions across 8 years. The total number of patients admitted for each disease is shown, followed by the number of emergency department (ED) discharges observed to occur within 45 days of an index hospital admission, and the point estimate for the number expected for that same interval. The final column is calculated as the ratio of excess ED discharges (observed and expected) to the sum of excess discharges plus index hospital admissions, representing the ratio of missed diagnostic opportunities to the sum of all diagnostic opportunities (recognized and unrecognized emergencies). The 95% CIs were calculated by clustered bootstrap as described in the Methodological Detail subsection of the Methods section.
Figure 2 shows longitudinal trends. The mean annual changes in the proportion of diagnostic opportunities missed (ie, the relative change) were as follows: ruptured AAA (8.7% per year increase; 95% CI, 1.5%-16.4%), AMI (2.3% per year decrease; 95% CI, 6.0% decrease to 1.6% increase), stroke (1.7% per year increase; 95% CI, 1.0%-2.4%), aortic dissection (1.1% per year decrease; 95% CI, 4.8% decrease to 2.7% increase), and SAH (6.5% per year increase; 95% CI, 0.0%-13.5%).
Figure 2. Unrecognized Emergencies by Type and Year.
Shown are longitudinal trends. Yearly point estimates and 95% CIs are calculated in the same manner as for the results presented in Table 2 but using the subset of patients whose index hospital admission began during the calendar year.
Results of sensitivity analyses are listed in eTables 2, 3, and 4 in the Supplement. Exclusion of patients who had been admitted to the hospital within 30 days before their index event (and eliminating ED discharges preceded by hospital admissions) reduced estimates slightly (eTable 2 in the Supplement). Exclusion of patients admitted under observation status and then discharged home (90 816 [10.1% of all ED discharges to home]) also reduced estimates slightly (eTable 3 in the Supplement). Using 30 or 60 days rather than 45 days as the duration of the pre–index event window for the observed vs expected calculation had negligible associations (eTable 4 in the Supplement).
Associations With Patient Characteristics
In multivariable analyses that adjusted for demographics, chronic conditions, and the number of visits at baseline, excess ED discharges within 45 days of index hospital admission were positively correlated with age younger than 65 years, female sex, Medicare and Medicaid dual eligibility, and a history of end-stage renal disease, dementia, depression, stroke or transient ischemic attack (TIA), hypertension, coronary artery disease, and chronic obstructive pulmonary disease (Table 3). The associations with race/ethnicity and diabetes varied across conditions.
Table 3. Multivariable Model Predicting Emergency Department (ED) Discharge Within 45 Days of Index Hospital Admissiona.
| Variable | Ruptured Abdominal Aortic Aneurysm | Acute Myocardial Infarction | Stroke | Aortic Dissection | Subarachnoid Hemorrhage |
|---|---|---|---|---|---|
| Age, yb | |||||
| <65 | 1.27 (0.94-1.73) | 1.40 (1.33-1.48) | 1.38 (1.35-1.42) | 1.33 (1.09-1.62) | 1.69 (1.48-1.92) |
| 70-74 | 0.73 (0.59-0.91) | 0.94 (0.90-1.00) | 0.93 (0.90-0.95) | 0.98 (0.81-1.17) | 0.97 (0.85-1.11) |
| 75-79 | 0.75 (0.61-0.93) | 0.96 (0.91-1.02) | 0.89 (0.87-0.92) | 0.89 (0.74-1.06) | 0.97 (0.85-1.11) |
| 80-84 | 0.67 (0.54-0.83) | 1.03 (0.98-1.09) | 0.90 (0.88-0.92) | 0.98 (0.81-1.17) | 0.96 (0.84-1.10) |
| >84 | 0.83 (0.68-1.02) | 1.05 (0.99-1.11) | 0.92 (0.90-0.94) | 0.85 (0.70-1.02) | 0.94 (0.82-1.08) |
| Sexc | |||||
| Female | 1.25 (1.11-1.41) | 1.14 (1.11-1.17) | 1.12 (1.11-1.14) | 1.19 (1.08-1.32) | 1.00 (0.92-1.08) |
| Race/ethnicityd | |||||
| Asian/Pacific | 0.78 (0.45-1.35) | 0.67 (0.59-0.77) | 0.68 (0.65-0.72) | 0.65 (0.46-0.93) | 0.52 (0.40-0.69) |
| Black | 1.35 (1.08-1.70) | 1.18 (1.12-1.24) | 1.09 (1.07-1.11) | 0.93 (0.80-1.07) | 1.00 (0.90-1.12) |
| Hispanic | 1.16 (0.82-1.66) | 0.91 (0.85-0.97) | 0.90 (0.88-0.93) | 1.03 (0.81-1.31) | 0.91 (0.79-1.05) |
| Other/unknown | 1.05 (0.60-1.85) | 1.01 (0.88-1.14) | 1.07 (1.01-1.14) | 1.19 (0.78-1.82) | 0.87 (0.65-1.17) |
| Medicare and Medicaid dual eligibilitye | |||||
| Dually eligible | 1.23 (1.05-1.42) | 1.40 (1.35-1.45) | 1.24 (1.22-1.25) | 1.33 (1.17-1.50) | 1.29 (1.18-1.41) |
| Chronic conditionsf | |||||
| End-stage renal disease | 1.59 (1.22-2.07) | 1.60 (1.49-1.71) | 1.53 (1.48-1.58) | 1.71 (1.40-2.08) | 1.77 (1.51-2.09) |
| Dementia | 1.22 (1.04-1.43) | 1.22 (1.17-1.27) | 1.17 (1.15-1.19) | 1.10 (0.96-1.26) | 1.21 (1.10-1.34) |
| Depression | 1.27 (1.12-1.45) | 1.32 (1.28-1.37) | 1.23 (1.21-1.25) | 1.31 (1.18-1.45) | 1.27 (1.18-1.38) |
| Diabetes | 0.96 (0.85-1.09) | 1.03 (1.00-1.07) | 1.05 (1.04-1.07) | 0.99 (0.89-1.11) | 1.10 (1.01-1.18) |
| Stroke/transient ischemic attack | 0.94 (0.82-1.09) | 1.14 (1.10-1.18) | 1.25 (1.23-1.26) | 1.14 (1.02-1.28) | 1.28 (1.18-1.38) |
| Hypertension | 1.61 (1.30-1.98) | 1.40 (1.33-1.47) | 1.36 (1.33-1.40) | 1.39 (1.14-1.70) | 1.16 (1.04-1.30) |
| Coronary artery disease | 1.30 (1.13-1.49) | 1.43 (1.38-1.48) | 1.14 (1.12-1.16) | 1.19 (1.07-1.33) | 1.14 (1.05-1.24) |
| Chronic obstructive pulmonary disease | 1.39 (1.23-1.56) | 1.30 (1.26-1.34) | 1.23 (1.21-1.24) | 1.27 (1.15-1.40) | 1.11 (1.03-1.20) |
| Baseline ED useg | |||||
| No. of ED discharges 365-46 d before index event (per discharge) | 1.40 (1.30-1.50) | 1.36 (1.34-1.39) | 1.26 (1.25-1.27) | 1.24 (1.17-1.31) | 1.33 (1.27-1.38) |
Each column shows results (adjusted odds ratios) for a disease-specific multivariable logistic regression model. The dependent variable, whether an individual had one or more ED discharges within the 45 d preceding the index hospital admission, is modeled as a function of age, sex, Medicare and Medicaid dual eligibility, race/ethnicity, the chronic conditions listed, and the number of ED discharges in the 365 to 46 d before the index hospital admission (which adjusts for an individual’s propensity to visit the ED for reasons unrelated to the acute cardiovascular emergency). Variables were chosen a priori, based on previously published studies, clinical plausibility, and availability.
Reference group is 65 to 69 years.
Dichotomous vs male.
Reference group is white, non-Hispanic.
Dichotomous vs not Medicaid eligible.
Dichotomous condition vs no condition for each.
Continuous per ED visit.
Discharge Diagnoses
Emergency department discharge diagnoses in greatest excess during the 45 days before admission were as follows: abdominal pain (unspecified site) for ruptured AAA (66 excess discharges; IRR for days 1-45 vs 46-365, 6.7; 95% CI, 4.5-9.4), chest pain (unspecified) for AMI (1194 excess discharges; IRR, 2.8; 95% CI, 2.6-2.9), unspecified transient cerebral ischemia for stroke (4532 excess discharges; IRR, 5.2; 95% CI, 5.0-5.4), chest pain (unspecified) for aortic dissection (149 excess discharges; IRR, 3.9; 95% CI, 3.3-4.7), and headache for SAH (290 excess discharges; IRR, 5.7; 95% CI, 4.9-6.6). eTable 5 in the Supplement lists the 20 diagnoses in greatest excess and the 20 diagnoses with the highest IRR, for each condition.
Discussion
This study introduces what is to our knowledge a novel approach to measuring missed diagnostic opportunities in the ED. Among Medicare patients with the 5 diseases studied, we estimate that between 2.3% (AMI) and 4.5% (aortic dissection) of ED visits for symptoms relating to an imminent emergency end in discharge home without diagnosis. In a multivariable analysis, we demonstrate that patients who are younger than 65 years, female, or poor (ie, dually eligible for Medicare and Medicaid) or those who have chronic medical conditions were at increased risk. We see no evidence of improvement during the time frame of this study from 2007 to 2014.
Direct comparisons with previous reports are complicated by differing definitions of outcomes and metrics. For example, some studies define missed AMI as the actual presence of an AMI at the time of ED discharge. Herein, we define each condition as the presence of the underlying acute pathology, where a ruptured plaque, an intimal tear, a TIA, or a sentinel bleed would be counted. As recommended in the IOM report cited at the beginning of this article, we use the proportion of diagnostic opportunities missed (ie, the false-negative rate) as our metric of interest. Compared with commonly used alternatives, our formulation has the advantage of not being conditional on patients’ propensity to visit the ED, allowing comparison between populations or health systems.
We relax several assumptions common to previous work and demonstrate that they are often violated. Whereas it has been typical to consider ED discharges only within 1 or 2 weeks of an index hospital admission, we took a data-driven approach and found that excess visits can be detected earlier. Older literature supports this finding; however, in an era when patients are quicker to seek emergency care, the possibility that acute cardiovascular pathology can remain undiagnosed for weeks may not be fully appreciated. A longer look-back period amplifies the importance of accounting for coincidental ED visits, which few previous studies do directly. We found that approximately two-thirds of the ED visits within the 45 days preceding an index hospital admission can be attributed to the population’s baseline rate of ED use. Failure to account for baseline use would substantially overestimate the frequency of events.
The ED discharge diagnosis has often been used to retrospectively adjudicate whether an unrecognized emergency was present at the time of an ED discharge. In our analysis, we found that the most common diagnoses associated with excess visits are what one might expect (eg, unspecified abdominal pain for ruptured AAA), but we also found that seemingly unrelated diagnoses are common. Published accounts of high-profile diagnostic errors support this contention. For example, the playwright Jonathan Larson was discharged from 2 EDs in the week before his death from an undiagnosed aortic dissection, with diagnoses of “food poisoning” and “viral syndrome.” While it is clear from subsequent investigation that his ED visits were prompted by symptoms of the dissection, discharge diagnosis alone could not have distinguished between misdiagnosis and coincidence.
Limitations
Our method relies on a key assumption that nothing other than the acute cardiovascular pathology should increase ED visits just before the index hospital admission. One can imagine confounding by other factors that prompt ED visits and also cause cardiovascular emergency to occur. For example, vascular surgery might prompt ED visits for wound complications and may increase the risk for postoperative myocardial infarction or stroke. Confounding conditions would only influence results if they arise acutely. For example, psychological stress might precipitate both ED visits and cardiovascular emergencies, but would be accounted for in the baseline rate of ED use if chronic. Because the most plausible confounders (particularly high-risk surgery) would usually involve a hospitalization, we tested the association of excluding patients whose index hospital admission was preceded by any other admission within 30 days, and only a small association with results was found. To the extent that confounding remains, it would bias estimates upward (ie, the true proportion of emergencies unrecognized would be lower than what we report).
In the other direction of bias (toward underestimating the diagnostic error rate), we do not account for patients who die before they are admitted to a hospital. This is a limitation of most previous work on the topic. Future study might take the approach of treating outpatient deaths as index events, although the inaccuracy of death certificate diagnoses (particularly for out-of-hospital deaths) presents a challenge. Another limitation is that we do not account for diagnoses that are not appreciated in the ED but made after admission (or after death).
Our task was simplified by choosing to focus on cardiovascular emergencies generally understood to mandate hospital admission from the ED (unless patients’ primary goal is palliation). However, we found TIA to be among ED discharge diagnoses occurring in excess in the weeks before stroke hospital admissions, suggesting that for TIA the risk reduction achieved with hospitalization might not always be perceived to be worth the costs, financial or otherwise. Stroke estimates must be interpreted with that caveat.
Conclusions
Among Medicare patients, opportunities to diagnose ruptured AAA, AMI, stroke, aortic dissection, and SAH are missed in less than 1 in 20 ED presentations. Further improvement may prove difficult.
The frequency of unrecognized emergencies can only be estimated retrospectively; therefore, the call for better measurement tools must be answered with studies such as this one. Our study was not designed to consider trade-offs between diagnostic sensitivity and specificity and thus cannot address clinical decision making. However, the absence of improvement over the 8-year time frame of the study raises the question of whether our ability to diagnose these acute emergencies has reached a plateau. Some researchers have suggested a natural asymptote, where the costs or risks of seeking additional diagnostic certainty become prohibitive. Studies of organization-level variation aimed at identifying high-performing or low-performing outliers might help us understand whether we can do better. Regardless, measurement is important because technological or practice innovation might improve or degrade diagnostic sensitivity in the future. Cardiovascular emergencies provide a straightforward example of how surveillance for excess ED visits can be used to monitor diagnostic error, much as it has been used in other public health contexts. More widespread application of probabilistic approaches—to other diagnoses and to other care venues—will require further methodological innovation but seems inevitable.
eTable 1. ICD-9-CM Codes Used to Define the Index Hospital Admission
eTable 2. Effect of Excluding Recent Hospitalizations
eTable 3. Effect of Excluding Discharges After Observation Admissions
eTable 4. Effect of Varying Duration of “Pre-admission Window”
eTable 5. Discharge Diagnoses Associated With Unrecognized Emergencies
eFigure. Distribution of ED Visits
References
- 1.Institute of Medicine, National Academies of Sciences, Engineering, and Medicine Improving Diagnosis in Health Care. Washington, DC: National Academies Press; 2015. [Google Scholar]
- 2.McGlynn EA, McDonald KM, Cassel CK. Measurement is essential for improving diagnosis and reducing diagnostic error: a report from the Institute of Medicine. JAMA. 2015;314(23):2501-2502. [DOI] [PubMed] [Google Scholar]
- 3.Research Data Assistance Center How to identify hospital claims for emergency room visits in the Medicare claims data. https://www.resdac.org/resconnect/articles/144. Published July 30, 2015. Accessed January 23, 2018.
- 4.Centers for Medicare & Medicaid Services Chronic Conditions Data Warehouse. https://www.ccwdata.org/web/guest/condition-categories. Accessed July 20, 2017.
- 5.Pope JH, Aufderheide TP, Ruthazer R, et al. . Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med. 2000;342(16):1163-1170. [DOI] [PubMed] [Google Scholar]
- 6.McCarthy BD, Beshansky JR, D’Agostino RB, Selker HP. Missed diagnoses of acute myocardial infarction in the emergency department: results from a multicenter study. Ann Emerg Med. 1993;22(3):579-582. [DOI] [PubMed] [Google Scholar]
- 7.Lee TH, Rouan GW, Weisberg MC, et al. . Clinical characteristics and natural history of patients with acute myocardial infarction sent home from the emergency room. Am J Cardiol. 1987;60(4):219-224. [DOI] [PubMed] [Google Scholar]
- 8.Wilson M, Welch J, Schuur J, O’Laughlin K, Cutler D. Hospital and emergency department factors associated with variations in missed diagnosis and costs for patients age 65 years and older with acute myocardial infarction who present to emergency departments. Acad Emerg Med. 2014;21(10):1101-1108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Schull MJ, Vermeulen MJ, Stukel TA. The risk of missed diagnosis of acute myocardial infarction associated with emergency department volume. Ann Emerg Med. 2006;48(6):647-655. [DOI] [PubMed] [Google Scholar]
- 10.Chua M, Ibrahim I, Neo X, Sorokin V, Shen L, Ooi SB. Acute aortic dissection in the ED: risk factors and predictors for missed diagnosis. Am J Emerg Med. 2012;30(8):1622-1626. [DOI] [PubMed] [Google Scholar]
- 11.Vermeulen MJ, Schull MJ. Missed diagnosis of subarachnoid hemorrhage in the emergency department. Stroke. 2007;38(4):1216-1221. [DOI] [PubMed] [Google Scholar]
- 12.Moy E, Barrett M, Coffey R, Hines AL, Newman-Toker DE. Missed diagnoses of acute myocardial infarction in the emergency department: variation by patient and facility characteristics. Diagnosis (Berl). 2014;2(1):29-40. Accessed January 19, 2018. doi: 10.1515/dx-2014-0053 [DOI] [PubMed] [Google Scholar]
- 13.Solomon HA, Edwards AL, Killip T. Prodromatan acute myocardial infarction. Circulation. 1969;40(4):463-471. [DOI] [PubMed] [Google Scholar]
- 14.Kassell NF, Kongable GL, Torner JC, Adams HP Jr, Mazuz H. Delay in referral of patients with ruptured aneurysms to neurosurgical attention. Stroke. 1985;16(4):587-590. [DOI] [PubMed] [Google Scholar]
- 15.Adams HP Jr, Jergenson DD, Kassell NF, Sahs AL. Pitfalls in the recognition of subarachnoid hemorrhage. JAMA. 1980;244(8):794-796. [PubMed] [Google Scholar]
- 16.Newman-Toker DE, Moy E, Valente E, Coffey R, Hines AL. Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample. Diagnosis (Berl). 2014;1(2):155-166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kowalski RG, Claassen J, Kreiter KT, et al. . Initial misdiagnosis and outcome after subarachnoid hemorrhage. JAMA. 2004;291(7):866-869. [DOI] [PubMed] [Google Scholar]
- 18.Nicholson J. State faults hosps for “Rent” tragedy. New York Daily News December 13, 1996. http://www.nydailynews.com/archives/news/state-faults-hosps-rent-tragedy-article-1.743185. Accessed January 23, 2018.
- 19.Obermeyer Z, Cohn B, Wilson M, Jena AB, Cutler DM. Early death after discharge from emergency departments: analysis of national US insurance claims data. BMJ. 2017;356:j239. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what’s the goal? Acad Med. 2002;77(10):981-992. [DOI] [PubMed] [Google Scholar]
- 21.Newman-Toker DE, McDonald KM, Meltzer DO. How much diagnostic safety can we afford, and how should we decide? A health economics perspective. BMJ Qual Saf. 2013;22(suppl 2):ii11-ii20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Heffernan R, Mostashari F, Das D, Karpati A, Kulldorff M, Weiss D. Syndromic surveillance in public health practice, New York City. Emerg Infect Dis. 2004;10(5):858-864. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable 1. ICD-9-CM Codes Used to Define the Index Hospital Admission
eTable 2. Effect of Excluding Recent Hospitalizations
eTable 3. Effect of Excluding Discharges After Observation Admissions
eTable 4. Effect of Varying Duration of “Pre-admission Window”
eTable 5. Discharge Diagnoses Associated With Unrecognized Emergencies
eFigure. Distribution of ED Visits