Cerebrovascular Disease Hospitalizations following Emergency Department Headache Visits: A Nested Case-Control Study

Abstract

Background

Delayed diagnosis of cerebrovascular disease (CVD) among patients can result in substantial harm. If diagnostic process failures can be identified at Emergency Department (ED) visits that precede CVD hospitalization, interventions to improve diagnostic accuracy can be developed.

Methods

We conducted a nested case-control study using a cohort of adult ED patients discharged from a single medical center with a benign headache diagnosis from 10/1/2015-3/31/2018. Hospitalizations for CVD within 1 year of index ED visit were identified using a regional health information exchange. Patients with subsequent CVD hospitalization (cases) were individually matched to patients without subsequent hospitalization (controls) using patient’s age and visit date. Demographic, clinical, and ED processes characteristics were assessed via detailed chart review. McNemar’s test for categorical and paired t-test for continuous variables were used with statistical significance set at ≤0.05.

Results

Of the 9,157 patients with ED headache visits, 57 (0.6%; 95% CI:0.5-0.8) had a subsequent CVD hospitalization. Median time from ED visit to hospitalization was 107 days. In twenty-five patients (43.9%; 25/57) the CVD hospitalization and the index ED visit were at different hospitals. Fifty-three cases and 53 matched controls were included in the final study analysis. Cases and controls had similar baseline demographic and headache characteristics. Cases more often had a history of stroke (32.1% vs. 13.2%, p=0.02) and neurosurgery (13.2% vs. 1.9%, p=0.03) prior to the index ED visit. Cases more often had <2 components of the neurological exam documented (30.2% vs. 11.3%, p=0.03).

Conclusion

We found that 0.6% of patients with an ED headache visit had subsequent CVD hospitalization, often at another medical center. ED visits for headache complaints among patients with prior stroke or neurosurgical procedures may be important opportunities for CVD prevention. Documented neurological examinations were poorer among cases, which may represent an opportunity for ED process improvement.

Introduction

Delayed diagnosis of cerebrovascular disease (CVD) including acute ischemic stroke, intracerebral hemorrhage, transient ischemic attack (TIA), and subarachnoid hemorrhage (SAH) can preclude time sensitive treatments resulting in significant patient harms.^1–4 Initial misdiagnosis in the emergency department (ED) occurs in approximately 9% of stroke patients;⁵ false-negative stroke patients who are discharged directly from the ED are often initially diagnosed with ‘benign’ neurological conditions, most frequently headache,⁶ whereas those that are admitted from the ED are often initially labeled as having altered mental status or acute confusional state.^1,7 Although the incidence of dangerous diseases among patients evaluated for headache complaints in the ED is low,⁸ various types of CVD are associated with headache.^9–11 Non-trivial rates of diagnostic error among patients discharged from the ED after headache diagnosis have been reported.^6,12,13 Understanding sources of diagnostic error among ED patients with headache complaints is an important step towards reducing misdiagnosis related harms. To date, only the presence of vascular risk factors and having a head computed tomography performed at index ED visit have been associated with increased risk of subsequent cerebrovascular events among ED treat-and-release headache patients.^14,15

In this paper, we first use administrative claims data from a regional health information exchange to measure the rate of potential CVD misdiagnosis among ED treat-and-release headache patients using Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) methods.¹⁶ We then conduct a detailed review of the ED electronic medical record (EMR) for all included patients. We use this nested case-control design to evaluate for differences in demographic, clinical, diagnostic, and radiographic features between headache patients with (cases) versus without (controls) a subsequent CVD hospitalization. We hypothesized that evidence of diagnostic process failures would be more frequent in cases.

Methods

Study Design and Setting

We first identified a cohort of patients discharged home with a ‘benign’ headache diagnosis from any one of the four main EDs of John Hopkins Medicine (JHM) in the State of Maryland from 10/1/2015, when International Classification of Disease 10 Clinical Medicine (ICD-10-CM) codes were adopted in the US, until 3/31/2018. Of the four JHM EDs, one is a joint commission certified comprehensive stroke center and two were joint commission certified primary stroke centers during the study period.

We used ICD-10-CM primary discharge diagnosis codes for tension headache, migraine, cluster headache, cephalgia, or headache not otherwise specified to identify our study cohort of ‘benign’ headache patients (Supplement Table 1).^14,15 Only index ED visits (first eligible visit per patient during the study period) that resulted in a discharge to home (ED treat-and-release visit) for patients age >18 years of age were included.

We defined all patients with a hospitalization for CVD within 365 days of their index JHM ED headache visit as potentially misdiagnosed cases; patients without a CVD hospitalization within 365 days of index ED visit were defined as controls. A research team member (AH), with expertise using large datasets, then matched all cases to controls in a one-to-one exact fashion with random selection of matched controls when there was more than one exact match. We used patients’ age at ED arrival as well as year and month of index ED visit to create case-control pairs.

The JHM Internal Review Board (IRB) approved this project and granted a waiver for patient consent. Access to the Chesapeake Regional Information System for our Patients (CRISP) hospitalization records had been granted to Johns Hopkins by the State of Maryland/Department of Health/ Health Service Cost Review Commission for quality improvement purposes, and this retrospective re-use of these data was approved by the JHM IRB as research.

Outcomes

To identify subsequent inpatient hospitalizations for CVD in our cohort, we used the CRISP regional health information exchange. CRISP contains data from 48 regional hospitals, including those that are part of JHM. Hospitals are located in the State of Maryland as well as adjacent areas in the State of Pennsylvania, Delaware, Northern Virginia, and Washington DC. CRISP’s data warehouse imports records from Epic, Cerner and Meditech electronic health record systems in these hospitals.¹⁷ We defined hospitalizations for CVD as those with an ICD-10-CM primary discharge diagnosis codes for ischemic stroke, TIA, intracerebral hemorrhage, and subarachnoid hemorrhage (Supplemental Table 2).¹⁸ We did not include outpatient or ED visits for CVD in our outcome measure.

Variables

Once case-control pairs were created using administrative claims data, we conducted a detailed review of the index JHM ED headache visit using the EMR for each included case and their matched control. We abstracted pre-specified demographic, clinical, and radiographic features. Record review was performed by a single board-certified vascular neurologist (ALL) who was completely masked to case-control assignment.

To fully describe all included headache patients, we abstracted documentation of red or orange flag clinical features included in the SNOOP10 list¹⁹ as well as additional high-risk features included in the Ottawa subarachnoid hemorrhage rule.²⁰ This approach is consistent with the recommendations of the American College of Emergency Medicine Physicians guidelines.²¹

To evaluate diagnostic processes at the index ED visit, we abstracted information regarding provider’s documented differential diagnosis (including any mention of CVD), number of providers seeing the patient, medication response, documented neurological examination, and whether or not the patient was seen by an Emergency Medicine trainee. We used a modified version of the Safer Stroke-Dx Instrument to evaluate for missed diagnostic opportunities at the index ED headache visit (Supplemental Table 3); the board certified vascular neurologist who developed this tool applied it in this study. The Safer Stroke-Dx Instrument is a 16 item medical record review tool which includes the original 13 statements from the Revised Safer Dx Instrument and 3 additional statements focused on neurological disease;^22–24 in a recent study of hospitalized patients with possible stroke misdiagnosis in the ED we found that the inter-rater reliability of Safer Stroke-Dx Instrument was high (κ=0.9 [SE=0.04]).²⁵ For this study, we modified the Safer Stroke-Dx Instrument to a 12 item measure by eliminating the items that require knowledge of a “final diagnosis.” Modification of the Safer Stroke-Dx instrument was necessary since all chart review in this study was done by a reviewer masked to whether or not a subsequent CVD hospitalization occurred (Supplemental Table 3).

To identify instances of potential missed TIA diagnosis at index ED headache visit, a board certified vascular neurologist (ALL) applied the recently validated modified version of the explicit diagnostic criteria for TIA (EDCT).²⁶ This tool consists of 5 criteria and was originally designed to differentiate migraine with aura patients from those with TIA.²⁷ The modified EDCT has a sensitivity of 98% and specificity of 74% for the diagnosis of TIA.²⁶

Medical Record Adequacy

Prior work using ED records to evaluate diagnostic accuracy in patients who complained of dizziness found poor documentation regarding essential elements of the patients’ chief complaints²⁸ as well as their neurological exam.²⁹ We therefore specified that if an ED record did not contain any mention of even one of the following: (1) headache onset characteristics (i.e., thunderclap, progressive), (2) headache pain quality, (3) headache triggers, or (4) associated headache features documented, then the record was deemed to be missing essential chief complaint information. We also categorized ED records as missing essential examination data if fewer than two components of a headache-focused neurological examination were not documented by any ED provider. We considered a headache-focused neurological examination to consist of 12 components: (1) fundoscopic evaluation (2) meningeal signs (3) mental status (4) otoscopic exam (5) pupils (6) visual fields (7) ocular motility (8) facial asymmetry (9) sensation (10) limb strength (11) coordination and (12) gait evaluation. Whether or not each component of the neurological examination was mentioned in the ED record was abstracted in addition to whether or not the findings were normal or abnormal.

We considered “inadequate” ED records to be ones where either essential chief complaint elements were missing or less than two components of the 12 component neurological examination were recorded. We included all eligible patients, regardless of medical record adequacy, in our primary study analysis.

Additional analysis

We conducted a pre-specified additional analysis comparing cases hospitalized for CVD within 30 days of index ED visit versus those that were hospitalized >30 days of their index ED visit to evaluate for factors associated with short-term CVD hospitalization. Using a 30-day time frame is in keeping with the known temporal patterns of cerebrovascular events after TIA/minor stroke where highest risk is in the days following the index event³⁰ and this time frame has been previously used to identify potential misdiagnosis at index ED visit.^{6, 12}

Based on the comments of an anonymous reviewer, we also report the number of patients with at least one interval ED headache treat-and-release visit to a JHM facility in the 365 days following their index ED visit and compare interval visits between cases and controls.

To evaluate the reliability of variables abstracted from the EMR, we randomly selected 12 patient charts (6 cases and 6 controls) and asked a second reviewer blinded to all other study data to abstract from the ED record: (1) history of stroke, (2) sudden onset of fully reversible neurological or retinal symptoms, the first component of EDCT, and (3) whether <2 components of the neurological exam were documented.

Statistical analysis

Standard descriptive statistics were used to characterize the study population, including means with standard deviations (SD) for normally distributed continuous variables and medians with interquartile range (IQR) for non-normally distributed ones. We used the modified Wald method to calculate 95% confidence intervals (CI) for proportions. We compared demographic, clinical, and radiographic features between case-control pairs using McNemar’s test for categorical variables and paired t-test for continuous variables. For our additional comparative analysis between cases who returned with CVD within 30 days of index ED visits and all other cases, we used chi-squared and Student’s t-test statistics. To compare the number of interval ED headache visits between cases and controls, we also used chi-squared. Inter-rater reliability for EMR data abstraction was assessed using percent agreement and Cohen’s kappa coefficient (κ). We did not impute any values for missing data. To depict rates of return with cerebrovascular disease, we used a histogram. The threshold for statistical significance was set as ≤0.05. All tests of comparison were two-sided. Analyses were performed using Stata/IC, version 16 (StataCorp, TX).

Results

We identified a total of 9,157 ED headache visits at JHM during the study period. The average age was 42 (SD 16.5) years and 67.5% were women. A total of 62 patients (0.7%; 95% CI:0.5-0.9) were initially identified as having an ED visit for headache followed by a CVD hospitalization. These 62 headache patients (cases) were matched to 62 headache patients who did not have a subsequent CVD hospitalization (controls). Detailed review of all initially identified cases and controls (n=124) revealed that 12 patients (5 cases and 7 controls) did not have a true ED treat-and-release visit and were therefore excluded (Figure 1).

Figure 1.

Study Flow

There were thus 57 (0.6%; 95% CI:0.5-0.8) patients with a confirmed ED treat-and-release visit from a JHM ED with a subsequent cerebrovascular disease hospitalization. A total of 25 cases (43.9%; 25/57) were hospitalized at a non-JHM hospital (health-system crossover). Median time from ED visit to CVD hospitalization was 107 days IQR (31-189).

A total of 106 patients (53 cases and 53 controls) were included in the final case-control analysis (Figure 1). Five patients in the original study cohort (4 cases and 1 control) were found on detailed EMR review to have left the ED against medical advice (AMA) or before being seen by a provider. After excluding these 5 patients, a total of 47 original case-control pairs remained. Six new case-control pairs were formed by matching the remaining unpaired cases to unpaired controls; three of these new case-control pairs were exactly matched for patient age, but three could not be (difference in age of 1, 16, and 17 years). A single control patient (n=1) originally paired to an excluded case was removed.

Primary Analysis

The included 53 cases-control pairs had similar baseline demographic and headache characteristics. Nearly two thirds of both groups had >1 red or orange flag headache feature at presentation. No thunderclap headaches were documented (Table 1). A total of 13 patients (24.5%, 13/53) were hospitalized within 30 days of their index ED visit (Figure 2). Among patients who were hospitalized within 30 days (n=13), the median time from index ED visit to hospitalization was 11 days (IQR: 6-13) among those who returned >30 days median time to hospitalization was 154 (IQR: 89-206) days. Only 2 patients were hospitalized within 72 hours of the index ED visit. Within 365 days of index ED visit, only 8 (15.1%) cases and 4 (7.5%) controls had at least one interval ED visit for headache (p= 0.22).

Table 1.

Baseline demographic and clinical features for cases and matched controls

	Cases (n=53)	Controls (n=53)	P-value
Demographics
Female Sex, n(%)	31 (58.5)	30 (56.6)	0.84
Race/Ethnicity, n(%)
- Non-Hispanic White	18 (34.0)	19 (35.8)
- Non-Hispanic Black	32 (60.4)	23 (43.4)
- Hispanic	0	0
- Asian	2(3.8)	9 (17.0)
- Other	1 (1.9)	2(3.8)
Translation services used	0	5(9.4)	0.08
John Hopkins ED
- A	14 (26.4)	12(22.6)
- B	8 (15.1)	17 32.1)
- C	23 43.4)	13 24.5)
- D	8 (15.1)	11 (20.8)
Past Medical History
Hypertension	38 (71.7)	29 (54.7)	0.07
Hyperlipidemia	20 (37.7)	12 (22.6)	0.17
Diabetes	15 (28.3)	12(22.6)	0.55
Prior stroke	17 (32.1)	7 (13.2)	0.02
Atrial fibrillation	5 (9.4)	4(7.5)	0.71
Heart failure	6 (11.3)	2 (3.8)	0.16
Smoker	20(37.7)	20 (37.7)	1.0
Alcohol use	4(7.5)	6 (11.3)	0.48
Human Immunodeficiency Virus	2 (3.8)	2(3.8)	1.0
Cancer	7(13.2)	2(3.8)	0.06
Prior Neurosurgery	7 (13.2)	1(1.9)	0.03
Recent trauma	4 (7.5)	4 (7.5)	1.0
Psychiatric illness	14 (26.4)	14 (26.4)	1.0
Drug use	4 (7.5)	2 (3.8)	0.41
Pregnant or post-partum	0	0
Home medications
Any antiplatelet agent*	21 (39.6)	10 (22.6)	0.02
Anticoagulation	4 (7.5)	5 (9.4)	1.0
Anti-hypertensive	32 (60.4)	23 (43.4)	0.09
Statin	25 (47.2)	11 (20.8)	0.01
Headache features
Presenting complaint in triage did not include headache**	15 (28.3)	9 (17.0)	0.16
Any prior headache disorder	20 (37.7)	20 (37.7)	1.0
Similar to prior headache	7 (13.2)	15 (28.3)	0.07
Thunderclap Quality	0	0
Duration < 6 hours	12 (22.6)	7 (13.2)	0.23
Duration >24 hours	28 (52.8)	29 (54.7)	0.86
Associated neurological complaints (including blurry vision and dizziness)	25 (47.2)	22 (41.5)	0.49
SNOOP10 >1	34 (64.2)	33 (62.2)	0.82
Improvement in headache pain while in the ED	15 (28.3)	20 (37.7)	0.34
Vitals at ED arrival
Temp > 100.4°F or < 96.8°F	0	1 (1.9)	0.32
Mean systolic blood pressure (SD)	153.5 (3.4) mm Hg	145.4 (3.9) mm Hg	0.09
Mean diastolic blood pressure (SD)	85.3 (2.0) mm Hg	82.1 (1.8) mm Hg	0.21
Heart rate (SD)	81.5 (2.1)	77.5 (2.1)	0.19

^*Two cases were prescribed dual antiplatelet therapy

^**No triage complaint was documented for two patients

Figure 2.

Histogram showing hospitalizations for cerebrovascular disease following index ED headache visit among all cases (n=53) (width of each bar = 30 days)

Table 2 shows the comparison of cases versus controls. Cases more often had a prior history of stroke (32.1% vs. 13.2%, p=0.02) and of a neurosurgical procedure (13.2% vs. 1.9%, p=0.03) compared to controls. Similarly, cases more often had radiographic evidence of a chronic infarct on imaging as compared to controls (28.3% vs. 9.4%, p=0.02) and were more likely to have been prescribed statins (47.2% vs. 20.8%; p=0.01) or antiplatelet agents (39.6% vs. 22.6%; p=0.02) than controls. Having a neurological consultation (18.9% vs. 7.5%, p=0.06) and being seen by multiple providers (11.3% vs. 3.8%, p=0.06) was also more frequent among cases than controls, but with borderline p-values.

Table 2.

Adequacy of record documentation and diagnostic process failures (other than neurologic exam) among cases versus controls

	Cases (n=53)	Controls (n=53)	P-value
ED record documentation quality
Essential headache-related information not documented	11(20.8)	7 (13.2)	0.32
Inadequate record documentation	22 (41.5)	13 (24.5)	0.08
Any differential diagnosis documented	35 (66.0)	36 (67.9)	0.83
Differential diagnosis by any provider included mention of cerebrovascular disease	25 (47.2)	28 (52.8)	0.53
Providers and consultations
Seen by more than two providers	6 (11.3)	2 (3.8)	0.06
Stroke code	1 (1.9)	0	0.32
Neurology consult	10 (18.9)	4 (7.5)	0.06
Other consult	6 (11.3)	3 (5.7)	0.16
Emergency Medicine trainee	20 (37.7)	13 (24.5)	0.09
Diagnostic testing done in the ED
Head CT	36 (67.9)	42 (79.2)	0.10
MRI brain	3 (5.7)	2(3.8)	0.32
Vessel imaging	2(3.8)	7 (13.2)	0.10
Lumbar puncture	2(3.8)	2(3.8)	1.0
Imaging results at index ED visit
Acute/subacute infarct	2(3.8)	0	0.16
Chronic infarct	15(28.3)	5(9.4)	0.02
Markers of misdiagnosis
Missed diagnostic opportunity using modified Safer Stroke-Dx	8 (15.1)	5 (9.4)	0.32
Sudden onset of reversible neurological symptoms	4 (7.5)	1 (1.9)	0.18

Using the modified Safer Stroke-Dx instrument, 13 instances of missed diagnostic opportunities were identified. A total of 8 cases and 5 controls were identified as having missed diagnostic opportunities (P=0.32; Table 2). In one of the 8 cases an acute infarct was seen on MR imaging and in another case a subacute infarct was seen; neither finding was clearly addressed by the ED care team despite the radiologists’ read of stroke (Supplemental Table 4).

No definitive missed TIA cases were identified at index ED visit using the modified EDCT. A total of 5 patients (5/106) had documentation of sudden onset of fully reversible neurological or retinal symptoms. Of those five patients, 4 were cases and 1 was a control (Table 2). Among the cases, one patient had symptoms of right sided numbness lasting for > 24 hours and was diagnosed with non-specific numbness after neurological consultation and an unremarkable MRI brain. The other 3 cases did not have enough information in the medical record to determine whether or not they satisfied the other EDCT criteria. The single control patient with documentation of sudden onset fully reversible neurological symptoms was unable to provide a detailed history or definite symptoms onset time due to delirium.

Abnormal neurological findings were reported in only 19% of included patients with no differences in the presence of abnormalities between cases and controls (Table 3). Documentation of individual components of neurological exam in the medical record by ED providers was similar between cases and controls except for facial asymmetry.

Table 3.

Headache-specific neurologic exam findings documented by ED providers among cases versus controls

	Cases (n=53)	Controls (n=53)	P-value
Fundoscopic exam	1 (1.9)	0	0.32
Meningeal signs	11(20.8)	13 (24.5)	0.62
Mental status	50 (94.3)	51 (96.2)	0.56
Otoscope	4 (7.5)	2 (3.8)	0.41
Visual fields	27 (50.9)	32 (60.4)	0.28
Pupillary exam	31 (58.5)	36 (67.9)	0.32
Ocular motility	31 (58.5)	37 (69.8)	0.20
Facial asymmetry	32 (60.4)	44 (83.0)	0.01
Sensation	19 (35.8)	19 (35.8)	1.0
Strength	25 (47.2)	26 (49.1)	0.84
Coordination	22 (41.5)	21 (39.6)	0.85
Gait evaluation	11 (20.8)	13 (24.5)	0.67
<2 components of the neurological examination documented	16 (30.2)	6 (11.3)	0.03
Any abnormality noted on neurological exam	8 (15.1)	12 (22.6)	0.41
Any abnormality noted on general medicine exam	19 (35.8)	27 (50.9)	0.84

Medical Record Completeness

Cases were significantly more likely than controls to have < 2 components of the neurological exam documented in the EMR (30.2% vs. 11.3%, p=0.03). A total of 18 encounters (11 cases and 7 controls) lacked enough information regarding headache chief complaint without significant differences between cases and controls (20.8% vs. 13.2%; p=0.32). Inadequate records, lacking either essential headache chief complaint information or with <2 components of the neurological examination were more common among cases than controls with a large effect size (41.5% vs. 24.5%; P=0.08) that did not reach statistical significance.

Inter-rater reliability

There was 91.7% agreement on chart abstraction (by two independent raters masked to case versus control status) of history of stroke (κ=0.8), 91.7% agreement on whether or not <2 components of the neurological exam was missing (κ=0.7), and 83.3% on whether the patient had sudden onset of reversible neurological symptoms (κ=0.6) between the two raters.

Additional Analysis

Differences in diagnostic process markers between cases with return visit within 30 days (n=13) versus all other cases (n=40) were minimal (Table 4). Inadequate records lacking essential chief complaint or neurological exam documentation were more common in cases with CVD hospitalization >30 days from index event than those with CVD hospitalizations ≤ 30 (50% vs. 15.4%; p=0.03).

Table 4.

Comparison of cases with stroke occurring ≤30 versus >30 days of the index ED visit

	Case within 30 days (n=13)	Other cases (n=40)	P-value
Headache not in triage complaint	3 (23.1)	12 (30)	0.70
Health-system Crossover	5 (38.5)	8 (20)	0.57
Missed diagnostic opportunity using modified Safer Stroke-Dx	1 (7.7)	7 (17.5)	0.36
Differential diagnosis documented	7 (53.8)	28 (70)	0.29
Stroke on differential diagnosis	4 (30.8)	21 (52.5)	0.43
SNOOP > 1	7 (53.8)	27 (67.5)	0.37
Essential headache-related information not documented	1 (7.7)	10 (25)	0.18
Inadequate record documentation	2(15.4)	20 (50)	0.03
Seen by an Emergency Medicine trainee	6 (46.2)	14 (35)	0.47
Acute/subacute infarct	0	2 (5)	0.43
Neuro consult	2 (15.4)	8 (20)	0.84
Neurological Examination
Fundoscopic exam	0	1 (2.5)	0.57
Meningeal signs	2 (15.4)	9 (22.5)	0.56
Mental status	13 (100)	37 (92.5)	0.31
Otoscopy	2 (15.4)	2 (5)	0.22
Visual fields	8 (61.5)	19 (47.5)	0.38
Pupils	10 (76.9)	21 (52.5)	0.12
Ocular motility	10 (76.9)	21 (52.5)	0.12
Facial droop	9 (69.2)	23 (57.5)	0.45
Limb sensory	4 (30.8)	15 (37.5)	0.66
Motor strength	7 (53.8)	18 (45)	0.58
Coordination	8 (32)	14 (35)	0.09
Ambulates	3 (23.1)	8 (20)	0.81
< 2 components of the neurological exam documented	2 (15.4)	14 (35)	0.18

Discussion

Using administrative claims data, we found that 0.6% of patients with an ED treat-and-release headache visit at JHM had a subsequent hospitalization for CVD within 1 year. After matching patients with versus without a subsequent hospitalization in a nested case-control design, detailed medical record review revealed that cases more often had a history and radiographic evidence of prior stroke as well as a history of neurosurgery. We found that documentation of key history and exam features was frequently inadequate, and charts with minimal neurologic exams documented at the ED index treat-and-release visit were more frequent among headache patients with subsequent CVD hospitalization (cases). Using a modified version of the Safer Stroke-Dx Instrument we found 15% of cases had missed diagnostic opportunities, but there was no clear difference between cases and controls. There were two headache patients with a missed stroke on imaging at index ED visit that were not addressed prior to discharge.

The yield of detailed medical record review to identify diagnostic process failures was constrained by missing elements. Nearly half (45.6%; 26/57) of all included patients with a subsequent cerebrovascular disease hospitalization in a SPADE-style analysis had an index ED encounter that could not be fully evaluated either because of inadequate documentation (n=22) or leaving AMA before being seen by a provider (n=4). We found that cases were more likely than controls to have ED records lacking critical information regarding chief complaint or have no more than one component of the 12-item neurological exam documented (inadequate records).

Others have noted a lack of information in the ED medical record for treat-and-release visit patients. Among patients who present with dizziness, a frequent presenting complaint in stroke misdiagnosis,⁵ a single center study found that 50% of ED patients lacked documentation of relevant diagnostic information.²⁸ Another study found that the documentation of exam findings regarding nystagmus enabled a meaningful inference about the cause of dizziness in only 5.4% of ED visits.²⁹ Finally, when reviewing ED visits that occur within 7 days of another ED visit as a marker of diagnostic error, researchers found that 20% of all ED encounters contained no usable information in the medical record.³¹ Whether or not missing ED documentation can serve as a marker or trigger tool for potential misdiagnosis requires further study.³² Based on our findings, relying on ED chart review alone to identify sources of possible diagnostic error is low-yield, but there appears to be substantial room for improvement in documentation (and perhaps performance of) bedside neurologic assessments.

Our rate of CVD hospitalization after ED treat-and-release headache visits of 0.6% is similar to previously published rates. A prior study using administrative claims data from 6 US states found that the rate of hospitalization for any dangerous neurological disease after an ED headache visit was 0.5%.¹⁰ A single center study reported that 0.4% of headache patients had a subsequent CVD hospitalization within a year of index ED visit.¹⁵

A major advantage of our study compared to prior work is that we were able to capture outcome events at unaffiliated centers using a health information exchange as well as assure all ED encounters were eligible for study inclusion via detailed chart review. Our finding that 44% of included ED treat-and-release headache patients were subsequently hospitalized at an unaffiliated center is consistent with previous research. In a Canadian study of missed SAH in the ED, a 37% health-system crossover rate was reported.¹³ These findings highlight the importance of using multi-institutional data sets (e.g., regional health information exchanges, administrative claims data) when evaluating for potential ED misdiagnosis. At the very least, single institutions without such data must be aware that their true rate of CVD hospitalizations after ED visits for neurological symptoms is likely to be meaningfully higher than the one measured using only internal data.

Another important finding from our study is that prior stroke and evidence of chronic infarct on imaging during an ED headache treat-and-release visit was associated with subsequent CVD hospitalization. This finding is in keeping with a previous case-control study of ED headache patients with subsequent stroke hospitalizations where only history of stroke or TIA was associated with increased odds of stroke hospitalization in a multivariable model.¹⁴ In combination, these results suggest that ED headache visits for patients with a history of or radiographic evidence of prior stroke or neurosurgery are important opportunities to assure evidence based secondary stroke prevention strategies are in place. Whether index ED headache visits in this subgroup of patients represent a biological trigger for subsequent stroke or reflects diagnostic error at the index ED visit remains somewhat uncertain based on available evidence.

Limitations

Our current study has a number of important limitations. To begin with, the results are based on retrospective EMR abstraction, so incorrect and potentially biased ED documentation of clinical features may have contributed to the findings, with unknown direction and magnitude of impact. Additionally, all evaluated ED visits occurred within a single academic health-system which may limit the generalizability of our results. Secondly, single reviewer evaluated all included encounters and, even though this reviewer was fully blinded to outcome events and applied previously established tools to evaluate the diagnostic processes at index visit, inter-rater reliability of data abstraction for the EMR was only partially assessed. The Safer Stroke-Dx tool was modified to account for the reviewer’s lack of knowledge regarding final diagnosis. Thirdly, outcome ascertainment for subsequent CVD was imperfect as it relied on subsequent hospitalization in the region, identified outcome events using ICD-10-CM codes, and could not identify patients who were never hospitalized (i.e., those who suffered cerebrovascular events at home and did not seek care, who presented to clinic and were never admitted, or who died). We also do not know the clinical severity of any CVD hospitalization. Fourthly, we did not use patient sex to match cases to controls because we wanted to study sex as a predictor. Given known sex differences in both ED diagnostic formulation following minor neurological events³³ and migraine prevalence,³⁴ our study design could have biased our results, again, with unknown magnitude and direction. Finally, we matched cases to controls before detailed medical record review requiring us to re-match a few patients, resulting in 3 case-control pairs that were not as well matched as all other pairs.

Conclusions

We found that 0.6% of patients with an ED headache treat-and-release visit returned with CVD in the year following their index ED visit using SPADE methods. On detailed EMR review, roughly one third of all included ED records lacked documentation of either essential headache specific history or contained only a single component of the neurologic exam. Furthermore, charts with less than two components of the neurological exam were more common among cases than controls, so this appears to be an important area for diagnostic process improvement. Headache patients with a subsequent CVD hospitalization were more likely to have had a prior stroke or prior neurosurgery, so index ED visits for these patients may represent important opportunities to assure all evidence based stroke prevention strategies are in place. Finally, quality improvement and research efforts seeking to measure misdiagnosis rates should account for health-system crossover and those assessing diagnostic care process or causes of diagnostic error should consider strategies beyond chart review.