Abstract
Introduction
Application of a clinical decision rule for subarachnoid hemorrhage, in combination with cranial computed tomography (CT) performed within six hours of ictus (early cranial CT), may be able to reasonably exclude a diagnosis of aneurysmal subarachnoid hemorrhage (aSAH). This study’s objective was to examine the sensitivity of both early cranial CT and a previously validated clinical decision rule among emergency department (ED) patients with aSAH and a normal mental status.
Methods
Patients were evaluated in the 21 EDs of an integrated health delivery system between January 2007 and June 2013. We identified by chart review a retrospective cohort of patients diagnosed with aSAH in the setting of a normal mental status and performance of early cranial CT. Variables comprising the SAH clinical decision rule (age ≥40, presence of neck pain or stiffness, headache onset with exertion, loss of consciousness at headache onset) were abstracted from the chart and assessed for inter-rater reliability.
Results
One hundred fifty-five patients with aSAH met study inclusion criteria. The sensitivity of early cranial CT was 95.5% (95% CI [90.9–98.2]). The sensitivity of the SAH clinical decision rule was also 95.5% (95% CI [90.9–98.2]). Since all false negative cases for each diagnostic modality were mutually independent, the combined use of both early cranial CT and the clinical decision rule improved sensitivity to 100% (95% CI [97.6–100.0]).
Conclusion
Neither early cranial CT nor the SAH clinical decision rule demonstrated ideal sensitivity for aSAH in this retrospective cohort. However, the combination of both strategies might optimize sensitivity for this life-threatening disease.
INTRODUCTION
Background
Approximately 80% of non-traumatic cases of subarachnoid hemorrhage are attributable to ruptured cerebral aneurysms, for which delays in definitive aneurysm treatment can increase the risk of disability or death. 1, 2 While the vast majority of aneurysmal subarachnoid hemorrhage (aSAH) cases are identified by cranial computed tomography (CT), the sensitivity of CT diminishes with time such that lumbar puncture is recommended as the definitive test to exclude a diagnosis of “CT-negative” SAH. While two large studies have reported that early cranial CT (i.e. performed within six hours of headache onset) may be up to 100% sensitive for SAH among patients presenting with a normal mental status, 3–6 a prior study by our research group demonstrated imperfect sensitivity of this definition of early cranial CT in the non-academic emergency department (ED) setting. 7 Accordingly, we proposed that sequential application of a validated SAH clinical decision rule (absence of all the following: age ≥40, neck pain or stiffness, headache onset with exertion, loss of consciousness at headache onset) in such a clinical scenario might further reduce the posterior probability of CT-negative SAH to an acceptable level of risk. 8–10
Goals of this investigation
We sought to further examine the potential incremental gain in sensitivity when applying a previously validated SAH clinical decision rule to a cohort of patients diagnosed with aSAH after presenting to the ED with normal mental status and undergoing early cranial CT.
METHODS
Study Population
We screened electronic health records of patients treated within the Kaiser Permanente Northern California (KPNC) integrated healthcare delivery system between January 2007 and June 2013 for case inclusion if they had an ED or hospital encounter with an associated International Statistical Classification of Diseases and Related Health Problems, ninth edition (ICD-9) diagnosis code of SAH (430). Emergency care within KPNC is provided through 21 non-academic medical center-based EDs, serving a population of over 3.3 million Kaiser Foundation Health Plan (KFHP) members. This study was part of a larger project examining outcomes following misdiagnosis of aSAH. Patients were electronically excluded if they had an ICD-9 coded diagnosis of head or neck trauma within 24 hours of the index encounter, lacked continuous KFHP membership within the two weeks preceding diagnosis, were under 18 years of age or had a prior diagnosis of SAH between 2002 and 2006. Data on age, sex and race were electronically collected. We then manually reviewed charts for the following inclusion criteria: initial diagnosis at a KPNC ED, Hunt-Hess clinical grade of 1 or 2 at the time of ED presentation, non-contrast cranial CT imaging within six hours of headache onset, either evidence of SAH on non-contrast cranial CT or greater than five red blood cells per microliter on cerebrospinal fluid analysis, and angiographic evidence of cerebral aneurysm thought to be consistent with the clinical presentation and pattern of hemorrhage visualized on imaging, if applicable. The study was approved by the Kaiser Foundation Research Institute Institutional Review Board with a waiver of the requirement for informed consent.
Methods and Measurements
Two investigators (DGM and MVK) conducted a structured explicit chart review and abstraction of records using a standardized paper form as part of a larger study examining outcomes following misdiagnosis of aSAH. 11 Abstractors confirmed the inclusion criteria and the final radiologist interpretation of the initial cranial CT, the location and size of the culprit aneurysm and documentation of the presence or absence of the following variables: neck pain or stiffness, loss of consciousness, physical exertion at the time of headache onset, need for external ventricular drainage and treatment of vasospasm during hospitalization. A best modified Rankin Scale (mRS) score at one year was assigned by reviewing neurosurgical, rehabilitation services and primary care clinical notes following hospital discharge, if applicable, using previously validated methodology. 12 We considered a mRS score ≤2 a favorable neurologic outcome. Both abstractors reviewed 20% of the sample to establish the inter-rater reliability of the following variables with an estimated error margin less than 20%: early cranial CT (inclusion criteria), Hunt-Hess grade at ED presentation and the clinical decision rule. 13 All CT examinations were performed without contrast using multi-slice cine technology (16 slice or higher). Either general radiologists or neuroradiologists made the final interpretation of CT images.
Outcomes and Analysis
The primary outcome of interest was the combined sensitivity of early cranial CT and the SAH clinical decision rule (a negative result for the latter being defined as absence of all four clinical criteria). Secondary outcomes were the independent sensitivities of early cranial CT and the SAH clinical decision rule. Missing variables from the SAH clinical decision rule were imputed as being absent to provide the most conservative estimate of sensitivity. We calculated binomial confidence intervals (CI) using the Clopper-Pearson (exact) method. All statistical analyses were performed using STATA v 13.0 (College Station, TX).
RESULTS
Characteristics of study subjects
We identified 155 patients following application of exclusion and inclusion criteria (Figure). The median age was 55 years and 79% were female. Hunt-Hess grade was 2 in 95% of patients, though none of these had notation of a cranial nerve deficit upon initial ED evaluation. The most common aneurysm location was the anterior communicating artery (30%), followed by the posterior communicating artery (21%). Eighty percent of patients had a favorable neurologic outcome one year from initial hospitalization. Summary statistics of the study population are provided in Table 1.
Figure.
Aneurysmal subarachnoid hemorrhage cohort assembly. CT, computed tomography; ED, emergency department; ICD-9, International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM); KFHP, Kaiser Foundation Health Plan; KPNC, Kaiser Permanente Northern California; SAH, subarachnoid hemorrhage
Table 1.
Patient characteristics and outcomes (n=155).
| Variable | Value |
|---|---|
| Age (median, years) | 55 |
| Female (%) | 79 |
| Race (%) | |
| Caucasian | 42 |
| Black | 17 |
| Asian | 23 |
| Hispanic | 2 |
| Unknown/other | 16 |
| Hunt-Hess grade (%) | |
| 1 | 5 |
| 2 | 95 |
| Neck pain (%) | |
| Yes | 45 |
| Unknown | 10 |
| Loss of consciousness (%) | |
| Yes | 14 |
| Unknown | 1 |
| Headache onset with exertion (%) | |
| Yes | 15 |
| Unknown | 16 |
| Aneurysm location (%) | |
| ACOM | 30 |
| PCOM | 21 |
| MCA | 15 |
| ACA | 6 |
| ICA | 8 |
| PICA | 5 |
| Basilar | 5 |
| Other* | 7 |
| Unknown | 3 |
| Inpatient treatments (%, n) | |
| Vasospasm requiring intervention | 21 (33/151) |
| Hydrocephalus requiring EVD | 26 (40/154) |
| Neurologic outcome by one year (%, n) | |
| Alive | 85 (132/155) |
| mRS ≤2 | 80 (122/152) |
ACOM, anterior communicating artery; PCOM, posterior communicating artery; MCA, middle cerebral artery; ACA, anterior cerebral artery; ICA, internal carotid artery; PICA, posterior inferior cerebellar artery; EVD, external ventricular drainage; mRS, modified Rankin scale.
Other locations (n) included the vertebral artery (5), superior cerebellar artery (3), pericallosal artery (3), anterior choriodal artery (3), ophthalmic artery (2) and the superior hypophyseal artery (1).
Main results
Early cranial CT was reported as positive for SAH in 148 patients, yielding an estimated sensitivity of 95.5% (95% CI [90.9–98.2]). The SAH clinical decision rule was likewise positive in 148 patients with the same estimated sensitivity (95.5%, 95% CI [90.0–98.2]). Since the false negative cases for early cranial CT were mutually independent from the false negative cases by the SAH clinical decision rule, the combined estimated sensitivity for application of both early cranial CT and the SAH clinical decision rule was 100% (95% CI [97.6–100.0]). Seven patients (4.5%) underwent lumbar puncture, all of whom had negative early cranial CT interpretations. Pertinent details for false negative cases of early cranial CT and the SAH decision rule are provided in Tables 2 and 3, respectively.
Table 2.
Imaging and laboratory details for the seven false negative cranial computed tomography studies.
| Age | CT scanner | CT slice thickness | CSF RBCs/microliter** | Xanthochromia | Angiography results |
|---|---|---|---|---|---|
| ≥90 | unavailable | 5mm | 280000 | Yes | 5mm ACOM aneurysm |
| 76 | GE lightspeed VCT (64 slice) | 5mm | 517500 | No | 4mm right PCOM aneurysm |
| 67 | GE lightspeed VCT (64 slice) | 5mm | 408000 | No | 6mm left PCOM aneurysm |
| 45 | GE lightspeed VCT (64 slice) | 5 mm | 190000 | No | 4mm left ICA aneurysm |
| 53 | GE lightspeed Pro 16 (16 slice) | 5mm | 49750 | No | 2mm right PCOM aneurysm |
| 50 | GE lightspeed VCT (64 slice) | 1.25mm | 9960 | No | 10mm ACOM aneurysm |
| 70 | GE lightspeed VCT (64 slice) | 5mm | 55000 | Yes | 2mm right vertebral artery |
Seven patients presenting with aneurysmal SAH had cranial CT studies performed within six hours of headache onset that were initially reported as negative for evidence of subarachnoid hemorrhage. Diagnosis of SAH was made by lumbar puncture in each case. Details of the CT technology used as well as the results of diagnostic lumbar punctures and formal cerebral angiography are presented for each case.
CSF RBC counts were the lowest values reported in cases where multiple tubes were analyzed.
GE, General Electric; VCT, volume computed tomography; CT, computed tomography; ACOM, anterior communicating artery; CSF, cerebrospinal fluid; RBC, red blood cell; ICA, internal carotid artery; PCOM, posterior communicating artery; SAH, subarachnoid hemorrhage
Table 3.
Imaging and clinical details for the seven patients with false negative clinical decision rules.
| Age | Neck pain or stiffness | Onset with exertion | Loss of consciousness | CT results | Angiography findings |
|---|---|---|---|---|---|
| 32 | No | Unknown | No | Positive | 5mm right ACA aneurysm |
| 39 | No | No | No | Positive | 6mm left MCA aneurysm |
| 27 | No | No | No | Positive | 7mm left ACOM aneurysm |
| 32 | No | Unknown | No | Positive | 6mm right PCOM aneurysm |
| 39 | Unknown | No | No | Positive | ACOM aneurysm (unknown size/location) |
| 25 | No | No | No | Positive | 2mm right ICA aneurysm |
| 29 | Unknown | No | No | Positive | Right MCA aneurysm (unknown size) |
Seven patients presenting with aneurysmal SAH had false negative results using the clinical decision rule (age >40, presence of neck pain or stiffness, headache onset with exertion, loss of consciousness at headache onset) for subarachnoid hemorrhage. Diagnosis of SAH was made by computed tomography in each case. Details of the decision rule elements and formal cerebral angiography are presented for each case.
ACA, anterior cerebral artery; ACOM, anterior communicating artery; CT, computed tomography; ICA, internal carotid artery; MCA, middle cerebral artery; PCOM, posterior communicating artery; SAH, subarachnoid hemorrhage
The inter-rater agreement for electronic health record abstraction was 100% for early cranial CT, 87% for Hunt-Hess grade (1 versus 2) and 100% for a negative result on the overall SAH clinical decision rule (71% for neck pain or stiffness in isolation).
DISCUSSION
The goal of this study was to help clinicians further refine and understand current testing strategies for aSAH, specifically by highlighting the potential gain in sensitivity obtained with the “post-hoc” application of a SAH clinical decision rule following early cranial CT. While we recognize that this was not the original derivation or validation setting of the SAH clinical decision rule, we feel our study demonstrates that such an approach can potentially help inform shared decision-making between clinicians and patients when faced with uncertainty over the absolute sensitivity of early cranial CT, especially in the face of non-low pretest probability.
This testing strategy is similar in concept to performing a serum d-dimer assay to reliably exclude a lower extremity deep vein thrombosis in a patient with a high pre-test probability for disease but a negative lower extremity ultrasound examination; only the combined sensitivities of the two tests offer a low enough post-test probability to forgo further testing. 14 Additionally, although early cranial CT failed to detect aSAH in seven out of 155 cases (4.5%) in our retrospective cohort, this is only a point estimate and is specific to our practice setting. The findings of 100% sensitivity in the prospective Perry et al. 4 cohort and the retrospective Backes et al. 3 study rightfully prompt consideration of using early CT alone to rule out aSAH in those particular practice settings.
Important contextual differences between our study and prior reports involve both spectrum bias and radiologist staffing practices. Perry et al. 4 enrolled patients with acute headaches reaching maximal intensity within one hour. Backes et al. 3 retrospectively identified patients presenting to a SAH referral center with a 50% incidence of SAH among patients undergoing early cranial CT (as opposed to 13% in the Perry et al. study). Thus it is possible that our study cohort includes patients with less severe presentations of aSAH who may be less likely to manifest positive CT findings on early cranial CT. Likewise, radiology staffing at a tertiary neurosurgical referral center as in Backes et al. 3 is not representative of the vast majority of EDs. While the Perry et al. 4 study setting was similar to ours in that radiographic studies were interpreted by a mix of neuroradiologists and general radiologists who routinely interpreted cranial CTs, several of the medical centers participating in that study had active radiology residency training programs with over-reading of studies by faculty in the daytime, making it difficult to extrapolate that level of scrutiny to practicing radiologists in a non-academic hospital setting. Of note, one early cranial CT in that series was initially misinterpreted as being negative by a radiology trainee, and was only retrospectively re-interpreted as positive when a magnetic resonance angiogram performed several days later revealed an aneurysm. This example highlights the potential for introducing hindsight bias by using final written radiology reports as the gold standard for CT interpretations, an issue for all studies on this topic to date.
Finally, it is notable that all of the patients with false negative cranial CT studies met the age criteria of the SAH clinical decision rule. It is thus possible that a post-imaging rule could be further refined for improved specificity and ease of applicability.
LIMITATIONS
Given the retrospective nature of the study, appropriate characterization of early cranial CT is only as accurate as the available documentation. While the inter-rater agreement for abstraction of this dichotomous variable was 100%, we cannot exclude errors in reporting the actual time of headache onset such that some patients may have been both included and excluded inappropriately. However, such errors in reporting can occur in prospective observational studies as well, and thus is a more general limitation of using historical factors as part of any decision rule.
The completeness of our case identification was also limited by the accuracy of the diagnostic coding from hospital and ED encounters. However, we searched databases specific for diagnostic codes assigned during treatment within KPNC as well as those used to track services billed for outside of KPNC, thus making it unlikely that we failed to capture cases of aSAH that were transferred to a non-KPNC hospital. Regardless, it seems improbable that we failed to capture enough cases of aSAH to appreciably alter our results; for example, to raise the sensitivity point estimate for detection of aSAH by early cranial CT to 99.0%, we would require an additional 545 cases of aSAH, all with positive early cranial CT findings (693/700=0.99).
Given that we conservatively imputed missing variables for the SAH decision rule as being absent, it is also possible that the SAH clinical decision rule may have performed better than reported if missing variables were in fact present. However, recent prospective and retrospective validations of the same SAH clinical decision rule also revealed suboptimal sensitivities at 98.5% and 96.6%, respectively. 8, 15 Additionally, we cannot comment on the specificity of the SAH clinical decision rule when applied following early cranial CT, given the case-only cohort design of our study. It is possible that such an approach would not reduce overall testing rates from current practice, although individual clinicians may be differentially influenced given variable testing thresholds. 16
Finally, we did not include potentially missed cases of aSAH who may have undergone early cranial CT imaging with no confirmatory (LP) or subsequent testing, only to later present with evidence of aneurysm rupture. Inclusion of such cases in this cohort would bias towards a lower sensitivity of early cranial CT since it is impossible to be certain that evidence of SAH would have been otherwise detected (i.e., by lumbar puncture) at the initial evaluation.
CONCLUSION
In summary, we found the sensitivity of early cranial CT for aSAH to be 95.5% among patients presenting to non-academic EDs in an integrated healthcare system. Application of a SAH clinical decision rule in addition to early cranial CT improved sensitivity to 100% (95% CI [97.6–100.0]). Prospective decision rule refinement and validation of this approach is warranted.
Footnotes
Section Editor: William Whetstone, MD
Full text available through open access at http://escholarship.org/uc/uciem_westjem
Author contribution statement: DGM, MVK, DRV and DWB conceived the study and obtained research funding. DGM and NU refined participant and variable selection. DGM and MVK undertook chart abstraction and managed the data, including quality control. DGM drafted the manuscript, and all authors contributed substantially to its revision. DGM takes responsibility for the paper as a whole.
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. Funded by a Kaiser Permanente Northern California Community Benefits Grant.
REFERENCES
- 1.Nibbelink DW, Torner JC, Henderson WG. Intracranial aneurysms and subarachnoid hemorrhage - report on a randomized treatment study. IV-A. regulated bed rest. Stroke. 1977;8:202–18. doi: 10.1161/01.str.8.2.202. [DOI] [PubMed] [Google Scholar]
- 2.Edlow JA, Caplan LR. Avoiding pitfalls in the diagnosis of subarachnoid hemorrhage. N Engl J Med. 2000;342:29–36. doi: 10.1056/NEJM200001063420106. [DOI] [PubMed] [Google Scholar]
- 3.Backes D, Rinkel GJ, Kemperman H, et al. Time-dependent test characteristics of head computed tomography in patients suspected of nontraumatic subarachnoid hemorrhage. Stroke. 2012;43:2115–9. doi: 10.1161/STROKEAHA.112.658880. [DOI] [PubMed] [Google Scholar]
- 4.Perry JJ, Stiell IG, Sivilotti ML, et al. Sensitivity of computed tomography performed within six hours of onset of headache for diagnosis of subarachnoid haemorrhage: prospective cohort study. BMJ. 2011;343:d4277. doi: 10.1136/bmj.d4277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Claveau D, Dankoff J. Is lumbar puncture still needed in suspected subarachnoid hemorrhage after a negative head computed tomographic scan? CJEM. 2013;15:1–3. doi: 10.2310/8000.2013.130967. [DOI] [PubMed] [Google Scholar]
- 6.Vergouwen MD, Rinkel GJ. Clinical suspicion of subarachnoid hemorrhage and negative head computed tomographic scan performed within 6 hours of headache onset--no need for lumbar puncture. Ann Emerg Med. 2013;61:503–4. doi: 10.1016/j.annemergmed.2012.10.027. [DOI] [PubMed] [Google Scholar]
- 7.Mark DG, Hung YY, Offerman SR, et al. Nontraumatic subarachnoid hemorrhage in the setting of negative cranial computed tomography results: external validation of a clinical and imaging prediction rule. Ann Emerg Med. 2013;62:1–10.e1. doi: 10.1016/j.annemergmed.2012.09.003. [DOI] [PubMed] [Google Scholar]
- 8.Perry JJ, Stiell IG, Sivilotti ML, et al. Clinical decision rules to rule out subarachnoid hemorrhage for acute headache. JAMA. 2013;310:1248–55. doi: 10.1001/jama.2013.278018. [DOI] [PubMed] [Google Scholar]
- 9.Perry JJ, Stiell IG, Sivilotti ML, et al. High risk clinical characteristics for subarachnoid haemorrhage in patients with acute headache: prospective cohort study. BMJ. 2010;341:c5204. doi: 10.1136/bmj.c5204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Newman-Toker DE, Edlow JA. High-stakes diagnostic decision rules for serious disorders: the Ottawa subarachnoid hemorrhage rule. JAMA. 2013;310:1237–9. doi: 10.1001/jama.2013.278019. [DOI] [PubMed] [Google Scholar]
- 11.Gilbert EH, Lowenstein SR, Koziol-McLain J, et al. Chart reviews in emergency medicine research: Where are the methods? Ann Emerg Med. 1996;27:305–8. doi: 10.1016/s0196-0644(96)70264-0. [DOI] [PubMed] [Google Scholar]
- 12.Raina KD, Callaway C, Rittenberger JC, et al. Neurological and functional status following cardiac arrest: method and tool utility. Resuscitation. 2008;79:249–56. doi: 10.1016/j.resuscitation.2008.06.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Gwet K. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters. 3rd ed. Maryland, USA: Advanced Analytics, LLC; 2012. [Google Scholar]
- 14.Bates SM, Jaeschke R, Stevens SM, et al. Diagnosis of DVT: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest. 2012;141:e351S–418S. doi: 10.1378/chest.11-2299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kelly AM, Klim S, Edward S, et al. Sensitivity of proposed clinical decision rules for subarachnoid haemorrhage: An external validation study. Emerg Med Australas. 2014;26:556–60. doi: 10.1111/1742-6723.12325. [DOI] [PubMed] [Google Scholar]
- 16.Perry JJ, Eagles D, Clement CM, et al. An international study of emergency physicians’ practice for acute headache management and the need for a clinical decision rule. CJEM. 2009;11:516–22. doi: 10.1017/s1481803500011775. [DOI] [PubMed] [Google Scholar]