Spontaneous Subarachnoid Hemorrhage: A Systematic Review and Meta-Analysis Describing the Diagnostic Accuracy of History, Physical Exam, Imaging, and Lumbar Puncture with an Exploration of Test Thresholds

Abstract

Background

Spontaneous subarachnoid hemorrhage (SAH) is a rare, but serious etiology of headache. The diagnosis of SAH is especially challenging in alert, neurologically intact patients, as missed or delayed diagnosis can be catastrophic.

Objectives

To perform a diagnostic accuracy systematic review and meta-analysis of history, physical examination, cerebrospinal fluid (CSF) tests, computed tomography (CT), and clinical decision rules for spontaneous SAH. A secondary objective was to delineate probability of disease thresholds for imaging and lumbar puncture (LP).

Methods

PUBMED, EMBASE, SCOPUS, and research meeting abstracts were searched up to June 2015 for studies of emergency department (ED) patients with acute headache clinically concerning for spontaneous SAH. QUADAS-2 was used to assess study quality and, when appropriate, meta-analysis was conducted using random effects models. Outcomes were sensitivity, specificity, positive (LR+) and negative (LR−) likelihood ratios. To identify test- and treatment-thresholds, we employed the Pauker-Kassirer method with Bernstein test-indication curves using the summary estimates of diagnostic accuracy.

Results

A total of 5,022 publications were identified, of which 122 underwent full text-review; 22 studies were included (average SAH prevalence 7.5%). Diagnostic studies differed in assessment of history and physical exam findings, CT technology, analytical techniques used to identify xanthochromia, and criterion standards for SAH. Study quality by QUADAS-2 was variable; however, most had a relatively low-risk of biases. A history of neck pain (LR+ 4.1 [95% CI 2.2-7.6]) and neck stiffness on physical exam (LR+ 6.6 [4.0-11.0]) were the individual findings most strongly associated with SAH. Combinations of findings may rule out SAH, yet promising clinical decision rules await external validation. Non-contrast cranial CT within 6 hours of headache onset accurately ruled-in (LR+ 230 [6-8700]) and ruled-out SAH (LR− 0.01 [0-0.04]); CT beyond 6 hours had a LR− of 0.07 [0.01-0.61]. CSF analyses had lower diagnostic accuracy, whether using red blood cell (RBC) count or xanthochromia. At a threshold RBC count of 1,000 × 10⁶/L, the LR+ was 5.7 [1.4-23] and LR− 0.21 [0.03-1.7]. Using the pooled estimates of diagnostic accuracy and testing risks and benefits, we estimate LP only benefits CT negative patients when the pre-LP probability of SAH is on the order of 5%, which corresponds to a pre-CT probability greater than 20%.

Conclusions

Less than one in ten headache patients concerning for SAH are ultimately diagnosed with SAH in recent studies. While certain symptoms and signs increase or decrease the likelihood of SAH, no single characteristic is sufficient to rule-in or rule-out SAH. Within 6 hours of symptom onset, non-contrast cranial CT is highly accurate, while a negative CT beyond 6 hours substantially reduces the likelihood of SAH. LP appears to benefit relatively few patients within a narrow pre-test probability range. With improvements in CT technology and an expanding body of evidence, test-thresholds for LP may become more precise, obviating the need for a post-CT LP in more acute headache patients. Existing SAH clinical decision rules await external validation, but offer the potential to identify subsets most likely to benefit from post-CT LP, angiography, or no further testing.

Introduction

Headache is a common presenting complaint to the Emergency Department (ED), and represents approximately 2% of ED visits annually in the United States.¹ While most headaches are self-limiting, the possibility of spontaneous subarachnoid hemorrhage (SAH) is an important diagnostic consideration in patients presenting with a sudden, severe headache. These sudden onset headaches include the classic “thunderclap headache” that instantaneously peaks at headache onset, as well as headaches that reach maximal severity within seconds to one hour of onset. Sudden onset headaches include a wide spectrum of possible diagnoses, including SAH (approximately 1% of acute headaches), benign post-coital headache, exertional headache, intracranial cysts or tumors, intracerebral hemorrhage, hypophyseal apoplexy, sphenoid sinusitis, sinus thrombosis, cough-related, vascular dissection, cerebral vasospasm, and migraine headaches.^2-10 Migraine and other more benign headaches mimic SAH and are estimated to be at least 50-times more common than SAH.¹¹ Moreover, while SAH is often caused by a ruptured cerebral aneurysm and represents a neurosurgical emergency, a variety of other SAH etiologies range from the benign, low-pressure perimesencephalic hemorrhage which can be treated conservatively^12,13 to vascular malformations, arterial dissection, and vasculitis which require time-sensitive interventions (Table 1).¹⁴ Thus, the diagnostic approach to acute headache epitomizes the practice of emergency medicine with a high stakes condition without a clear-cut presentation lurking within a high volume complaint, and ultimately most patients do not have a serious diagnosis.

Table 1.

Potential Etiologies of Spontaneous SAH

Cerebral aneurysm

Perimesencephalic

Isolated convexity

Vascular malformations (arteriovenous malformations/AVF)

Arterial dissection

Cerebral amyloid angiopathy

Moyamoya

Vasculitis (PRES, RCVS, lupus)

Coagulopathy (thrombocytopenia, anticoagulation)

Sickle cell disease

Hypertension

Sympathomimetic drugs

Between 1960 and 1995, the six-month mortality of aneurysmal SAH decreased by 15% ¹⁵, but mortality remains 27%-44% with substantial regional variability.¹⁶ Up to 25% of patients die within 24 hours, and the three-month mortality rate can be as high as 50% without early definitive treatment.¹⁷ In addition, one-third of SAH survivors suffer neurological deficits affecting their daily lives¹⁵ and up to 50% never return to work.¹⁸ Early detection and treatment can significantly reduce morbidity and mortality.^11,19 However, SAH can be a challenging diagnosis to make, particularly in alert, neurologically intact adults.²⁰ Missed diagnosis is the primary reason for delayed treatment and case series suggest that 12%-53% of SAH cases are misdiagnosed on their initial presentation in a variety of settings, including the ED.²¹ Patients with misdiagnosed or undiagnosed warning bleeds that subsequently re-bleed have a 70% mortality.^22-25 ED providers miss the diagnosis of SAH in about 5% of cases, primarily those with lower acuity at presentation and misdiagnosis is estimated to be more likely in non-teaching hospitals.^26,27 Most emergency medicine and neurosurgery textbooks provide limited guidance on the diagnostic accuracy and utility of history, physical exam, cerebrospinal fluid (CSF) analysis, or CT for the diagnosis of SAH.^28,29 Indeed, given the serious nature of the condition, few have proposed a sensible threshold below which testing is not necessary, which contributes to over-testing and the unintended consequences that include incidental findings with prolonged patient uncertainty.³⁰

Unenhanced cranial CT is usually the first-line diagnostic test to evaluate for suspected SAH. Because early generation CTs did not detect up to 5% of SAHs, traditional teaching has emphasized the need to perform lumbar puncture (LP) to exclude SAH to reduce the probability of error. Yet many patients fear having an LP because an LP is painful, invasive, and can lead to a post-LP headache in 6%-30% of patients depending on the gauge of the LP needle.³¹ Additional clinician-level barriers to LP testing include the time needed to perform³² and low diagnostic yield.^20,33-36 In addition, LP may be contraindicated if a patient has a bleeding disorder, increased intracranial pressure, or simply does not consent to the procedure.³⁷ Despite dogma, LP is not always performed after CT and multicenter observational studies report that fewer than half of acute headache patients undergo LP following a negative CT to “rule-out” SAH. Even in these studies less than one in 100 LPs are ultimately deemed “true positives” after negative CT using a third generation scanner.^38,39 Additionally, the specificity of LP is reduced by “traumatic taps” in which bleeding from the procedure contaminates the CSF being collected, estimated to occur in about one in six LPs.⁴⁰ Identifying xanthochromia due to the breakdown of hemoglobin in CSF is considered to be pathognomonic for SAH, yet xanthochromia requires up to 12 hours to develop, and is prone to false positives due to ex vivo hemolysis or hyperbilirubinemia.^21,41 National guidelines in the United Kingdom advocate using spectrophotometry to identify xanthochromia, yet nearly all North American hospitals rely on visual inspection alone.⁴² A recent systematic review comparing visual inspection to spectrophotometry to detect CSF xanthochromia was unable to draw a conclusion due to significant heterogeneity in the underlying studies.⁴³

To our knowledge, there are no prior systematic reviews assessing the diagnostic accuracy of history, physical exam, and imaging studies for SAH in the setting of acute onset headache patients in the ED. Therefore, we set out to collect and summarize the available published literature regarding these characteristics as well as LP findings and clinical decision rules for spontaneous (non-traumatic) SAH. Since recent systematic reviews have evaluated CT angiography (CTA)⁴⁴ and magnetic resonance angiography (MRA),⁴⁵ we did not repeat these meta-analyses. However, additional studies exploring visible xanthochromia diagnostic accuracy have been published since the 2014 systematic review by Chu et al⁴³ so we report an updated meta-analysis of visible xanthochromia. We also used these diagnostic accuracy estimates to delineate an optimal diagnostic strategy using the test-treatment threshold approach of Pauker and Kassirer.^46,47 In particular, we used the graphical Bayesian approach of test-indication curves first proposed by Bernstein⁴⁸ to identify the a priori likelihood of disease in patients for whom LP would be rational despite a negative cranial CT.

Methods

Study Design

We conducted a systematic review and meta-analysis of original research studies that reported data on the diagnosis of SAH in ED patients with acute headache. The design and manuscript structure conform to the recommendations from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)⁴⁹ and the Meta-analysis of Observational Studies (MOOSE) criteria.⁵⁰ Studies were included if they described adult patients (14 years or older) seen in an ED with acute headache or other symptoms or signs (such as syncope, acute mental status change, or otherwise unexplained nausea) and in whom SAH was a diagnostic consideration. In addition, studies had to report sufficient detail on the diagnostic test and criterion standard to be able to construct two-by-two tables. We elected to define “disease positive” as being an acute SAH including non-aneurysmal SAH (e.g. perimesencepahlic hemorrhage) as most original studies did not distinguish between SAH subtypes. We recognized that CT findings of subarachnoid blood were therefore incorporated into this definition, thereby inflating CT sensitivity, specificity, LR+ and LR− (incorporation bias), but report these estimates for completeness.⁵¹ Recognizing the absence of an accepted definition for a positive LP, we analyzed separately an erythrocyte count above 1,000×10⁶/L in the final tube as well as either visible or spectrophotometric xanthochromia. Patients with negative imaging in whom LP was not performed were deemed as “disease negative” only when the authors reported effort at follow-up after hospital discharge. We excluded studies that solely reported data on patients with SAH (i.e. where only sensitivity could be calculated), precluding an estimate of specificity or likelihood ratios.

Search Strategy

We searched the published literature using strategies created for the concepts of Emergency Department, Headache, Subarachnoid Hemorrhage, Physical Exam, and Diagnostic Accuracy. Search strategies were established using a combination of standardized terms and key words, and were implemented in PubMed 1946-, Embase 1947-, and Scopus 1823-. All searches were completed in June 2015 and limited to English using database supplied limits. Full search strategies for PubMed and Embase are provided in the Online Appendix 1. One author (AMH) independently reviewed the titles and abstracts to identify potentially relevant articles. Where there were questions about inclusion, this was discussed with a second author (JMP) and a consensus decision was made about whether the article should be included for full-text review. Studies deemed not to contain clear inclusion or exclusion criteria were evaluated by a second author (JMP) before final inclusion or exclusion.

Data from each study were independently abstracted by two authors (AMH, CRC). Information abstracted included the individual study setting, inclusion criteria, exclusion criteria, specific CT (machine, slice number, and type of CT image), definition of disease, criteria for a positive LP, follow-up description, and whether or not there was neurosurgical intervention. The abstracted data were constructed into tables, which were confirmed by a second author (CRC); any uncertainty was discussed between the two authors and a consensus decision was made. In addition, the references for each full-text article were reviewed to assess whether additional studies could be included in the review, which were then assessed by full-text review. The authors conducted bibliographic searches of research abstract presented at scientific meetings published in Academic Emergency Medicine, Annals of Emergency Medicine, Stroke, Neurology, and Neurosurgery, from 2002 through October 2015.

Individual Evidence Quality Appraisal

Two authors (AMH, MJW) independently used the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS-2)⁵² for systematic reviews to evaluate the quality of evidence for the studies identified. These two authors used several a priori conditions to evaluate an individual study’s risk of bias and degree of applicability.

• In the assessment of “inappropriate exclusions” reviewers assessed whether a study’s inclusion and exclusion criteria led investigators to evaluate suspected SAH patients who were more or less acutely ill than those typically evaluated in ED settings, which could introduce spectrum bias⁵¹ or spectrum effect⁵³ and skew observed estimates of sensitivity (with sicker populations) or specificity (with less sick populations) upwards.

• If the study setting was anything other than an ED (for example, a neurosurgery clinic) then the results were assessed as “low applicability”.

• In the assessment of continuous data like CSF RBCs, if the authors pre-defined a threshold for abnormal (example, CSF RBC > 5 × 10⁶ cells/L in the last tube is “abnormal”) then the study results were assessed as “low risk” of bias.

• If the conduct of the index test differed from routine care (example, xanthochromia assessed using only spectrophotometry), then the results were assessed as “low applicability”.

• No widely accepted criterion standard for SAH yet exists, so we accepted the gold standard of subarachnoid blood on unenhanced CT of the head, or CSF xanthochromia, or CSF RBCs > 5 × 10⁶/L in the final sample of CSF when any of these findings included an aneurysm or arteriovenous malformation evident on cerebral angiography, as previously established by a consensus panel of emergency physicians and a neurosurgeon.^39,54 If less than 100% of a study’s subjects underwent cerebral angiography, a surrogate outcome of medical record review and structured telephone follow-up of at least 6-months was necessary.²⁰ Any other criterion standard was assessed as “high risk” for bias.

• Assessing whether the target condition is identified is challenging in SAH diagnostic research. The ideal patient-centric diagnostic test would identify SAH patients with a lesion amenable to surgical or endovascular intervention rendering measurable benefit in mortality and morbidity.⁵⁵ Since perimesencephalic bleeds cause about 10% of SAH cases yet require no intervention other than symptom control, diagnostic studies that report aneurysmal SAH separately from perimesencephalic SAH would provide a higher level of evidence by which to associate testing with patient-benefit. Unfortunately, few SAH studies report aneurysmal versus non-aneurysmal SAH separately, so the decision was made that any SAH identified using an acceptable criterion standard as defined above would be assessed as “high applicability”.

• In assessing the uniformity of the criterion standard reported, if all patients with non-diagnostic CT and LP had either subsequent advanced imaging (angiography) or post-ED follow-up, then the risk of differential verification bias was assessed as being “low”.⁵¹

• If a substantial number of patients were reported lost to follow-up and no sensitivity analysis assuming worst-case scenario (lost to follow-up = false negative SAH index test), then the risk of bias was assessed to be “high”.

Inter-rater QUADAS-2 reliability was assessed using prevalence adjusted, bias adjusted kappa values to adjust for unique biases between reviewers and the distribution of responses across categories.⁵⁶ Discrepancies were then resolved by consensus.

Data Collection

We abstracted the study setting, study inclusion criteria, the criterion standard employed to diagnose SAH, CT technology, CSF analysis, follow-up procedures in patients who did not undergo both CT and LP, disease prevalence, and every variable on history, physical, or testing for which a 2×2 diagnostic accuracy table could be constructed. When reconstructed 2×2 contingency tables differed from the original investigators’ reports of sensitivity and specificity, we contacted the authors for clarity.

Data Analysis

Meta-analysis estimates were computed by one author (CRC) when ≥1 studies assessed the same finding on history or physical examination, or index testing. We chose to combine unenhanced cranial CT studies without distinguishing specific generations of technology since the detector number or geometry and the thickness of slices were frequently not specified. We generated combined estimates for diagnostic accuracy using a random-effects model (Meta-DiSc Version 1.4, Hospital Universitario Ramón y Cajal, Madrid, Spain).^57,58 Interstudy heterogeneity was assessed using the Der-Simonian-Laird random effects model and the Index of Inconsistency (I²).^59,60 Pooled estimates of dichotomous positive (LR+) and negative (LR−) likelihood ratios are also reported from the random effects model. Publication bias was not assessed because this is not an accepted approach in diagnostic meta-analyses due to the small number of studies generally identified.⁶¹

The added value of likelihood ratios over positive and negative predictive values is that likelihood ratios are independent of disease prevalence.⁶² In addition, likelihood ratios can be used on an individual patient to estimate post-test probabilities, unlike sensitivity and specificity.⁶³ Larger LR+ increase the post-test probability of a disease more than smaller values of LR+; similarly, smaller LR− decrease the post-test probability of a disease more than larger values of LR−. Note that a likelihood ratio of 1 does not change the post-test probability at all. One rule of thumb is that LR+ >10 and LR− < 0.1 provide considerable diagnostic value.⁶⁴

Test–Treatment Threshold

We used the Pauker-Kassirer method to estimate thresholds for further testing or treatment.⁴⁷ This technique is based on the diagnostic accuracy of a test, as well as the estimated risks of testing and of treatment, and the anticipated benefit of treatment. We performed a separate analysis for each of the two primary tests in question, namely cranial CT (stratified by time post headache onset) and LP (using either CSF RBC or visible xanthochromia to define “positive LP”). We used the pooled estimates from the meta-analysis for test sensitivity and specificity, and our collective clinical experience and selected references to estimate risks and benefits. We extended the test-indication curve method first proposed by Bernstein⁴⁸ to visually represent this analysis. Specifically, the curves display how the information conveyed by a positive or negative test result affects a given patient’s post-test probability of disease relative to the treatment threshold. These test-indication curves were originally developed to illustrate the pre-test probability at which testing is no longer indicated based only on the net utilities of treating or not treating patients with and without disease.⁴⁶ We extended this graphical approach to incorporate the non-zero risks of testing, which necessarily narrow the range of probabilities for which testing remains rational.^47,65 Because the precise utilities of both testing and treatment are difficult to estimate on a common scale, we performed sensitivity analyses by doubling and halving the calculated treatment threshold (i.e. the ratio of risks of treatment to the sum of risks and benefits),⁴⁸ and also by reducing the risk of testing by half and recalculating the test thresholds. Recognizing that these estimates are nevertheless somewhat arbitrary, we also provide an interactive calculator to permit readers to re-compute thresholds using different estimates of test performance or anticipated risks and benefits that may be more applicable to the end users’ patient populations and clinical environments.

In developing the test-treatment threshold, we were most interested in the common clinical dilemma of whether to perform LP after a negative cranial CT, or to forgo LP and proceed directly to angiography. Because CT-positive or LP-positive patients do not immediately undergo neurosurgical clipping or endovascular coiling, the next step being contemplated is usually angiography, typically by CT or MRA (and sometimes by digital subtraction fluoroscopy), followed by intervention when a culprit aneurysm is identified. Therefore, for Pauker-Kassirer modeling we designated the “test” of interest as being either the unenhanced CT or the LP, and the “treatment” as being angiography. Furthermore, because an LP is usually done after a negative CT, we considered how a negative unenhanced cranial CT would affect the post-CT (i.e. pre-LP) probability of still having a SAH across the range of pre-CT probabilities.

Results

A total of 5,022 citations were identified through a search of PUBMED, EMBASE, and SCOPUS. From this search, 399 duplicate citations were removed. 122 studies underwent full text-review; of these, 22 studies were included in the current analysis. (Figure 1 and Table 2). Although most studies enrolled only awake and alert patients with a GCS of 15, some also included those with decreased level of consciousness which could introduce spectrum effect into observed estimates of diagnostic accuracy. All studies were conducted in North American or European countries. Each study included the presence of subarachnoid blood on unenhanced cranial CT as representing “disease positive”. Descriptors of the specific technology utilized for CT scanners and spectrophotometric assessment for CSF xanthochromia were inconsistent. Definitions of xanthochromia ranged from direct visual inspection to spectrophotometric analysis. When using spectrophotometry, studies varied with regards to wavelengths sampled and specific algorithms used to adjust for interference by oxyhemoglobin, protein, and other corrections. Clinical follow-up proportion varied considerably, ranging from no follow-up to complete follow-up with cross-checking of coroner records to ensure no missed deaths. Assessment of studies by QUADAS-2 (Table 3) showed significant heterogeneity in reference standards used, blinding of index test interpreters to the reference standard and vice versa. Agreement between reviewers was fair for all QUADAS-2 elements except for index and reference standard blinding, reference standard appropriateness, and acceptable application of index test in which cases the agreement was poor.⁶⁶ However, most studies had a relatively low-risk of biases overall.

Figure 1.

Study Selection Process

Table 2.

Summary of Included Studies

Study	Location	Setting	No. Patients	Mean age*	Inclusion Criteria	Study Design	SAH Criterion Standard	SAH prevalence %	Variables Assessed	Follow- Up
Backes 2012	University Medical Center, Utrecht, Netherlands, 2005-2012	ED	250	48*	Age >16 years, clinical suspicion of acute onset SAH, GCS 15	Two prospective databases. First database consecutive patients with confirmed SAH. Second all LP patients with SPT	Not explicitly defined	42.0	Sen/Spec CT at <6- hours and >6-hours for SAH	None
Bo 2008	Akershus University Hospital, Norway, 1998-2002	ED	433	44	Age >14 years, acute onset worst headache of life	Prospective observational consecutive sampling cohort	Consensus between two neurologists, but not explicitly defined	16.4	Sen/Spec of elements of history including activity at onset, and peak intensity for SAH	None
Boesiger 2005	Pitt County Memorial Hospital, Greenville SC, 2002	ED	177	NR	Adults with headache work-up including CT and LP	Retrospective chart review	SAH observed on CT or ≥400 RBC in CSF tube 1 that did not clear by 10-fold	3.4	Sen/Spec of CT for SAH	Phone f/u at 1-year if ≥400 RBC in CSF tube 1 that did not clear by 10-fold
Brunell 2013	Uppsala University Hospital, Uppsala Sweden, 2009-2011	Hospit al	453	NR	Age>10 years with LP to exclude SAH	Lab information system review	CSF hemoglobin detection via SPT using 415 nm wavelength or CSF bilirubin detection via automatic analyzer	1.1	Sen/spec of CSF RBC for SAH	None
Carstairs 2006	Naval Medical Center San Diego CA, July 2002-July 2004	ED	116	39	Age>18 with either worst headache of life or onset <60 seconds or associated neuro deficit	Prospective observational consecutive sampling cohort	SAH observed on CT or RBC in CSF tube 1 that did not clear by >25% in tube 4 or presence of xanthochromi a on visual inspection by lab	4.3	Sen/Spec of CTA & elements of history including stiff neck, blurred vision, & altered mental status	None
Czuczman 2013	Unspecified urban tertiary care hospital, 2001-2009	ED	220	44	Adults with headaches billed for LPs, ≥5 RBC in final CSF tube, and either CT angiogram or magnetic resonance angiogram within 2- weeks	Retrospective case series	Either 1) presence of SAH on imaging; 2) xanthochromi a with aneurysm or AVM>2mm; 3) xanthochromi a and culture- or PCR- negative meningitis	11.8	Sen/Spec of elements of history including sudden onset headache, worst headache, & onset with exertion; CSF RBC iLRs	None
Dupont 2008	St. Mary’s Hospital, Rochester MN, 1998- 2007	ED	117	45	Age>18 with headache onset <2 weeks prior with both non-contrast CT for detection of blood and LP	Retrospective chart review	Not specified	12.0	Sen/Spec visual xanthochr omia for aneurysm	None
Gangloff 2015	Hôpital de l’Enfant-Jésus, Quebec Canada, 2003- 2009	ED	706	41	Age>14 with acute headache suspicious for SAH, GCS 15, and initial head CT negative for SAH with subsequent LP	Retrospective case series	Presence of any aneurysm and presence of either visual xantho or >5 × 106 RBC/L in last CSF tube	0.7	Sen/Spec visual xantho, iterative SPT, or UK NEQUA SPT	None
Hann 2015	Royal Brisbane & Women’s Hospital, June 2005-July 2012	ED	409	38	Adults discharged with headache after CSF evaluation for xantho	Retrospective case series	Presence of vascular aneurysm on angiogram within 30- days of headache or no repeat ED visit or SAH death in 30- days	1.5	Sen/Spec visual xantho via visual inspection	Review of Queensland death registry
Landtblom 2002	University Hospital, Linköping Sweden over unspecified 31-month period	ED	137	42*	Age>18 years presenting within 10 days of headache onset with abrupt onset (<10 second) headache, examined by on-call Neurologist	Prospective observational cohort	Not specified	11.3	Sen/Spec of elements of history including sudden onset headache, worst headache, & nausea	Phone at 2-4 days, 7-9 days, and 1-, 3-, 6-, and 12- months for 90 subjects without SAH
Linn 1998	Utrecht University Hospital, Utrecht, Netherlands, Jan 1992-Oct 1994	ED	102	48	Alert adult patients without focal neuro deficit referred to ED by general practitioner with concern for SAH with CT and LP	Prospective observational cohort	Not specified	41.2	Sen/Spec of elements of history including activity at onset and acuity of onset	None
Morgenstern 1998	Hermann Hospital, Houston TX, March 1995- June 1996	ED	97	39	Adult patients with 10/10 worst headache of life	Prospective observational cohort	Either 1) presence of SAH on imaging OR 2) CSF RBC > 1000 with <25% decrement from first to last tube with either visual xantho, SPT xantho, or elevated D- dimer	18.6	Sen/Spec of elements of history & physical exam including nausea, stiff neck, and lethargy	Phone f/u at mean 24 months
O’Neill 2005	Unspecified hospital over unspecified 1- year period	ED	116	NR	Acute headache <24 hours, clinical suspicion SAH, referred for CT from ED	Retrospective chart review	Not specified	16.4	Sen/Spec of CT for SAH	None
Perry 2006	6 Canadian tertiary care EDs, July 2002-Jan 2004	ED	220	42	Age>15 years with acute, spontaneous headache with or without syncope	Pre-planned sub- study of prospective, consecutive patient cohort study	Any one of the following: subarachnoid blood on CT, >5×106/L RBC in 3rd or 4th tube of CSF, or visible xantho + aneurysm on cerebral angiography	0.9	Sen/Spec of visual inspection for xanth and 4 SPT methods for xantho for SAH	Phone f/u at 1- month for CT-, LP- patients; chart review at 1-month for all patients
Perry 2010	6 Canadian tertiary care EDs, Nov 2000-Nov 2005	ED	1999	43	Age≥16 years with spontaneous headache peaking within 1- hour or syncope associated with a headache	Prospective cohort study	Any one of the following: subarachnoid blood on CT, visual xantho, >5×106/L RBC in the final tube of CSF with an aneurysm or AVM on cerebral angiography	6.5	Sen/Spec of elements of history including activity at onset, neck stiffness, and arrival mode. Also, derivation of 3 CDRs	Phone f/u at 1- and 6-months for all patients without CT or LP; chart review & Ontario registry review at study end for same patients
Perry 2011	11 Canadian tertiary care EDs, Nov 2000-Dec 2009	ED	3132	45	Age>15 years with acute, spontaneous headache with or without syncope	Prospective cohort study	Any one of the following: subarachnoid blood on CT, visual xantho, >5×106/L RBC in the final tube of CSF with an aneurysm or AVM on cerebral angiography	7.7	Sen/Spec CT at <6- hours and >6-hours for SAH	Phone f/u at 1- and 6-months for all patients without CT or LP; chart review & provincial coroner’s records at study end for same patients
Perry 2013	10 Canadian tertiary care EDs, Nov 2000-Dec 2009	ED	2131	44	Age≥16 years with spontaneous headache peaking within 1- hour or syncope associated with a headache	Prospective cohort study	Any one of the following: subarachnoid blood on CT, visual xantho, >5×106/L RBC in the final tube of CSF with an aneurysm or AVM on cerebral angiography	6.2	Sen/Spec of elements of history & physical exam including activity at onset, neck stiffness, and arrival mode. Also, derivation of 4 SAH CDRs	Phone f/u at 1- and 6-months for all patients without CT or LP; chart review & provincial coroner’s records at study end for same patients
Perry 2015	12 Canadian tertiary care EDs, April 2006-July 2010	ED	1739 1098 normal LP 641 abnl LP	44	Age>15 years with acute, spontaneous headache with or without syncope with LP to evaluate potential SAH	Pre-planned sub- study of prospective, consecutive patient cohort study	Aneurysmal SAH if: subarachnoid blood on CT, visual xantho,or any RBC in the final tube of CSF with an aneurysm on cerebral angiography	2.3	Sen/Spec of <2000 × 106/L RBC in CSF + no xantho for diagnosis aneurysmal SAH	Phone f/u at 1- and 6-months for all patients without CT or LP; chart review & provincial coroner’s records at study end for same patients
Van der Wee 1995	University hospitals in Utrecht and Rotterdam Netherlands, 1989-1993	NR	175	NR	Acute headache without confusion or focal deficit with CT within 12- hours	Unclear whether retrospective or prospective “consecutive” case series	Not specified	68	Sen/Spec CT at <12- hours for SAH	None
Wood 2015	Princess Alexandra Hospital, Brisbane Queensland Australia, Jan 2000-April 2003	NR	240	NR	Acute or unusually severe headache with LP after a normal cranial CT for possible diagnosis of SAH	Retrospective chart review	Uniform CSF bloodstaining across serial samples with visual xantho and “positive” angiography	0.8	Sen/Spec of CSF SPT (XI > 0.080)	None

^*Median

Abbreviations: ED = emergency department, SAH = subarachnoid hemorrhage, GCS = Glasgow Coma Scale, sen = sensitivity, spec = specificity, CT = computed tomography, f/u = follow-up, RBC = red blood cell, CTA = CT angiogram, AVM = arteriovenous malformation, LP = lumbar puncture, CSF = cerebrospinal fluid, iLR = interval likelihood ratio, UKNEQUASPT = United Kingdom National External Quality Assessment Service photochromatography, xantho = xanthochromia, CDR = clinical decision rule

Table 3.

Overview of Quality Assessment of Included Studies

Study	Sample	Avoided Inappropriate Exclusions?	Patients and Settings Match Study Question?	Blinded Interpretation of Index Test?	Acceptable Index Test Application?	Acceptable Reference (Gold) Standard?	Outcomes Assessed by Blinded Assessor?	Assessed Outcomes Pertinent?	Appropriate Interval Between Index Test & Reference Standard?	Same Reference (Gold) Standard for All Patients?	All Patients Analyzed?
Backes 2012	Consecutive	YES	YES	NO	YES	NO	NO	NO	YES	YES	YES
Bo 2008	Consecutive	YES	YES	NO	NO	NO	NO	NO	YES	YES	YES
Boesiger 2005	Consecutive	YES	YES	NO	YES	YES	YES	NO	YES	YES	YES
Brunell 2013	Consecutive	YES	YES	YES	NO	NO	YES	NO	YES	YES	YES
Carstairs 2006	Consecutive	YES	YES	YES	YES	YES	YES	YES	YES	YES	YES
Czuczman 2013	Consecutive	YES	NO	YES	YES	YES	NO	YES	YES	YES	YES
Dupont 2008	Consecutive	NO	YES	NO	YES	YES	NO	YES	YES	NO	YES
Gangloff 2015	Consecutive	YES	YES	NO	NO	YES	NO	YES	YES	NO	NO
Hann 2014	Consecutive	NO	NO	NO	NO	YES	NO	YES	YES	YES	YES
Landtblom 2002	Consecutive	YES	YES	NO	NO	YES	NO	YES	YES	YES	YES
Linn 1998	Consecutive	YES	YES	NO	YES	NO	NO	YES	YES	NO	YES
Morgenstern 1998	Consecutive	YES	YES	YES	NO	NO	YES	NO	YES	YES	NO
O’Neill 2005	Consecutive	YES	YES	YES	YES	NO	YES	YES	YES	YES	NO
Perry 2006	Consecutive	NO	YES	YES	YES	YES	NO	YES	YES	YES	YES
Perry 2010	Consecutive	YES	YES	YES	YES	YES	NO	YES	YES	YES	NO
Perry 2011	Consecutive	YES	YES	YES	YES	YES	YES	YES	YES	YES	NO
Perry 2013	Consecutive	YES	YES	YES	YES	YES	YES	YES	YES	YES	YES
Perry 2015	Consecutive	YES	YES	YES	YES	YES	NO	YES	YES	YES	YES
van der Wee 1995	Consecutive	NO	NO	NO	NO	NO	NO	NO	YES	YES	YES
Wood 2005	Consecutive	NO	NO	NO	NO	NO	NO	YES	NO	YES	YES
Kappa *	1.00	0.33	0.52	0.05	0.14	0.05	0.05	0.43	0.52	0.62	0.33

^*Prevalence adjusted, bias adjusted kappa as described by .

Prevalence

The overall prevalence of SAH in these studies ranged from 0.91% to 68%. In prospective studies within the last ten years, the prevalence ranged from 0.91% to 17% with a weighted average of 7.5%. Two of the studies with the lowest prevalence enrolled patients undergoing LP following a negative CT.^67,68

History and Physical Examination

Eight studies described the diagnostic accuracy for 22 components of history^{2,17,33,38,39,69-71} (Table 4 and Figure 2) and six studies described the diagnostic accuracy for four physical exam tests ^2,17,69,70 (Table 4 and Figure 3) for SAH. Within these studies there was significant heterogeneity in the manner in which history and physical characteristics were obtained and defined. For example, Perry et al. defined limited neck flexion on physical exam as “inability to touch chin to chest or raise the head 8 cm off the bed if supine”,³⁹ whereas Carstairs provided no descriptor of how “nuchal rigidity” was objectively assessed on physical exam.¹⁷ Perry et al. reported the inter-physician reliability of elements of the history and physical exam, with kappa (κ) values ranging from 0.44 to 1.00. The least reliable findings were thunderclap headache (κ= 0.49) and limited neck flexion (κ= 0.44).³⁹

Table 4.

Single-Study Predictors of SAH from History and Physical Exam Available

Risk Factor	Sensitivity, % (95% CI)	Specificity, % (95% CI)	Positive LR (95% CI)	Negative LR (95% CI)
Diplopia
Landtblom 2002	0 (0-15)	98 (94-100)	0.96 (0.05-19.33)	1.00 (0.94-1.07)
Family History cerebral aneurysm
Czuczman 2013	0 (0-13)	92 (87-95)	0.22 (0.01-3.54)	1.07 (1.00-1.15)
Lethargy
Morgenstern 1998	39 (17-64)	82 (72-90)	2.19 (1.04-4.64)	0.74 (0.51-1.09)
Onset < 1 minute	50 (34-66)	45 (32-58)	0.91 (0.62-1.33)	1.11 (0.74-1.68)
Linn 1998
Onset 1-5 minutes	24 (12-39)	87 (75-94)	1.79 (0.77-4.14)	0.88 (0.72-1.07)
Linn 1998
Onset <1 hour	100 (95-100)	12 (9-16)	1.13 (1.09-1.19)	0.06 (0-0.95)
Bo 2008
PMH chronic headache	19 (7-39)	79 (73-85)	0.93 (0.40-2.91)	1.02 (0.83-1.24)
Czuczman 2013
PMH of hypertension	31 (14-52)	80 (74-85)	1.53 (0.81-2.91)	0.87 (0.66-1.13)
Czuczman 2013
Scotomata	0 (0-15)	93 (87-97)	0.28 (0.02-4.72)	1.06 (0.98-1.14)
Landtblom 2002
Similar headache in past	19 (9-34)	90 (79-96)	1.90 (0.71-5.09)	0.90 (0.76-1.07)
Linn 1998

Figure 2.

Forest Plots for Diagnostic Elements of History for SAH

Figure 3.

Forest Plots for Diagnostic Elements of Physical Exam for SAH

No single element of history had a very high pooled LR+. The findings on history that increased the probability of SAH the most included subjective neck stiffness (LR+ 4.12, 95% CI 2.24-7.59), while the absence of “worst headache of life” (LR− 0.36, 95% CI 0.01-14.22) or onset of headache over more than one hour (LR− 0.06, 95% CI 0-0.95) each reduced the probability of SAH.

On physical exam, neck stiffness (LR+ 6.59, 95% CI 3.95-11.00) was strongly associated with SAH as reported in three studies, but no single physical exam finding had a LR− less than 0.74. Most elements of history and physical exam demonstrated significant statistical heterogeneity.

Clinical Decision Rules

Four related SAH clinical decision rules have been described, three of which were prospectively validated in a subsequent study by the same investigator group to generate the final refined Ottawa SAH Rule. Since more than one study is needed to perform a meta-analysis, we were unable to pool diagnostic accuracy results for SAH Clinical Decision Rules. The component elements of each decision rule, along with their combined accuracy and reliability are available in Online Appendix 2. Rule 1 appears sufficient to rule-out SAH (LR− 0.06, 95% CI 0.01-0.22), was uncomfortable to use for only 18% of surveyed emergency physicians, was misinterpreted in 4.7% of cases, and would theoretically decrease CT and/or LP testing rates from 84% to 74%. On the other hand, the Ottawa SAH Rule more accurately rules out SAH (LR− 0.02, 95% CI 0.00-0.39), but could increase CT and/or LP testing rates if strictly applied.³⁹

Advanced Imaging

Five studies examined the diagnostic accuracy of non-contrast CT (Figure 4).^20,72-75 The pooled estimate for sensitivity was 94% (95% CI 91%-96%, I² = 74%) and LR− 0.07 (95% CI 0.03-0.17, I² = 78%). van der Wee et al assessed CT accuracy at 12 hours, but did not provide sufficient data to reconstruct 2×2 tables stratified by that time threshold.⁷² Three other studies reported diagnostic accuracy stratified by the time interval since symptom onset, but only two reported the threshold of less than 6 hours.^20,70,75 Pooled sensitivity within the first 6 hours of symptom onset is 100% (95% CI 98%-100%, I² = 0%) and LR− 0.01 (95% CI 0-0.04, I² = 0%). Beyond 6 hours the pooled sensitivity of CT is 89% (95% CI 83%-93%, I² = 89%) and LR− 0.07 (CI 0.01-0.61, I² = 63%).

Figure 4.

Forest Plots for Diagnostic Accuracy of CT for SAH

CSF Erythrocytes and Xanthochromia

Six studies examined diagnostic accuracy of xanthrochromia on CSF analysis, but investigators used variable methods to assess xanthochromia ranging from visible inspection^33,67,68,76 to multiple different spectrophotometric assays (Table 5, Figure 5).^34,67,68,77 For visible xanthochromia the pooled meta-analysis results were: sensitivity 85% (95% CI 66%-96%, I² = 0%), specificity 97% (95% CI 96%-98%, I² = 13%), LR+ 24.67 (95% CI 12.13-50.14, I² = 64%), and LR− 0.22 (95% CI 0.09-0.54, I² = 13%). Both Perry and Gangloff evaluated spectrophotometric bilirubin using the United Kingdom National External Quality Assessment Service (UKNEQUAS) algorithm⁷⁸ with pooled sensitivity 100% (95% CI 59%-100%, I² = 0%), specificity 95% (95% CI 93%-96%, I² = 98%), LR+ 15.23 (95% CI 1.58-146.73, I² = 96%), and LR− 0.13 (0.02-0.83, I² = 0%).^67,68 The diagnostic accuracies of other spectrophotometric methods to evaluate for CSF bilirubin are also summarized in Table 5.

Table 5.

Diagnostic Accuracy of CSF Erythrocytes and Xanthochromia for SAH

Risk Factor	Sensitivity, % (95% CI)	Specificity, % (95% CI)	Positive LR (95% CI)	Negative LR (95% CI)
CSF RBC >1000
Czuczman 2013	65 (44-83)	79 (72-84)	3.09 (2.09-4.57)	0.44 (0.26-0.75)
Perry 2015	93 (68-100)	91 (88-93)	10.25 (7.73-13.59)	0.07 (0.01-0.49)
Pooled Accuracy	76 (60-88)	88 (86-90)	5.66 (1.38-23.27)	0.21 (0.03-1.66)
Xanthochromia – UK NEQAS	100 (16-100)	83 (77-88)	4.87 (2.71-8.73)	0.20 (0.02-2.53)
Perry 2006	100 (48-100)	98 (97-99)	47.67 (26.67-85.20)	0.08 (0.01-1.21)
Gangloff 2015	100 (59-100)	95 (93-96)	15.23 (1.58-146.73)	0.13 (0.02-0.83)
Pooled Accuracy
Xanthochromia -- Visual
Perry 2006	50 (1-99)	97 (93-99)	15.57 (3.25-74.54)	0.52 (0.13-2.07)
Dupont 2008	93 (66-100)	95 (89-98)	19.13 (8.04-45.45)	0.08 (0.01-0.50)
Gangloff 2015	80 (28-99)	99 (98-99)	62.31 (28.47-136.37)	0.20 (0.04-1.17)
Pooled Accuracy	31 (21-41)	98 (97-99)	28.79 (9.77-84.80)	0.22 (0.06-0.80)
Xanthochromia -- Traditional
Perry 2006	100 (16-100)	29 (23-35)	1.17 (0.70-1.96)	0.57 (0.05-7.28)
Xanthochromia – Chalmers/Kiley
Perry 2006	0 (0-84)	89 (84-93)	1.49 (0.12-19.23)	0.94 (0.56-1.56)
Xanthochromia – Chalmers revised	100 (16-100)	29 (23-35)	1.17 (0.70-1.96)	0.57 (0.05-7.28)
Perry 2006

Figure 5.

Forest Plots for Diagnostic Accuracy of CSF Analysis for SAH

Two studies evaluated CSF erythrocyte counts to distinguish a traumatic LP from an aneurysmal SAH. Perry et al. noted a sensitivity of 93% (95% CI 68%-100%), specificity 91% (95% CI 88%-93%), LR+ 10.25 (95% CI 7.7-13.6), and LR− 0.07 (95% CI 0.01-0.49) at a threshold CSF RBC <2,000 × 10⁶/L in the final tube of CSF collected.⁷⁹ Czuczman et al. reported interval likelihood ratios (iLR)⁸⁰ for CSF RBC counts in the final tube collected (using units of cells per microliter): iLR_<100 = 0, iLR_100-10,000 = 1.6, iLR_>10,000 = 6.3.³³ Using raw data from these two studies to evaluate the summary diagnostic accuracy for CSF RBC in the final tube at an arbitrarily selected new threshold of 1,000 × 10⁶/L, the pooled sensitivity was 76% (95% CI 60%-88%, I² = 79%), specificity was 88% (95% CI 86%-90%, I² = 95%), LR+ was 5.7 (95% CI 1.4-23, I² = 97%), and LR− 0.21 (95% CI 0.03-1.66 I² = 78%). The diagnostic accuracy of CT and LP, and specifically how the test results affect the post-test probability of disease in a Bayesian fashion, are represented in Figure 6.

Figure 6. Bernstein Test-Indication Curves for CT and LP.

Test Indication Curves with 95% CI natural scale in panels A through D. Diagnostic accuracy of computed tomography or lumbar puncture for subarachnoid hemorrhage. Raw test-indication curves as proposed by Bernstein are shown, providing a graphical representation of the Bayesian post-test probability (y-axis) based on either a positive (upper curved black line, with 95% confidence intervals in grey) or negative (lower curved lines) test result as a function of the pre-test probability (x-axis). The graphs use the same principle as the Fagan nomogram, but provide more intuitive representations of the diagnostic accuracy of a test. The four tests considered are cranial CT obtained within 6 hours (panel A) or later (panel B) from headache onset, or LP with more than 1,000×10⁶/L erythrocytes (panel C) or visible xanthochromia (panel D). The distance of the curves from the main diagonal of zero diagnostic information provide a visual representation of the information gained from the test result.

Test-Treatment Threshold

The risks of treatment (designated R_rx by Pauker and Kassirer⁴⁷) were primarily those ascribed to angiography itself, and not those of eventual neurosurgical intervention. These risks included additional exposure to ionizing radiation and to contrast, the risks of vascular access and allergic reactions, and the remote risk of stroke or other end-organ damage. The life-time risk of malignancy secondary to exposure to ionizing radiation is a function of dose and age; an adult is typically exposed to 20 mGy radiation during cranial CT. Using an average lifetime cancer risk attributable to such an exposure of almost 1%,⁸¹ we deemed the total risks related to angiography to be relatively low at 0.02.

When considering the benefits of treatment (B_rx), we considered primarily the benefits of treating an aneurysmal SAH. Overall, SAH has 12% mortality prior to arrival in the ED, and 40% in-hospital mortality at one month despite treatment. In addition, one-third of survivors suffer a severe neurological deficit.^82,83 Neurosurgical and interventional radiology treatment options include clipping and coiling the culprit aneurysm when present, with several large randomized trials reporting long-term poor outcomes in up to 35% of treated subjects.^84-86 The long-term benefits of early diagnosis and treatment in the subset of patients who present awake and neurologically intact are not well characterized but presumably somewhat better, so we assigned an optimistic overall treatment benefit (B_rx) of 0.8 with immediate diagnosis.

For LP we considered the common morbidity of a post-dural puncture headache, discomfort, and remote risks of infection, and assigned a test risk (R_t) of 0.01. The testing thresholds from these estimates of potential risk and potential benefit are depicted in Figure 7. The thresholds bracketing indifference between LP versus either no further testing (T_test) or angiography (T_treat) were very narrow, suggesting that LP should only be considered for patients with a pre-LP probability between 2% and 7% (when using visible xanthochromia) or between 2% and 4% (when using CSF erythrocyte count). Lower risk patients need no further testing and avoid LP, while higher risk patients benefit from angiography and again avoid LP.

Figure 7. Test-indication curves illustrating when benefits of testing outweigh the risks for subarachnoid hemorrhage.

The complete test-indication curves as proposed by Bernstein are shown which incorporate the Pauker-Kassirer threshold approach to deciding whether to perform the test in question. This technique compares the risks versus benefits for treatment (in this case proceeding to CT or formal angiography), and calculates the corresponding treatment threshold (shown as horizontal dashed lines, with the point estimate in black and the upper and lower bands of the sensitivity analysis in grey). The vertical line dashed line (or rightmost line when two are visible) therefore represents the pre-test probability range below which the diagnostic test might be appropriate, assuming the test in question had neither risks nor costs; when the pre-test probability is higher, empirical treatment is recommended since the post-test probability exceeds this threshold even if the test is negative. Because tests have a non-zero risk, the actual pre-test range for which performing the test is rational is narrower, and is shown by the thick line between the arrowheads. The thinner solid line extending beyond the arrows illustrates the effect of reducing the test risk by half. At pre-test probabilities to the left of this range, no further testing is indicated. For example, this approach suggests proceeding directly to angiography when the pre-CT probability exceeds 10% (panel B), unless the unenhanced CT can be obtained within 6 hours (panel A) in which case a pre-test probability of nearly 70% seems appropriate before proceeding directly to angiography. On the other hand, the use of CT is warranted even in very low risk patients, down to perhaps 0.7% pre-test probability as determined primarily by the risks of the CT itself. The LP curves, however, illustrate that the pre-LP probabilities that justify performing LP are very narrow, ranging from 2% to 4% in panel C and 2% to 7% in panel D. Moreover, these pre-LP probabilities in turn can only arise after an implausibly high pre-CT probability (>>20%).

Testing Thresholds

Of note, these rather narrow pre-LP probability bands correspond to the post-CT disease probability, not the original estimate based on history and physical examination (i.e. not the pre-CT estimate). In other words, one should take the pre-CT probability estimate and incorporate the information gained from a negative CT in a Bayesian fashion. The impact of a negative CT on the post-CT (i.e. pre-LP) probability is also shown in Figure 7, depending on the interval between headache onset and imaging. Thus, a pre-LP probability of 2% to 7% represents a pre-CT prior probability of over 70% (based on history and physical) for a CT obtained within 6 hours, and over 20% for a CT obtained beyond 6 hours. The sensitivity analyses showed that these estimates were stable across a range of utilities and were primarily influenced by the diagnostic accuracy of the tests being considered, as illustrated in the Figures. When the risks of testing (LP) were allowed to approach the risks of treatment (angiography), then LP was no longer indicated at any pre-test probability. Table 6 provides an example illustrating how to use Test Indication curves for diagnostic decision-making.

Table 6.

Using Test-Indication Curves

You have just seen a 48-year-old housewife who presents to the ED by private vehicle 8 hours after the onset of a severe headache which peaked within 10 minutes of onset. She is nauseated and complains of neck stiffness but her neurological exam is intact and she is able to fully flex her neck. You assign a pre-test probability of 10% for SAH, and await the final read by the neuroradiologist on duty even though the imaging study appears to be unremarkable to you and to the CT technician. The test treatment curves can be used like the Fagan nomogram to calculate the post-test probability of disease. Therefore you can estimate that the post-test probability will be around 0.8% should the study prove negative (Fig TIC panel B). You can also estimate that, if you had performed an LP rather than CT, the post-test likelihood would be around 2% if fewer than 1000 × 10^6/L RBC were present in the final tube of CSF (panel C), and similarly 2% if there were no visible xanthochromia. But you can also “chain” sequential test results, with the usual caveat that the tests may not be perfectly independent. Thus, after the neuroradiologist confirms that there are no signs of SAH on the CT, the same patient would have her probability of disease lowered from 0.8% pre-LP to about 0.1% after an LP negative for xanthochromia (panel D). Moreover, decision analysis would suggest that, for this patient, the risks of LP outweigh the potential benefits, since the pre-test probability of 0.8% is already below the range when LP is beneficial whether one uses erythrocyte count or xanthochromia, as indicated by the horizontal arrows on the test indication curves.

Discussion

The results of this systematic review demonstrate that neither the presence nor the absence of commonly cited SAH risk factors from history and physical examination accurately rule-in or rule-out this potentially lethal diagnosis. Specifically, no single history or physical exam finding significantly increases (LR+>10) or decreases (LR− < 0.1) the post-test probability of SAH for severe headaches that peak within one hour of onset. In addition, many elements of history and physical exam for SAH have only fair to good inter-physician reliability, with the characterization of the headache as “thunderclap” being one of the least reproducible findings.³⁹ Nonetheless, many physicians would prefer to never miss a SAH, and missed SAH remains an important cause of litigation in emergency medicine.^23,87

Lacking accurate features on clinical presentation and recognizing the tension between malpractice risk and a growing imperative to preserve limited healthcare resources, to adhere to Choosing Wisely recommendations,^23,87,88 and to avoid the unintended harms of over-diagnosis,³⁰ clinical researchers have endeavored to develop more efficient prediction instruments. Yet the development of highly sensitive instruments requires both large datasets and rigorous adherence to methodological standards. To illustrate, a retrospective validation of Rule 1 using medical record review in Northern California reported a LR− of 0.13 (95% CI 0.03-0.61) and reportedly missed 11 cases (20%) of SAH cases despite cranial CT within 6 hours.⁸⁹ Another retrospective study of aneurysmal SAH patients demonstrated 100% sensitivity (95% CI 97.6%-100%) using the combination of negative cranial CT and all negative responses on the Ottawa SAH Rule.⁹⁰ However, validating a clinical decision rule retrospectively is sub-optimal since all components of the rule may not be recorded in the medical records and whether to code missing variables as unknown or not present is frequently a question.⁹¹ In the retrospective validation of Rule 1, the “onset during exertion” variable was missing in one-third of patients.⁹² Therefore, these instruments should ideally undergo prospective validation in different settings before widespread adoption.^91,93

Unenhanced CT remains the first-line imaging test in ED headache patients with suspected spontaneous SAH. Within 6 hours of headache onset, CT demonstrates sufficient accuracy to rule-in or to rule-out SAH, notwithstanding the heterogeneity of the estimated specificity and LR+ of the two studies identified. Beyond 6 hours the estimated LR− (95% CI 0.01-0.61) and sensitivity (95% CI 83%-93%) become less precise and concerns regarding heterogeneity (I² 89% and 63%, respectively) allow the interpretation that the accuracy of cranial CT to rule-out SAH could range from very good (LR− 0.01) to unhelpful (LR− 0.61) taking the extremes of the 95% confidence interval. Therefore, the timing of CT relative to symptom onset has remained an important qualifier when estimating the diagnostic accuracy of the CT. Of course, CT can identify other causes of headache or other abnormalities, and similarly can miss SAH due to cervical arteriovenous malformations.⁷⁵ Some researchers in the field stipulate that the CT should be interpreted by an experienced neuro-radiologist, although recent research indicates that radiologists at non-academic centers may be equally adept.³⁵

Despite advances in CT imaging, substantial debate persists surrounding the necessity of LP following a negative head CT, particularly within the first six hours after onset of headache.^94-98 While CSF can provide crucial diagnostic information for alternative etiologies of headache like meningitis, it remains an imperfect test for SAH. Traumatic (blood-contaminated) LPs occur in 1 of 6 cases.⁴⁰ Xanthochromia takes many hours to develop, and is variably defined. In addition, post-LP headaches are common^31,99 despite increasing use of smaller gauge, atraumatic spinal needles and re-inserting the stylet before needle removal.³² Other potential complications include low back pain, brainstem herniation, iatrogenic infection, epidural nerve injury, and spinal hemorrhage.^100,101 These factors combined with patients’ negative bias against the procedure and the physician time needed help explain emergency physicians’ reluctance to pursue LP despite a long-held belief that this procedure was mandatory in the setting of suspected SAH.^102-105 Our analysis provides new insight into the limited utility of performing LP to rule out SAH after a negative CT. We found that, by incorporating the risks and benefits of both testing and continued investigation into the decision analysis, the benefits of LP are unlikely to outweigh the harm across a wide range of reasonable estimates of pre-test disease prevalence and given the limited diagnostic accuracy of the CSF analysis.

Brunell et al. noted that LP provided an alternative diagnosis in 3% of suspected SAH cases, but only altered subsequent management in 0.44% of individuals who had an LP.³⁴ This equates to 227 LPs (1/0.0044) to identify one central nervous system infection requiring antibiotics in suspected SAH patients. In addition, using spectrophotometry, Brunell et al. noted five additional cases of SAH representing 1.1% of LPs and a Number Needed to LP (NN_LP) of 91 (1/0.011) to identify one additional SAH. Of note, none of these additional cases of “SAH” were aneurysmal, and none underwent subsequent surgery; thus the NN_LP approaches infinity for detecting SAH that might benefit from intervention. Perry et al. reported 17/1546 false negative CTs (with post-headache onset imaging delays ranging from eight hours to eight days) that were identified by LP obtained at the discretion of the attending emergency physician.²⁰ However, only six of these individuals underwent neurosurgical intervention (ventricular drain, aneurysm coiling, or clipping), which equates to a NN_LP = 258 (6/1546 = 0.38%, 1/0.0038 = 258) to identify one aneurysm amenable to neurosurgical intervention following a negative cranial CT. Sayer et al reported NN_LP = 250 to identify one additional aneurysmal SAH.³⁶ Blok et al identified one perimesencephalic bleed amongst 760 CT-negative patients with suspected aneurysmal SAH, which they extrapolated to NN_LP = 15,200 to identify one additional aneurysmal SAH given that 1 in 20 perimesencephalic bleeds are ultimately linked to an aneurysm (760 × 20 = 15,200).^35,106-108

Another approach to more definitively rule-out SAH after a non-diagnostic, non-contrast CT is to proceed to CT angiography (CTA) without an LP, an approach that 75% of patients favored in one survey^109,110 and supporting a role for shared decision-making in evaluation of suspected SAH.¹¹¹ Some physicians have even advocated for pre-discharge CTA after a non-diagnostic CT and LP. Carstairs et al examined CTA in addition to traditional non-contrast CT. Patients that were CT negative and LP negative underwent CTA to more definitively identify either SAH or cerebral aneurysm.¹⁷ After evaluating 103 CT-negative/LP-negative patients, they detected two cerebral aneurysms, which equates to a Number Needed to CTA (NN_CTA) = 52 to detect one cerebral aneurysm that might or might not be the cause of the presenting headache (2/103 = 1.9%, 1/0.019 = 52). Contemplating the NN_CTA is essential because CTA is neither completely safe, nor always clinically meaningful. CTA complication rates range from 0.25%-1.8%.^112,113 Carstairs et al. interpreted the presence of cerebral aneurysm by CTA to be equivalent to being disease positive for SAH, despite being unable to establish whether the aneurysm was the etiology of the patient’s headache or an incidental finding.

Indeed, intracranial saccular or berry aneurysms are common, occurring in 1%-2% of the population.¹¹⁴ Linking a non-ruptured aneurysm detected via CTA to an individual patient’s headache is problematic. The discovery of an aneurysm in the context of headache is often considered to be a symptomatic aneurysm, and neurosurgical intervention may be advised depending on aneurysm size, location, and specific patient characteristics.¹¹⁵ However, the relative risk of rupture for cerebral aneurysms detected in symptomatic individuals is 8-fold higher than the rupture rate in asymptomatic individuals with incidentally noted cerebral aneurysms.¹¹⁶ Based upon one recent meta-analysis of 50 studies, the pooled sensitivity and specificity for CTA to detect aneurysms are 98% and 100%, respectively.⁴⁴ MRA can also be utilized to detect intracranial aneurysm, with pooled sensitivity and specificity of 95% and 89% respectively. Although MRA is more time-consuming and generally less available from the ED, it may be useful in circumstances where radiation exposure is to be avoided or a patient has an iodine contrast dye allergy.⁴⁵

Given the risks of over-investigation and incidental findings, it is important to scrutinize the testing thresholds for CT, LP and angiography. We found that only patients with a pre-CT likelihood of SAH well above 20% were likely to benefit from an LP after a negative unenhanced CT. Such pre-CT probabilities are exceptionally high and are far higher than the average diagnostic yield of around 5% for SAH on initial CT. The limited ability of any feature on history or physical examination to identify SAH also renders such high pre-CT probabilities unlikely, except for severe cases likely to have extensive bleeding visible on CT. Moreover, patients with only slightly higher pre-CT probabilities benefit more from angiography than LP, given the limitations of CSF analysis. Accordingly, this analysis suggests that LP after a negative CT to “rule out” SAH no longer seems rational for the large majority of patients.

The utilitarian approach to decision making test thresholds is considered by some too arbitrary or artificial, yet it is critically important to recognize how to apply these analyses at the bedside. The separation between the test positive and test negative curves are a visual representation of the diagnostic performance of the test, and thus illustrate the summary findings from this meta-analysis. The vertical location of the horizontal line that is used to calculate the test thresholds where it intersects each curve is determined by the ratio between benefits and risks of further treatment, and can easily be shifted up or down to visually estimate new thresholds on the graph. We used a 40:1 ratio of the benefits:risk of angiography, but allowed this to range from 20:1 to 80:1 in a sensitivity analysis and obtained similar findings. We extended this graphical approach to include the non-zero risks of testing, as illustrated by a narrowing of the range of pre-test probabilities for which the test is beneficial. We selected a test:treatment risk of 1:4 for CT and 1:2 for LP compared to angiography, but reduced the risk by half in another sensitivity analysis with again similar findings.

Yet the most compelling argument in support of this analysis is that the findings are strikingly similar to experienced clinicians’ own practice. Initial unenhanced CT is rational across a wide range of pre-test probabilities, as low as 1%. This 1 in 100 miss rate is consistent with surveys of emergency physicians and a threshold used in clinical decision rule construction for SAH and other serious conditions.^38,104 On the other hand, after a negative CT, the low LR− coupled with a low pre-CT probability for SAH renders the post-CT probability so low that the only moderately accurate LP is rarely warranted. As such, the observed practice of forgoing LP after a negative CT, long considered to be a pitfall in the management of these patients, is easily justified on Bayesian grounds, as represented graphically on the test indication curves.

The test-indication curves also help clinicians avoid the common fallacy sometimes invoked to persuade headache patients that an LP is advisable. Many explain a “95% sensitivity” of CT by counseling the patient that “there is still a 1 in 20 (i.e. 100% - 95% = 5%) chance that you have a ruptured aneurysm”. This is only true if the patient were certain to have a SAH prior to the test being ordered (pre-test probability 100%). It is implausible that even the most skilled clinician can predict SAH prior to any testing in more than 1 in 5 awake and alert patients, the very patient in whom an informed consent discussion is needed prior to LP. Our meta-analysis confirms that there are few findings on history or physical exam with a sufficiently strong association with SAH, and clinical decision rules suffer from modest to poor specificity precisely because of this difficulty identifying disease positive patients prior to testing. Arguably, if one’s own diagnostic yield for SAH on initial CT approaches 20% in awake headache patients, one might expect one’s miss rate may be unacceptably high. Moreover, such high-risk patients benefit from proceeding directly to CTA especially if the unenhanced CT cannot be done within 6 hours of headache onset, also as indicated on the test threshold curves. On the other hand, using a more reasonable pre-CT probability of 1 in 10 (still higher than the 7% prevalence in prospective studies of emergency patients with headache), a test with 95% sensitivity and perfect specificity implies that the patient being counseled actually has only a 1 in 180 chance of a ruptured aneurysm (post-test odds = pre-test odds × LR− = 1:9 × 0.05).

A cost-effectiveness analysis comparing CT-only, CT/CTA, CT/LP, and CT/MRA as diagnostic modalities found that CT-only and CT/LP are the preferred strategies of diagnosing ED patients with suspected SAH.¹¹⁷ The CT/LP strategy was slightly more costly, but more effective due to the LP reducing the possibility of a missed diagnosis and subsequent harm. However, since the results of this meta-analysis indicate a much lower threshold for LP than used in the sensitivity analyses of the cost-effectiveness analysis, an updated cost-effectiveness analysis would likely shift the preference towards a CT-only strategy by decreasing the effectiveness of the CT/LP strategy. Further, this cost-effectiveness analysis did not consider crowding in the ED or physician and patient reluctance towards LP, which are practical barriers to the implementation of a CT/LP strategy.

Implications for Future Research

LP has long been considered to be the criterion standard for diagnosing SAH as it may detect small amounts of xanthochromia or blood in the CSF missed by CT. Due to the advancement of multislice CT, sensitivity to detect SAH has increased substantially. The primary clinical question for emergency physicians remains whether LP is indicated to rule out SAH if head CT is negative. Recent literature suggests within 6 hours of symptom onset sensitivity of CT approaches 100%. Additional studies in heterogeneous settings (rural, non-North American, community hospitals) are needed to confirm this diagnostic accuracy.

The significant inter-study heterogeneity observed in our meta-analysis indicates that more research is needed to quantify the diagnostic accuracy of history and physical exam, as well as visible or spectrophotometric assessment of xanthochromia for the diagnosis of SAH. Unfortunately, few SAH studies adhere to the Standards for Reporting of Diagnostic Accuracy (STARD) reporting guidelines.¹¹⁸ Failure to use reporting standards makes diagnostic studies more difficult for clinicians, researchers, and guideline developers to locate, interpret, and compare.¹¹⁹ This is reflected in the poor inter-rater reliability results observed in our QUADAS-2 assessments of blinding and appropriately reproducible conduct of the index tests. In addition, studies that do not use the STARD criteria are less likely to report key details necessary for clinicians to understand the risk of bias and expected skew of observed point estimates. Future diagnostic studies should adhere to the STARD reporting standards¹¹⁸ and history/physical exam studies should follow the STARD criteria recommended for such studies.¹²⁰

Spectrophotometric assessment of xanthochromia should be standardized in terms of conduct and reporting.⁷⁸ Most studies did not obtain an LP in all patients or report any standard follow-up protocol to capture false-negatives. Opening pressures were only obtained in 2% of suspected SAH cases³⁴ and no studies evaluated the diagnostic accuracy of opening pressures. Future studies reporting the diagnostic accuracy of CT should uniformly report the resolution and timing of imaging, while incorporating post-ED follow-up at reasonable intervals including review of death registries for detection of false-negative results. In addition, the volume of CSF blood and rapidity of aneurysmal bleeding probably influence both CT and LP test accuracy, threatening test independence implicit in Bayesian analyses, and therefore merits assessment for interactions. Future diagnostic researchers should also evaluate test performance stratified by age since aneurysmal SAH are rare in young patients. Furthermore, the skew of observed sensitivities and specificities as a result of spectrum bias, partial verification bias, differential verification bias, and imperfect gold standard bias should be contemplated in future studies.⁵¹

A planned secondary outcome of this systematic review was to examine clinically significant SAH cases, defined as requiring neurosurgical intervention. The studies in this systematic review did not report adequately to be able to assess these outcomes. Specifically, perimesencephalic hemorrhage is traditionally considered to be “true positive” in SAH diagnostic research, but has a benign clinical course without specific interventions to alter the natural course of disease other than analgesic symptom control. However, perimesencephalic SAH can recur¹²¹ and some neurosurgeons recommend catheter angiography in suspected SAH patients with xanthochromia after a non-diagnostic CTA since a culprit aneurysm will be identified in 8.3% of these cases.¹²² In clinical practice SAH is deemed perimesencephalic if two catheter angiography studies 7-days apart demonstrate no cerebral aneurysm, since a single catheter angiogram and CTA can miss 4.2% of aneurysms.¹²³ None of the diagnostic studies included in this meta-analysis reported uniform CT or catheter angiograms in all patients, let alone repeat catheter angiograms in those with initially “negative” or non-diagnostic advanced imaging. Future studies examining SAH should distinguish aneurysmal SAH from perimesencephalic hemorrhage to better examine SAH that merits surgical intervention.

Limitations

Identification of studies for this meta-analysis used only one individual with adjudication of uncertainty of inclusion/exclusion criteria by a second author, which could introduce a bias in study selection. In addition, we excluded non-English studies which could have limited the scope of our findings and yield biased summary estimates, although review of interventional meta-analyses do not suggest significant differences when non-English studies are excluded.¹²⁴ Of the 22 studies that provided sufficient detail to be able to reconstruct 2×2 tables, there was significant heterogeneity in the manner in which data were collected and reported. For example, when examining sensitivity and specificity of non-contrast head CT, not all studies reported the specific technology utilized. When technology utilized was reported, it was done so in an inconsistent manner making it unclear which generation (how many slice) CT scanner was used. Variation in technology undoubtedly affects diagnostic test performance, and standardized reporting of the particular technology used should be encouraged in all future studies.¹¹⁸

Significant variation existed in mechanisms of follow-up for patients that were deemed SAH negative. Some studies did not conduct any follow-up, while others had structured telephone questionnaires, examined coroner records and public death certificates to ensure that patients deemed disease free did not later turn out to have SAH. Without follow-up for patients that were deemed SAH negative it is impossible to know that the patient truly did not have a small CT undetectable SAH or cerebral aneurysm when they were discharged from the ED. Finally, because some of the studies included patients who were unconscious or had neurological deficits, the observed estimates of sensitivity and specificity may be influenced by the spectrum of disease severity.⁵³

There was also significant variation in definition of disease, particularly in the incorporation of CSF findings into the definition of being SAH positive. Definition of disease positive ranged from number of red blood cells in the final tube via LP to xanthochromia of CSF. Number of red blood cells was not standardized. Xanthochromia was defined by visual inspection in some studies, and spectrophotometry in others. Xanthochromia via spectrophotometry had substantial variation in the absorption spectrum used to define a positive LP. Other studies used spectrophotometry to detect bilirubin and oxyhemoglobin as indicating a patient with SAH.

Conclusions

This meta-analysis identified a 7.5% weighted prevalence of SAH in ED patients with concerning headache. Although the available high quality diagnostic evidence is limited, there is no single characteristic on history or physical examination that is adequate to rule in or rule out SAH in ED settings. Using a threshold of 1,000 × 10⁶/L RBCs in the final tube of has a summary LR+ 5.7 and LR− 0.21. Within 6 hours of symptom onset, non-contrast cranial CT accurately both rules in and rules out SAH. Given the risks and benefits of further testing and the limited diagnostic accuracy of CSF erythrocyte counts and of various xanthochromia definitions, LP is likely to benefit only patients within a narrow band of pre-LP probabilities, around 2% to 7%. Given the accuracy of CT, even several hours after headache onset, such pre-LP probabilities correspond to pre-CT probabilities of 20% or higher. While many still consider CT followed by LP to be the gold standard for the diagnosis of SAH, few studies use this gold standard given practice patterns evolving away from routine LP. The role of LP may well wane further with improving multi-slice CT and increasing evidence to refute the utility of LP. Validation of SAH clinical decision rules offers the opportunity to more accurately risk-stratify ED headache patients in order to identify subsets most likely to benefit from post-CT LP, CTA, or no further testing.