To Head CT Scan or Not: The Clinical Quandary in Suspected Subarachnoid Hemorrhage; a Validation Study on Ottawa Subarachnoid Hemorrhage Rule

Abstract

Introduction:

The Ottawa Subarachnoid Hemorrhage rule (OSR) is a clinical decision tool identified for ruling out subarachnoid hemorrhage (SAH) in those patient above 15 years of age who present to the emergency department (ED) with acute onset atraumatic headache.

Objective:

The primary objective of this study was to externally validate the OSR in a single national health service (NHS) setting in the UK and secondly, to compare it with our current practice without using a decision rule.

Method:

A retrospective review of computerized medical records was done for all patients registered with headaches from January to December 2016. The data were manually charted on a data sheet from individual patient records. Patients fulfilling the preset inclusion and exclusion criteria as per the OSR were enrolled in the analysis. According to the OSR, if patient had any of the 6 criteria enlisted (age > 40 years, neck stiffness/pain, witnessed loss of consciousness, onset during exertion, thunderclap headache, limited neck flexion on examination), further diagnostic decision was required. All patients were followed up for 6 months on the computer system as it gets highlighted if the patient is represented again to the ED or is deceased.

Results:

A total of 737 ED visits with acute headache were reviewed for potential eligibility. Out of these, 649 were estimated to be eligible. On excluding 485 patients that could not meet the predetermined inclusion criteria and 19 patients as per the exclusion criteria, 145 (19.7%) patients were included in the analysis. There were 5 cases of SAH, yielding an incidence of 3.4 % (95% CI 1.3 % – 8.3 %). The sensitivity for SAH was 100% (95% CI, 46.3 % - 100 %); specificity of 44.2 % (95% CI, 36 % - 53 %); positive predictive value of 6.02 % (95% CI 2.2 % - 14.1 %); and negative predictive value of 100% (95% CI, 92.7 % - 100%).

Conclusion:

Although being poorly specific, the OSR is a highly sensitive, simple tool for ruling out SAH in alert patients with a headache in ED settings.

INTRODUCTION

Headache is one of the most ancient and common presenting complaint that has intrigued medical practitioners from prehistoric times and continue to pose a diagnostic challenge to the current-day. They account for 1-4% of emergency department (ED) visits globally and of these, 1-3% present due to a subarachnoid hemorrhage (SAH) (1-4). With an average SAH prevalence of 7.5%, atraumatic SAH has an annual incidence of 1 to 2.5 per 10000 in the general population (5, 6). While pertinent clinical findings such as neurological deficits accompanying a severe, sudden headache can make the diagnosis and management of patients with a SAH obvious; half of all patients with a SAH present without any neurological deficits (4). The absence of a fail-proof clinical rule to help rule out an underlying neurological pathology in patients presenting with headache raises the spectre of missed diagnoses, and can subject the patient to costly avoidable procedures. The estimated rates of missed diagnosis range between 5% to 12%, and can result in catastrophic morbidity and death (3, 4). Overall mortality with SAH is about 25% in 24 hours and 50% within six months. Of those who survive, 42% suffer neurological deficits (6).

The sensitivity and specificity of using a non-contrast computed tomography (CT) scan followed by a lumbar puncture (LP) in those with negative scan results are 100% and 67% respectively (3). The sensitivity of the CT is higher if done within 6 hours of onset of headache (5). Typically, if the scan result is negative, or if the ED visit was significantly delayed, these patients often undergo a LP to rule out SAH. It is estimated that out of 90 LPs performed following a negative CT scan, only one turns out to be positive for SAH (3). In addition, diagnostic testing results in a prolonged hospital stay, increased cost burden, poor patient flow, and a potentially unsatisfactory patient experience (7).

Due to the variable etiology of SAH, there is no single clinical characteristic that could reliably exclude a clinical diagnosis of SAH (4, 5). Patient’s description of the pain as “the worst headache of life” is subjective and hence unreliable; and headaches that got worse gradually over greater than an hour are less likely (LR= -0.06) to be due to SAH (8). The complexity of diagnosing non-traumatic SAH in neurologically intact patients had inspired a search for variables strongly associated with the diagnosis of SAH. Ideally, only those patient with high-risk features should be subjected to further diagnostics, but there were no set criteria to safely and rigorously stratify these high-risk individuals until the first study came out in 2010 (4). In one study published in 2005, Perry et al evaluated the attitudes and judgment of ED physicians in managing patients with an acute headache. They found ED physicians moderately distinguished SAH from other types of headaches clinically, and a large number of patients underwent imaging followed by an LP. They concluded that a clinical-decision rule could standardize the clinical judgment, improve physician comfort without ordering diagnostic tests in low-risk patients, and improve the efficiency of testing without missing any cases of SAH (6). Surveys conducted among ED physicians suggested a decision rule with 100% sensitivity for SAH would significantly help in delivering cost-effective and less invasive care (9). After series of analyses that looked at high-risk variables for SAH, Perry et al in 2010 published 3 Canadian Clinical Decision Rules for SAH (CDR) (Appendix 1) (4). These rules were further refined to generate the Ottawa Subarachnoid Hemorrhage Rule (OSR) in 2013 which can safely rule out SAH in an alert patient with headache (Appendix 2) (2).

As with any other clinical decision rule, before implementation of the OSR, it is essential to validate in the external setting. Through this study, an attempt to externally validate the rule in a single national health service (NHS) setting was done. In addition, it presents a comparison of the use of OSR against our current practice to analyze the number of interventions performed.

METHODS

Study design

A retrospective review of computerized medical records of all patients registered with a headache at the Luton & Dunstable NHS University Hospital ED, an academic district general hospital with 90000 – 100000 annual visits located in Luton, United Kingdom. It provides a full range of acute services to the South Bedfordshire and West Hertfordshire areas. The retrospective analysis of these patients’ record was approved by the ED clinical governance committee.

Study population

All patients registered with a primary complaint of a headache from 1st January 2016 to 31st December 2016 were identified. Age older than 15 years, new atraumatic headache, and headaches that reached maximal intensity in 1 hour were considered as inclusion criteria; and absence of any new neurological deficits, prior diagnosis of cerebral aneurysms/SAH/brain tumors, and those with recurrent headaches in last 6 months were the exclusion criteria in this study.

Data gathering

Manual review of computerized medical records (Evolve & ICE) with patient’s hospital registration number was performed. Evolve & ICE are electronic health record programs used at the Luton & Dunstable NHS Trust for maintaining patient’s records. Evolve stores all entries that are scanned from paper documentation and ICE is utilized for electronic reporting and to provide continuity of care. The demographic data included patient’s registration number, age, and sex were recorded on a data sheet. The data sheet also included features mentioned in the inclusion criteria and exclusion criteria. Further, the six clinical features of OSR were charted on the data sheet (Age > 40 years, Neck Pain/Stiffness, Witnessed loss of consciousness, Onset during Exertion, Thunderclap headache, and Limited Neck flexion). The data sheet also included information about whether a diagnosis of SAH was considered in our provisional workup, and whether a CT scan or a LP was performed. The imaging and laboratory results were examined to look for a final diagnosis. The definitions for diagnosis of a SAH was subarachnoid blood visible on a plain CT film or xanthochromia in the cerebrospinal fluid. The data was obtained after six months from 31^st December 2016 on the Evolve system which automatically highlights patient’s that are deceased through the patient’s NHS number.

Statistical analysis

The Data was analyzed using 2 × 2 contingency tables plotted on the Microsoft Excel Program. The measures of diagnostic accuracy (sensitivity, specificity, positive predictive value (PPV) and negative predictive value(NPV)) were calculated for the OSR and for our current clinical practice for comparison. The 95% confidence intervals (CIs) including continuity correction were calculated using an online tool (Vassarstats) that deploys the Robert Newcombe method.

RESULTS

The record yielded a total of 837 patients registered with a headache (Figure 1). 100 records (12%) were eliminated from the analysis as they were either misclassified as a headache (9 patients) or were less than or equal to 15 years of age (91 patients). 737 patients (88%) were deemed potentially eligible before applying the preset inclusion and exclusion criteria as per the OSR. 88 patients (10.5%) were considered missed eligible for having incompletely documented records (73 patients were missing the documentation related to the progress of pain), and 15 patients had left without being initially assessed. The inclusion criteria as per the OSR were alert patients with a new-onset, atraumatic headache that reached maximal intensity in an hour. 485 patients (58%) were excluded as they did not meet the inclusion criteria. The OSR exclusion criteria were patients with any new neurological deficits+, or with a history of prior aneurysms, prior SAH or brain tumors; and any history of recurrent headaches (> 3 headache episodes in last 6 months). After excluding further 19 patients (2%) as per the exclusion criteria, 145 patients (17.3%) with the mean age of 42.2 ± 8.67 years old were enrolled in the final analysis. Two patients returned to the ED within 14 days and none of these had a diagnosis of SAH. The demographic data of included patients were reported in table 1.

Figure 1.

Flow diagram of patient selection in current study

Table 1.

The demographic data of the patients (n=145)

Variable	Number (%)
Sex
Male	54 (37.3)
Female	91 (62.7)
Age (year)
< 40	79 (54.5)
> 40	66 (45.5)
History findings
Witnessed loss of consciousness	1 (0.7)
Onset during exertion	4 (2.8)
Thunderclap headache	16 (11.0)
Physical exam findings
Neck Pain/Stiffness	12 (8.3)
Limited neck flexion	6 (4.1)
Conducted procedures
Head CT scan	108 (74.5)
Lumbar puncture	46 (31.7)

There were 5 cases of SAH, yielding an incidence of 3.4 % (95% CI 1.3 % – 8.3 %). Head CT scan and LP was performed in 87 and 35 patients respectively. The 2 by 2 contingency table was generated for both the OSR (Figure 2) and our current clinical practice (without any rule) (table 2). The sensitivity for SAH using the OSR was 100% (95% CI, 46.3 % - 100 %); specificity of 44.2 % (95% CI, 36 % - 53 %); positive predictive value of 6.02 % (95% CI 2.2 % - 14.1 %); and negative predictive value of 100% (95% CI, 92.7 % - 100%). Using the OSR, there were 62 patients that required no further investigations (true negatives). 5 of these underwent CT imaging, and 3 underwent lumbar puncture with a negative result for SAH. The OSR predicted further workup in 83 patients with either a head CT + LP (12.8 % of eligible 649 patients).

Figure 2.

Recommended approach to atraumatic headache in alert patients in ED

Table 2.

Comparison of results of Ottawa subarachnoid hemorrhage rule with diagnosis of SAH in studied patients

		Subarachnoid hemorrhage
		Present	Absent
Ottawa subarachnoid hemorrhage rule	Positive	5	78
	Negative	0	62

By our current practice (without a clinical decision rule) the sensitivity was 80% (95% CI, 30% - 99%) with a specificity of 85% (95% CI 82% - 87%). The PPV was 3.5% (95% CI, 1.1% - 9.4%) and the NPV was 99.8% (95% CI 99% -100%) (table 3).

Table 3.

Comparison of results of current clinical practice with diagnosis of SAH in studied patients

		Subarachnoid hemorrhage
		Present	Absent
Current clinical practice	Suspected	4	108
	Not suspected	1	624

Head CT scan and LP was performed in 108 (+1 offered but patient refused) and 46 (+3 offered but patient refused) patients respectively. We had missed 1 case of SAH which was initially thought to be migraine refractory to parenteral treatment and was referred to the medical team, who then requested a CT scan that revealed a SAH. However, the same case was positive for a SAH by applying the OSR. Compared to our current practice, the OSR showed promise in reducing the number of CT scans and LP too. By our clinical practice, we had requested 108 CT Scans (in addition one was offered and the patient did not consent) and with the application of OSR, we would have only done a head CT for 83 patients (if all 83 were deemed necessary to be scanned for a diagnosis). Similarly, we would have done far lesser invasive LPs by deploying the OSR as compared to our clinical practice which was done for 46 patients. In all 5 cases of diagnosed SAH, an LP was not required.

DISCUSSION

Although based on the available evidence, the current practice of CT followed by an LP remains the most cost-effective approach to diagnose a SAH (10), the OSR is highly sensitive in clinically ruling out SAH. Based on the findings, it is likely that the OSR, although nonspecific, but is highly sensitive in clinically ruling out SAH. In return, current practice of our ED physicians is not sensitive enough to rule out possible SAH cases. Compared to our current practice, the OSR showed promise in reducing the number of CT scans and LP too.

Literature review

As described earlier, in a prospective, multi-center study in Canada during 2000 - 2005, Perry et al (2010) derived high-risk variables that showed strong association with SAH and three clinical-decision rules (CDR) for SAH were proposed (4). These rules gave a sensitivity of 100% but had poor specificity. Some variables mentioned in these rules had a poor inter-rater agreement. Hence, the authors proposed using a composite of seven variables from all the three rules to stratify high-risk individuals for SAH until further prospective external validation of these rules was carried out (4). Perry and colleagues (2013) in another prospective study further refined the previous CDR by adding two additional variables and coined the OSR (2). The OSR gave a high sensitivity of 100% (95%CI 97.2%-100%), but again a poor specificity of 15.3% (95%CI 13.8%-16.9%), and just above acceptable C statistic of 0.60 (95%CI 0.59-0.61) (2). In conclusion, the authors determined that the application of OSR could help in reducing missed SAH, and possibly standardize the workup requirements in acute headache ⁽²⁾. The OSR yielded a sensitivity 100% in yet another prospective cohort examined by Perry et al. (2014) (11) in Canada during 2011 -2013. Newman-Toker and Eldow (2013) (12) suggested that the use of OSR may even be useful in situations where there are barriers to the application of CT-LP combination for diagnosing SAH (12). A retrospective validation of the OSR in an academic ED setting in the United States done by Bellolio et al. (2015) showed 100% sensitivity and a 100% NPV (1).

Recommendations

The data analysis at the Luton and Dunstable NHS Hospital proved that the OSR can be an ideal tool with 100% sensitivity, and in addition, would have reduced the number of patients subjected to further investigations. While analyzing the data, it was found that many junior doctors (who may lack the experience to make a clinical diagnosis) approached a senior clinician when they were concerned about SAH, and the senior colleagues safely discharged many of these patients without further work-up. The OSR could be a useful decision tool especially for junior ED physicians to safely rule out SAH clinically. A recommended approach to the OSR application in atraumatic headache patients suspected of SAH is given in figure 2. This approach would not only help the junior doctors to confidently make decisions for excluding SAH clinically, but also may be useful in reducing the avoidable interventions that would lead to increased costs, the length of stays, and procedural morbidities. Implementation of the algorithmic approach would involve education of physician colleagues in the ED, and continuous data gathering from patient medical records. A potential caveat which may be encountered is that a clinicians’ clinical gestalt may supersede the clinical-decision rule, and non-adherence to the rule which may lead to over or under investigations. Another potential limitation is some physicians may overlook the inclusion and exclusion criteria when applying the rule, which may lead to diagnostic inaccuracies.

Further prospective validation of the OSR in NHS settings would be recommended. In our analysis, it was found that all 5 cases of SAH were diagnosed radiographically. The LPs done after a negative CT were found to be futile. A decision rule on who receives an LP following a normal CT would be recommended (13).

Limitations

Being retrospectively audited, the data contained cases with incompletely documented variables, and these cases had to be eliminated from the analysis. Computerized registries with a primary symptom of a headache were audited. Often, patient registrations are done by non-medical front desk staff, and there could be a possibility of missing some cases that may have arrived with other symptoms in addition to headaches. There were 7 patients who were found to be deceased during the analysis, but their cause of death was not verified for confidentiality purposes. All 7 patients had a concomitant terminal illness and their cause of death may not be primarily due to a SAH. The current status of 18 patients in the audit without an NHS number is unknown. In addition, the incidence of SAH was 3.4% as compared to the primary study which was 6% done by Perry et al². Only 5 cases of SAH were identified in our study population over a year, which could mean that the sample is too small to conclude a significant impact of the rule at the current stage, and may need a larger cohort to be examined.

CONCLUSIONS

Currently, many authors are recommending different additional variables in the original OSR, and are proposing different rules. Until further refined tools are available, the OSR remains a highly sensitive, simple tool for ruling out SAH in alert patients with a headache in ED settings. A future prospective validation of this rule may present with its potential impact on improved safety and cost-effectiveness in managing such patients in our NHS settings.

Appendix 1

Canadian Clinical Decision Rule

Rule 1:

• Age > 40

• Complaint of Neck Pain or stiffness

• Witnessed loss of consciousness

• Onset with exertion

Rule 2:

• Arrival by Ambulance

• Age > 45

• Vomiting at least once

• Diastolic Blood Pressure > 100 mm Hg

Rule 3

• Arrival by Ambulance

• Systolic Blood Pressure > 160 mm Hg

• Complaint of neck pain or stiffness

• Age 45 – 55

For each rule, patients should be investigated if one or more of the variables are present.

Appendix 2:

Canadian Clinical Decision Rule

In patients with acute headache of recent onset (<14 days), without history of trauma or fall in the last 7 days, having no neurological signs, and fulfilling the strict inclusion and exclusion criteria.

Inclusion Criteria:

1. Alert patients > 15 Years of age,

2. New severe atraumatic headache, &

3. Maximum intensity within 1 hour.

Exclusion Criteria:

1. Patients without any new neurological deficits, prior aneurysms, prior SAH or brain tumors; and

2. History of recurrent headaches (> 3 headache episodes in last 6 months)

Variable	Score
Age > 40	+1
Neck pain / stiffness	+1
Witnessed loss of consciousness	+1
Onset during exertion	+1
Thunderclap headache (instantly peaking pain)	+1
Limited neck flexion on examination	+1

With any score of > 1, it is recommended to consider subarachnoid Hemorrhage as a part of our differentials, but owing to its low specificity, it does not necessarily mean that a work up for subarachnoid Hemorrhage should be performed.

AUTHORS’ CONTRIBUTION

All the authors met the standards of authorship based on the recommendations of the International Committee of Medical Journal Editors. ASP was the primary author responsible for study design. EC helped in data gathering and analysis. AT helped with peer reviewing and providing recommendations. All authors reviewed final version and confirmed all of its content.

CONFLICT OF INTEREST

None declared.

FUNDING

No funding was received for this project.