Clinical scoring system may improve yield of head CT of non-trauma emergency department patients

Abstract

Purpose

The positive rate of head CT in non-trauma patients presenting to the Emergency Department (ED) is low. Currently, indications for imaging are based on the individual experience of the treating physician, which contributes to overutilization and variability in imaging utilization. The goals of this study are to ascertain the predictors of positive head CT in non-trauma patients and demonstrate feasibility of a clinical scoring algorithm to improve yield.

Methods

We retrospectively reviewed 500 consecutive ED non-trauma patients evaluated with non-contrast head CT after presenting with headache, altered mentation, syncope, dizziness, or focal neurologic deficit. Medical records were assessed for clinical risk factors: focal neurologic deficit, altered mental status, nausea/vomiting, known malignancy, coagulopathy, and age. Data was analyzed using logistic regression and receiver operator characteristic (ROC) curves and 3 derived algorithms.

Results

Positive CTs were found in 51 of 500 patients (10.2%). Only two clinical factors were significant. Focal neurologic deficit (adjusted OR 20.7; 95% CI 9.4–45.7) and age >55 (adjusted OR 3.08; CI 1.44–6.56). Area under the ROC curve for all 3 algorithms were of 0.73–0.83. In proposed Algorithm C, only patients with focal neurologic deficit (major risk factor) or ≥2 of the five minor risk factors (altered mental status, nausea/vomiting, known malignancy, coagulopathy, and age) would undergo CT imaging. This may reduce utilization by 34% with only a small decrease in sensitivity (98%).

Conclusion

Our simple scoring algorithm utilizing multiple clinical risk factors could help to predict the non-trauma patients who will benefit from CT imaging, resulting in reduced radiation exposure without sacrificing sensitivity.

INTRODUCTION

During the last two decades emergency department (ED) utilization of CT scans has increased dramatically with one in seven emergency department patients undergoing a CT evaluation in 2007. This represents a 14.9% annual compound growth rate in the utilization of ED CTs; an increase of nearly 600% since 1995 (1,2). As a result, there has been growing concern from both public media (3) as well as clinical and research communities (4,5) regarding increasing medical radiation exposure and cancer.

The latest National Council for Radiation Protection report in 2009 on radiation exposure estimates approximately 50% of American annual radiation exposure is due to medical exams, mainly CT imaging (5). Furthermore, cost estimates for unenhanced head CTs performed in EDs within the United States alone are estimated at $6.1 billion annually (6) with a recent study estimating neuroimaging for dizziness alone costs at $470 million annually (7). Even a small improvement in the ability to triage those patients likely to have critical pathology requiring head CT could result in substantial cost and radiation exposure reduction. It is to this aim that developing clinical appropriateness criteria for the use of advanced imaging is considered a “Top Five” policy agenda within the emergency medicine literature (8).

Approximately one-third of the non-contrast head CTs performed in our emergency department are on our non-trauma patients. This population is composed of patients with alteration in mentation such as delirium/drug intoxication, dementia, vertigo/dizziness, and syncope. The studies focused on this patient population have demonstrated relatively poor clinical utility of CT with roughly 10 scans being performed for each clinically actionable positive CT finding (9,10).

To the best of our knowledge, other studies investigating ways to improve CT utilization have mainly focused on trauma. Recently, Yang and You identified 5 clinical risk factors after analyzing the CTs and requisition forms of 3967 patients from 3 academic Emergency Departments. (6). These include the presence of nausea or vomiting, focal neurologic deficit, history of malignancy, known/suspected coagulation profile abnormality, and age (6).

Our objective was to determine the important clinical risk factors described in the literature that may stratify patient’s overall risk for actionable CT results. We hypothesized that our detailed evaluation of the electronic medical record (EMR), such as labs (coagulation profile) and clinical notes would produce accurate determination of risk factors and improve our ability to determine the patients most likely to benefit from CT imaging.

MATERIAL AND METHODS

SAMPLE SIZE DETERMINATION

Based on results of prior studies (6,9,10), we estimated that approximately 10% of CT scans performed on patients in our study population would have positive results. To avoid an over-fitted statistical model, approximately 10 positive results were required for each independent variable entered into the multiple logistic regression model (11). Since 5 independent variables were being evaluated, a total of 500 scans comprised the study samples with the expectation of 50 positive scans.

DATA COLLECTION

The Institutional Review Board approved this retrospective study waived the requirement for informed consent. We used the picture archival and communication system (Centricity; GE Healthcare) and dictation software (Powerscribe 360, Nuance) to conduct a systemic search of consecutive patients 18 years old or older undergoing non-contrast head CT from March 2013 through June 2013 after presenting to the ED with focal neurologic deficit, headache, altered mentation, dizziness, syncope, or pre-syncope. Patients who had a history of trauma (e.g. motor vehicle accident, assault, fall with head trauma) or a known active intracranial or neurologic process (e.g. recent intracranial hemorrhage, recent neurosurgery, ventricular shunt, known intracranial mass/metastases) were excluded. Patients with known malignancy but without known intracranial metastases were not excluded. 500 consecutive patients were selected for inclusion. CT was defined as positive if the following were identified: acute/sub-acute stroke, intracranial hemorrhage, mass, edema or mass effect.

The specific clinical information retrieved from EMR for this study were: patient’s age, sex and clinical evidence indicating 1) focal neurologic deficit, 2) altered mental status, 3) coagulopathy, 4) nausea and/or vomiting, and 5) history of malignancy. The risk factors were assessed from in-depth evaluation of the initial clinical notes written by the ED physician, triage nurse, or physician assistant including vital signs and labs available during the visit. Definitions, of the above factors were kept intentionally broad yet reflect current clinical practice. For example, terms such as dysarthria, aphasia, facial droop or left arm weakness discovered in the EMR were considered meeting criteria of focal neurologic deficit. Coagulopathy included thrombocytopenia, active warfarin/clopidogrel therapy, abnormal international normalized ratio (INR), abnormal partial thromboplastin time (PTT), abnormal prothrombin time (PT), and history of coagulopathic disorder (e.g. sickle cell disease, hemophilia, Factor V Leiden). Altered mental status included any alteration in mentation, including dementia, delirium, transient loss of consciousness, witnessed seizure/syncope, unresponsiveness, and abnormal Glasgow Coma Scale. All cases of nausea and/or vomiting were suspected to be central nervous system (CNS) related and associated with neurologic deficit, headache or dizziness. Results of the head CT were obtained from review of the radiologist’s report. Approximately 95% of CT scan interpretations were performed by subspecialty trained neuroradiologists. The remaining 5% were interpreted by general radiologists or those with other subspecialization. In cases of indeterminate or negative CT results, follow up MRI or CT scans performed during the hospital encounter/admission and subsequent clinical notes were reviewed to substantiate the final outcome. Patients with an initially negative CT but positive follow-up MRI or CT were included as positive results to preserve the utility of the clinical exam and risk factors. A neuroradiologist blinded to the final outcome reviewed these CTs initially interpreted as “negative” to differentiate between a misdiagnosed “false negative” exam and those limited by the sensitivity of CT.

DATA ANALYSIS: STATISTICAL METHODS

To study the association of risk factors and the outcome, we used simple and multiple logistic regression, where association was assessed by odds ratio and its confidence interval. We derived scoring algorithms, informed by regression analysis and evaluated them via ROC curve and standard diagnostic measures such as sensitivity, specificity and positive and negative predictive values. For statistical analysis, SAS 9.3 (SAS Institute, Cary, NC) was used.

Clinical risk factor scoring algorithm, see Table 1

• Algorithm A: ONE Point for each risk factor, AGE excluded

  • 1) focal neurologic deficit, 2) altered mental status, 3) coagulopathy, 4) nausea and/or vomiting, and 5) history of malignancy
• Algorithm B: ONE Point for each risk factor, AGE included

  • 1) focal neurologic deficit 2) altered mental status, 3) coagulopathy, 4) nausea and/or vomiting, 5) history of malignancy and 6) age
• Algorithm C: TWO points for focal neurologic deficit, ONE point for others

  • 1) focal neurologic deficit, 2) altered mental status 3) coagulopathy, 4) nausea and/or vomiting, 5) history of malignancy and 6) age

Table 1.

Different scoring algorithms to predict abnormal head CT in nontrauma ED patients

Clinical Risk Factors	A*	B&	C^
	Value per Factor	Value per Factor	Value per Factor
Focal Neurologic Deficit	1	1	2
Altered Mentation	1	1	1
Nausea ± Vomiting	1	1	1
Coagulopathy	1	1	1
History of Malignancy	1	1	1
Age >55		1	1
Total Score	5	6	7

^*= model of Wang & You (6),

^&= modified Wang & You with age (6),

^{^}=proposed model

RESULTS

Between March and June 2013, 1,528 unenhanced head CTs were performed in patients 18 years or older presenting to the ED. Of these, 879 (57.5%) were excluded due to history of trauma on the requisition. Additionally, 61 cases of trauma (4%) which were not included on the requisition but present in the EMR notes were also excluded. 88 patients (5.8%) with active intracranial processes were also excluded. The final cohort consisted of 500 patients.

Fifty-one of 500 patients had examinations (10.2%) with positive findings, which is consistent with rates reported in the literature (6,9,10,12). Forty-two patients had positive findings on their initial CT. See Table 2 for complete demographics and CT findings. Nine patients initially evaluated by CT had further imaging by MRI and were found to have positive imaging findings. Three of these nine CTs were classified as false negatives after review of the initial CT by a neuroradiologist blinded from the MRI results. The remaining 6 examinations were challenging cases of small infarcts of the posterior circulation: cerebellar peduncle (n=2), pons (n=2), inferior occipital lobe (n=1) and PRES-posterior reversible encephalopathy syndrome (n=1).

Table 2.

Demographics and findings from head CT patients (N=500)

Variable	Mean (SD) or N (%)
Age (in years)	56.9 (18.8)
Female	268 (54)
Abnormal CT Findings	51 (10.2)
Prevalence of Hemorrhage	8 (1.6)
Prevalence of Acute Ischemia	30 (6)
Prevalence of Mass Lesion/Metastasis	6 (1.2)
Other Entities*	7 (1.4)

^*includes 2 cases of posterior reversible encephalopathy syndrome, and 1 case of each of the following nonspecific demyelinating process, obstructive hydrocephalus, aneurysm, septic emboli, severe atheromatous disease with carotid stenosis and vertebral occlusion without acute ischemia

SD indicates standard deviation; N indicates total sample size

Table 3 summarizes the frequency, rate of positive outcome, unadjusted and multivariable-adjusted ORs for the 5 clinical risk factors and age. Focal neurologic deficit was found to be the strongest predictor of abnormality on CT with adjusted odds ratio (OR) >20 (with p<0.0001). Age>55 was the second strongest predictor with adjusted OR>3 (with p<0.004). The presence of coagulopathy showed an OR of 2.5 (CI: 1.36–4.63) in unadjusted regression but 1.6 (CI: 0.77–3.22) in adjusted regression. Nausea/vomiting, altered mentation, and history of malignancy demonstrated increased odds but were not found to be statistically significant.

Table 3.

Frequency and odds ratios of clinical factors in nontrauma ED patients undergoing head CT

Clinical Factor	Frequency (%)	No. of Positive Cases [%]	Unadjusted Odds Ratio* (95% CI)	Adjusted Odds Ratio& (95% CI)
Focal Neurologic deficit	133 (26.6)	42 [31.5]	18.4 (8.6–39.1)	20.7 (9.4–45.7)
Coagulopathy	105 (21)	19 [18.1]	2.51 (1.36–4.63)	1.57 (0.77–3.22)
History of Malignancy	283 (56.6)	31 [11.0]	1.07 (0.46–2.49)	1.11 (0.42–2.94)
Altered Mentation	65 (13)	7 [10.8]	1.21 (0.67–2.19)	1.70 (0.84–3.43)
Nausea ± Vomiting	108 (21.6)	10 [9.3]	0.87 (0.42–1.81)	1.68 (0.70–4.02)
Age > 55	266 (53.2)	39 [14.7]	3.18 (1.62–6.23)	3.08 (1.44–6.56)

(%) indicates percentage of all 500 cases, [%] indicates percentage of positive cases divided by frequency of factor,

^*from simple logistic regression,

^&from multiple regression,

CI indicates confidence interval

The three algorithms were compared at different threshold scores for imaging, summarized in Table 4. In algorithm A, the risk factors were weighted equally but excluded age as a risk factor with a maximum total score of 5. If patients in our study had only been scanned with a risk factor score of 1 or greater, then 100% sensitivity would be maintained and the number of CT scans performed could have been reduced by 62 scans out of 500 (12.4%). However, specificity would only be 13.8% and consistent with the results of Wang and You (6). If only patients with sum score greater than or equal to 2 were scanned, a dramatic improvement in specificity of 63.3% is achieved, but sensitivity would decrease to an unacceptable 70.6%. In algorithm B, the risk factors were also weighted equally but included age as a risk factor with a maximum total score of 6. In algorithm B, the threshold score of 1 or greater would produce 100% sensitivity and 8.7% specificity but would only reduce scans by 7.8%. Similarly, a threshold score of two or greater would improve the specificity to 40% from 8.7%, but sensitivity would remain less than ideal at 88% while reducing the number of unnecessary by 37.4% (187 scans).

Table 4.

Performance of different cutpoints from different algorithms

Model A- Utilizing 5 clinical factors, with AUC of 0.858/0.728
Cutpoint	Number (and Percentage) who Would be Scanned	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value
≥1	438 (87.6%)	100	13.8	11.6	100
≥2	201 (40.2%)	70.6	63.3	17.9	95.0
≥3	49 (9.8%)	33.3	92.9	34.7	92.5
≥4	5 (1%)	7.8	99.8	80.0	90.5
≥5	1 (0.2%)	2.0	100	100	90.0
Model B- Utilizing 5 clinical factors and age >55, with AUC of 0.873/0.755
≥1	461 (92.2%)	100	8.7	11.1	100
≥2	313 (62.6%)	88.2	40.3	14.4	96.8
≥3	144 (28.8%)	64.7	75.3	22.9	94.9
≥4	38 (7.6%)	31.4	95.1	42.1	92.4
≥5	3 (0.6%)	3.9	99.8	66.7	90.1
≥6	1 (0.2%)	2.0	100	100	90.0
Model C- Age>55 added and 2 points to Focal neurologic deficit, with AUC=0.873/0.828
≥1	461 (92.2%)	100	8.7	11.1	100
≥2	329 (65.8%)	98.0	38.9	15.2	99.4
≥3	192 (38.4%)	82.4	66.6	21.9	97.1
≥4	80 (16.0%)	54.9	88.4	35.0	94.5
≥5	27 (5.4%)	31.4	97.6	59.3	92.6
≥6	3 (0.6%)	3.9	99.8	66.7	90.1
≥7	1 (0.2%)	2.0	100	100	90.0

A (Green) : 1 Point assigned for each of 5 clinical factors: focal neurologic deficit, coagulopathy, history of malignancy, altered mentation, nausea and/or vomiting

B (Red) : Same as Model A but 1 point assigned to “Age>55”.

C (Blue): Same as Model B but 2 points assigned for focal neurologic deficit. The first AUC is when individual factors are modeled separately/jointly and the second AUC is the sum score was modeled as single predictor/summary score.

In our proposed algorithm C, focal neurologic deficit is categorized as a major risk factor and assigned 2 points, versus the other minor risk factors which are weighted equally with 1 point each. Similar to algorithm A and B, a threshold score of 1 or greater would also only reduce scans by a total of 39 (8.7%) with 100% sensitivity and 8.7% specificity, identical to B. However, a threshold score of 2 or greater maintains a sensitivity of 98% while increasing specificity to 38.9% and reducing the number of scans by 171 (34.2%). The one case missed using the 2 point threshold was a female in her forties presenting with fever and acute on chronic left ankle swelling status after remote motor vehicle collision and ankle fusion. There was an indeterminate right thalamic hypodensity on the initial CT with MRI recommended. She was alert and oriented to name and place but not date. She acknowledged intravenous heroin abuse 2 days before admission and presented with tachycardia, fever and numerous track marks on her extremities and Janeway lesions on the palms. Subsequent MRI/CT revealed septic emboli from cardiac source on echo, and she was treated for endocarditis.

The area under curve (AUC) for our statistical models are: Model A, 0.728; Model B, 0.755 and Model C, 0.828. Figure 1, demonstrates the ROC curves for the 3 models compared. Overall all three models showed good discrimination with increasing sum risk factor scores associated with increasing odds of positive CT abnormality and specificity for actionable CT findings.

Figure 1.

Receiver operating characteristic curves for 3 algorithms. Please see Table 1 regarding differences among models.

DISCUSSION

Our goal was to improve the current method of evaluating non trauma patients in the emergency department. Our research furthers the work published in the literature evaluating risk factors of patients in the emergency department (Wang & You). In contrast to their analysis based on CT requisitions, which is often inaccurate or incomplete, we analyzed the electronic medical records available to the emergency department physicians evaluating the patient. For example, 61 of the 1528 cases evaluated for inclusion had history of trauma not evident on the requisition but was present in the EMR notes. However, determining the rate of inaccuracy and discrepancy of the requisition forms is beyond the scope of this paper.

In our analysis, focal neurologic deficit and age > 55 were the only the individual significant predictors of an abnormal head CT. The other variables (history of malignancy, nausea/vomiting, coagulopathy and altered mental status) were not found to be independently significant. However, these risk factors were significant as part of the total sum score which coincides with the results of Wang and You, who found these factors to be independently predictive. This difference may be partly explained by differences in sample size between our studies, statistical interaction and interpretation. While our sample size was smaller than Wang and You, we were able to produce a clinically useful algorithm by combining and stratifying risk factors of patients’ into our clinical algorithm while maintaining high sensitivity (98%) and significantly reducing the number unnecessary CTs being performed.

Our results emphasize the low yield of CT imaging in the non-trauma emergency department patients. Heavy reliance on CT imaging continues to be standard practice by emergency departments despite data and acknowledgment by both emergency medicine and radiology societies of its poor specificity, economic burden, and radiation risks. The need for a standardized clinical decision algorithm to help identify higher risk patients is evident.

We have demonstrated that a stratified model utilizing multiple clinical risk factors can successfully select the patients who will benefit from CT imaging, while maintaining sensitivity. This is a promising step forward to the goal of imaging selectively without incurring immediate adverse events for patients. The fact that our head CT interpretation was not solely provided by neuroradiologists may be a factor confounding our data. However, we believe this mix of interpreting radiologists is more representative of most emergency departments without exclusive interpretation by neuroradiologists. The one case missed in our series was a patient with septic emboli evident on the subsequent MRI. But the patient’s impressive clinical presentation of sepsis and Janeway lesions is highly suggestive of endocarditis requiring treatment regardless of embolic infarcts to the central nervous system.

Limitations of this study were its retrospective, single center design and its relatively small sample size given the infrequency of positive findings on head CT in nontrauma patients.

Our proposed aforementioned Algorithm C, is a simple clinical risk factor stratification algorithm, which can be easily adapted into clinical practice with acceptable threshold for imaging. It is not surprising that focal neurologic deficit was the most reliable and important risk factor, since the majority of questions on the National Institutes of Health Stroke Scale (13) are dedicated to determining this. Prospective studies are needed to validate our conclusions but have great potential to improve and optimize limited health care resources and decrease unnecessary radiation exposure.

Abbreviation Key

OR

odds ratio

AUC

area under curve

CT

computed tomography

ED

Emergency department

EMR

electronic medical records

PRES

posterior reversible encephalopathy syndrome