Stroke severity quantification by critical care physicians in a mobile stroke unit

Supplemental Digital Content is available in the text.

Abstract

Background

Cerebral revascularization in acute stroke requires robust diagnostic tools close to symptom onset. The quantitative National Institute of Health Stroke Scale (NIHSS) is widely used in-hospital, whereas shorter and less specific stroke scales are used in the prehospital field. This study explored the accuracy and potential clinical benefit of using NIHSS prehospitally.

Patients and methods

Thirteen anesthesiologists trained in prehospital critical care enrolled patients with suspected acute stroke in a mobile stroke unit. NIHSS was completed twice in the acute phase: first prehospitally and then by an on-call resident neurologist at the receiving hospital. The agreement between prehospital and in-hospital NIHSS scores was assessed by a Bland–Altman plot, and inter-rater agreement for predefined clinical categories was tested using Cohen’s κ.

Results

This Norwegian Acute Stroke Prehospital Project study included 40 patients for analyses. The mean numerical difference between prehospital and in-hospital NIHSS scores was 0.85, with corresponding limits of agreement from − 5.94 to 7.64. Inter-rater agreement (κ) for the corresponding clinical categories was 0.38. A prehospital diagnostic workup (NIHSS and computed tomographic examination) was completed in median (quartiles) 10 min (range: 7–14 min). Time between the prehospital and in-hospital NIHSS scores was median (quartiles) 40 min (32–48 min).

Conclusion

Critical care physicians in a mobile stroke unit may use the NIHSS as a clinical tool in the assessment of patients experiencing acute stroke. The disagreement in NIHSS scores was mainly for very low values and would not have changed the handling of the patients.

Introduction

Acute stroke care greatly depends on saving time to treatment by the early identification of patients who may benefit from cerebral revascularization, either as intravenous thrombolysis or mechanical thrombectomy, or as a combination of these two options. New treatment options call for restructuring of both prehospital and in-hospital settings. Recent publications suggest that new models for stroke identification should be tested in the prehospital setting and preferably in mobile stroke units (MSU) [1]. However, prehospital models solely dedicated to acute stroke care and run by in-hospital specialists may be difficult and costly to implement.

The ideal prehospital stroke scale should be accurate and rapid to perform. Several different easy-to-use scales, such as the Cincinnati prehospital stroke scale [2], the Face Arm Speech Test [3], and the Los Angeles Prehospital Stroke Scale [4], have been assessed in a prehospital setting. Unfortunately, these scales show low sensitivity and specificity and fail to identify up to 30% of patients with acute stroke [5]. Recent literature suggests that the in-hospital scale, National Institute of Health Stroke Scale (NIHSS), may be used in the prehospital setting if carried out by a stroke specialist via telemedicine [6]. Moreover, NIHSS may likely be the best nonradiological tool to identify patients with cerebral large vessel occlusion for direct triage to an invasive stroke center [7].

This study was carried out in a MSU staffed like the national Norwegian helicopter emergency medical services (HEMS) with an anesthesiologist trained to provide prehospital critical care to all critically ill patients including acute stroke. We investigated their ability and accuracy to perform real-time NIHSS scoring in a pre-existing nationwide emergency medical service (EMS), and the main aim of the study was to analyze the level of agreement between the prehospital and in-hospital NIHSS scores in the acute phase of stroke.

Patients and methods

The study is part of the Norwegian Acute Stroke Prehospital Project (NASPP), a pilot study conducted in the county of Østfold Norway, inhabited by ~285 000 people. It was carried out using a MSU (Mercedes Springer, Stuttgart, Germany), operating on weekdays from 8: 00 a.m. to 8: 00 p.m. for 85 days between October 2014 and January 2016. As NASPP was a pilot study, patients within a 15-min driving time to the hospital were not included to avoid the possibility of a prehospital delay.

The emergency medical communication center used the Norwegian index of emergency medicine as a decision tool for determination of the appropriate response [8]. Patients included in the study met the following inclusion criteria: age older than 18 years, not pregnant, clinical symptoms of acute stroke, and symptom onset within 4 h.

The NIHSS quantifies the severity of neurological symptoms in stroke by the functionality of 11 physical parameters and a scoring system ranging from 0–42 points (Supplementary Table 1, Supplemental digital content 1, http://links.lww.com/EJEM/A190). High scores correspond with increased severity [9]. We aimed to explore the possible benefits of prehospital NIHSS and to group patients in a three-category scale indicating the clinical relevance of the prehospital score (Table 1).

Table 1.

Prehospital algorithm for treatment and triage categorization based on National Institute of Health Stroke Scale

The critical care physicians completed a 1-day course in the clinical assessment of acute stroke. This included a one-hour introductory course in NIHSS assessment followed by practical training including simulation with vibrant markers mimicking different neurological deficits. After the practical NIHSS session, all physicians completed a web-based certification program [10].

On arrival at patient’s site, the critical care physician performed a short patient history evaluation, a rapid assessment of vital signs, a NIHSS scoring, blood testing, and a cerebral computed tomography (CT) scan. The NIHSS score and the tentative diagnosis were noted in an online study form and kept blinded to the other physicians involved in the study. The patient’s history and the time of symptom onset were reported directly to the on-call resident neurologist who performed a new NIHSS scoring immediately after hospital arrival. The in-hospital physicians were responsible for the final diagnostic and therapeutic decisions.

The Norwegian regional ethics committee approved the study (project ID: 2098/2013). The patients gave their initial oral consent in the MSU, and a deferred written consent. In situations where a written consent could not be completed by the patient, the next of kin provided consent.

Statistical analysis

The prehospital and in-hospital NIHSS scores are presented in a Bland–Altman plot, where the difference between the two measurements are plotted against their mean [11]. The corresponding limits of agreement (LoA) are the limits for 95% of the observed differences, representing the actual variation in the data [12]. The LoA enables a comparison between the actual variation in the collected data and the clinically acceptable variation. In this study, a NIHSS score variability that led to a change in clinical category was considered of relevance to patient care, as a change in category may result in altered treatment options.

Inter-rater agreement for the corresponding categorized NIHSS data was calculated with Cohen’s κ. κ value less than or equal to 0.2 represents poor agreement, 0.21–0.4 fair agreement, 0.41–0.6 moderate agreement, 0.61–0.80 good agreement, and 0.81–1.0 excellent agreement [13].

Continuous data are presented as mean (SD) for Gaussian distributed and median (quartiles) for skewed data and data with outliers.

Statistical analyses were performed in SPSS, version 23 (IBM Corp., Armonk, New York, USA) and R 3.3.1 (University of Auckland, Auckland, New Zealand).

Results

Of 68 patients examined with a prehospital cerebral CT scan in the NASPP MSU, 40 patients were finally included for statistical analyses (Fig. 1). Fourteen patients were excluded from the analyses (Supplementary Table 2, Supplemental digital content 2, http://links.lww.com/EJEM/A191). Patients were predominantly female (65%) with a mean age of 67 years (range: 21–91 years).

Fig. 1.

Patient flowchart. MSU, mobile stroke unit.

The Bland–Altman plot is shown in Fig. 2. The mean difference between prehospital and in-hospital NIHSS score measurements was 0.85, and LoAs were − 5.94–7.64 (95% confidence interval: − 7.86–4.02 and 5.72–9.56), respectively. The wide LoAs are, however, mainly owing to a few patients with large differences in prehospital and in-hospital NIHSS scores. Patient’s prehospital and in-hospital NIHSS scores and clinical categorization are presented in Table 1.

Fig. 2.

Bland–Altman plot showing the mean difference in prehospital and in-hospital National Institute of Health Stroke Scale scores and limits of agreement. MSU, mobile stroke unit.

Inter-rater agreement between anesthesiologist and neurologist in defining patient’s NIHSS values to clinical categories was fair (κ = 0.38). Most of the misclassifications were owing to 10 patients scored as NIHSS 2–5 in the MSU, but NIHSS 0 or 1 at hospital admission.

Time from first encounter with the MSU to a complete diagnostic workup including NIHSS, blood tests, and CT examination was median (quartiles) 10 min (range: 7–14 min). Time spent to carry out NIHSS specifically was not reported. Median time (quartiles) from symptom onset to prehospital NIHSS score was 1 h and 06 min (range: 47 min–1 h 59 min). The time difference between prehospital and in-hospital NIHSS scores was median (quartiles) 40 min (range: 32–48 min).

Discussion

This pilot study shows that critical care physicians in a MSU independently may use the NIHSS as a reliable clinical tool for quantification of neurological symptoms in acute stroke. This opens for more specific prehospital clinical stroke assessment and hopefully a better and faster selection to revascularization therapy already in the very early prehospital phase.

After the breakthrough thrombectomy studies published in 2015 [14,15], early identification and selection of patients who may benefit from thrombectomy has become increasingly important. By incorporating quantitative acute stroke diagnostics in the existing EMS or HEMS system, many more ‘of the right patients’ may be offered revascularization therapy.

An exact prehospital notification and triage in acute stroke seems efficient [16,17], and stroke symptom quantification with NIHSS is relatively simple, fast, and well validated. NIHSS may help to identify patients with a probable large vessel occlusion, which is essential when making the decision for direct triage to an invasive stroke center for thrombectomy [7]. A Danish study suggested that a NIHSS score of 6 or more would identify most patients with large vessel occlusion [18]. This study grouped NIHSS values from prehospital and in-hospital assessment in the same patients and found that inter-rater agreement was fair, and the highest level of variability was in patients with mild symptoms. In patients with moderate to severe symptoms (NIHSS ≥ 6), there was little variability between prehospital and in-hospital scores. These findings are comparable to the results presented in this manuscript.

We found slightly higher NIHSS scores prehospital than in-hospital (Fig. 2). This tendency may primarily be interpreted as a systematic difference and could be explained by the often seen fluctuating clinical presentation, where spontaneous recanalization occur in up to 17% of patients with acute ischemic stroke [19]. A recent in-hospital study by Naess et al. [20] showed that the mean of NIHSS scores in the acute phase (first 3 h after symptom onset) improved by more than 3 score points.

Critical care physicians are trained to observe patients in a very systematically and quantitative manner. A prehospital incorporation of the NIHSS scale may allow prehospital and in-hospital physicians to assess their patients similarly during the first hours and days after a stroke. Furthermore, by incorporating NIHSS in the existing EMS and HEMS, the need to train neurologists and radiologists in prehospital critical care will be reduced [21–23]. Our NASPP model, combining a prehospital NIHSS scoring with a prehospital cerebral CT, may allow both initiation of prehospital thrombolysis and a high-quality triage to revascularization therapy [24].

This study has some limitations. First, the number of patients included was rather low as the MSU operated in a rural area, and this results in a large confidence interval in the analysis. In addition, patients with intracranial bleeding were admitted directly from the MSU to the regional neurosurgical department. Second, the critical care physicians and the neurologists conducted the NIHSS with a mean time difference of 40 min, which creates a bias for a direct comparison of prehospital and in-hospital NIHSS scores. However, the real-time setting in our study makes time intervals impossible to avoid.

Conclusion

Incorporation of NIHSS in the EMS may result in higher level of prehospital stroke competence, by establishing a ‘common language’ throughout the acute phase. We will explore stroke quantification using NIHSS in a real-time EMS run by paramedics in our future studies.

Acknowledgements

The authors thank Ann Kristin Wiik, Head of Research Department at the Norwegian Air Ambulance Foundation, for administrative support, and Professor Hans Morten Lossius, General Secretary of the Norwegian Air Ambulance Foundation, for making the Norwegian MSU concept a reality.

The Norwegian Air Ambulance Foundation funded this study.

Conflicts of interest

There are no conflicts of interest.