Validation of the use of the ROSIER scale in prehospital assessment of stroke

Abstract

Aim:

To determine the utility of the Recognition of Stroke in the Emergency Room (ROSIER) scale as a stroke recognition tool among Chinese patients in the prehospital setting.

Materials and Methods:

Compared with the Cincinnati Prehospital Stroke Scale (CPSS), emergency physicians prospectively used the ROSIER as a stroke recognition tool on suspected patients in the prehospital setting. And, the final discharge diagnosis of stroke or transient ischemic attack made by neurologists, after assessment and review of clinical symptomatology and brain imaging findings, was used as the reference standard for diagnosis in the study. Then, the ROSIER and the CPSS like sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV), related coefficient (r) and Kappa value were calculated.

Results:

In this study, 540 of 582 suspected stroke patients met the study criteria. The CPSS showed a diagnostic Se of 88.77% (95% confidence intervals [CI] 86.11–91.43%), Sp of 68.79% (95% CI 64.88–72.70%), PPV of 87.40% (95% CI 85.97–88.83%), NPV of 71.52% (95% CI 67.71–75.33%) and r of 0.503. Relatively, the ROSIER showed a diagnostic Se of 89.97% (95% CI 87.44–92.64%), Sp of 83.23% (95% CI 80.08–86.38%), PPV of 92.66% (95% CI 90.46–94.86%), NPV of 77.91% (95% CI 74.41–81.41%) and r of 0.584. According to the final discharge diagnosis, both the ROSIER and the CPSS were associated with the final discharge diagnosis (P < 0.05).The Kappa statistic value of the ROSIER and the CPSS were 0.718 and 0.582, respectively. However, there was no statistical significance of the positive rate between the ROSIER and the CPSS in this study (P > 0.05).

Conclusions:

The ROSIER is a sensitive and specific stroke recognition tool for health providers’ use among Chinese patients in the prehospital setting. However, it cannot be used to confidently rule out or identify stroke as a diagnosis. Comprehensive clinical assessment and further examination on potential stroke patients are still important and cannot be replaced. When it is difficult to objectively complete the ROSIER for patients, the CPSS could replace it in the prehospital setting.

Introduction

Stroke is the leading cause of death in Chinese urban communities. Each year, in China, about 2,000,000 people of all ages suffer a new stroke, and 1,500,000 stroke-related deaths occur.[1] Early thrombolytic therapy has been proven to be safe and effective for patients with acute ischemic stroke. When given early enough, it can highly improve the outcome and reduce disability. However, it has been delayed in up to two-thirds of the patients because of ineffective assessment and inappropriate triage.[2,3] These subsequent delays may cause serious implications for long-term disability, quality of life and financial resources.[4,5]

Most stroke events occur at home. 29% to 65% of acute stroke patients access their initial medical care via local emergency medical service (EMS), which made EMS at the forefront of stroke management.[6,7] However, misdiagnosis rates for suspected stroke patients is high, up to 19% by stroke specialists and 33% by emergency physicians.[8] In order to increase the accuracy of stroke diagnosis and improve the rapid triage of stroke patients, numerous stroke recognition tools have been designed for health providers’ use, such as the Cincinnati Prehospital Stroke Scale (CPSS), the Los Angeles Prehospital Stroke Screen (LAPSS) and the Los Angeles Motor Scale (LAMS) in the USA, the Recognition of Stroke in the Emergency Room (ROSIER) scale and the Face Arm Speech Test (FAST) in the UK, and the Melbourne Ambulance Stroke Screen (MASS) in the Australia.[9–18] At present, the CPSS, the LAPSS and the FAST have been adopted worldwide and are recommended by the American Heart Association (AHA), American Stroke Association (ASA) and The European Stroke Organization (ESO). These stroke recognition tools increased the diagnostic accuracy, but their sensitivity and effectiveness are still in debate.[19–21] The ROSIER was recently developed and has proven to be better than the CPSS, FAST and LAPSS in the emergency department (ED) setting.[18,22] However, its performance in the prehospital setting is still unknown. Here, we conducted a prospective study to validate the ROSIER scale in Chinese patients, and compared its performance with the CPSS, with the aim to find out whether the ROSIER can be used in the prehospital setting.

Materials and Methods

Setting

This study was conducted in the ED of the Foshan Hospital of Traditional Chinese Medicine (FSTCM) from April 2010 to November 2011. The FSTCM is affiliated to the Guangzhou University of Traditional Chinese Medicine and it is a tertiary care teaching hospital in Foshan City, Guangdong Province, South China. Its ED is one of the large urban EMS dispatch centers. It assesses approximately 76,000 patients per year and, among them, about 6000 patients use EMS for transport to FSTCM. There is a combined police-fire-EMS dispatch center in Foshan city. When citizens dial 110 or 120 and ask for emergency medical service, the EMS teams will be activated and dispatch the physician-staffed ambulances immediately. In Foshan city, we dispatch routinely an emergency physician, a professional nurse, two stretcher-bearers and an ambulance with a full-time driver.[23] And, the average distance per transport is 5 km.

Participants

Inclusion criteria

Stroke was defined as a focal or global neurological deficit with symptoms lasting for 24 h or resulting in death before 24 h, which was thought to be due to a vascular cause after investigation. Transient ischemic attack (TIA) was defined as clinical syndromes characterized by an acute loss of focal cerebral or monocular function with symptoms lasting less than 24 h and thought to be caused by inadequate blood supply as a result of thrombosis or embolism.[18] All patients aged older than 18 years old with suspected stroke or TIA with symptoms or signs seen by emergency physician in the prehospital setting were included. According to the ASA guidelines, patients who got one or more of these suggestive clinical elements as follows were defined as suspect acute stroke or TIA patients. The suggestive clinical elements include sudden weakness or numbness of the face, arm or leg, especially on one side of the body; sudden confusion, trouble speaking or understanding; sudden trouble seeing in one or both eyes; sudden trouble walking, dizziness, loss of balance or coordination; or sudden severe headache with no known cause.[24]

Exclusion criteria

Patients were excluded if they had a history of head trauma or did not accept medical treatment in a prehospital setting or refused to get emergent computerized tomography (CT) or magnetic resonance imaging (MRI).

Design

All emergency physicians got a 6-h course on performing the ROSIER and the CPSS before the study, and performed both of them on suspected stroke patients on the scene. When rapidly transported to the ED, these patients were subjected to neurologic screening assessment by the stroke team and got an emergent CT scan of the brain. Then, the neurologists with no prior information of the CPSS and ROSIER scores made the further consultations. Patients with uncertain diagnosis got a further MRI brain scan within 48 h of admission. Patients with a definite stroke diagnosis and indications of thrombolytic therapy were admitted immediately to the stroke unit. Patients with potential indications for surgery were admitted to the neurosurgery department. The final discharge diagnosis of stroke or TIA made by the neurologists was used as the reference standard for diagnosis in this study. All the neurologists were blinded to the results of the CPSS and the ROSIER scores.

The instrument

The ROSIER[18] [Figure 1] assesses elements of history and physical examination to produce a score between 2 and 5. Patients with a total score of >0 are taken as being consistent with stroke, whereas scores of 0 signify a low probability of stroke.

Figure 1.

The Recognition of Stroke in the Emergency Room scale, GCS = Glasgow Coma Scale; E = eye; M = motor; V = verbal component; BM = blood glucose; BP = blood pressure (mmHg)

The CPSS[6] [Table 1] is a three-item neurologic physical examination that assesses facial droop, arm drift and abnormal speech. Each item is assessed as normal or abnormal. If any one of these three items is abnormal, the probability of a stroke is 72%. The presence of a single abnormality on the CPSS has a sensitivity of 59% and a specificity of 89% when scored by prehospital providers.[11]

Table 1.

the cincinnati prehospital stroke scale

Ethics approval

The study was performed in accordance with the Declaration of Helsinki. Hospital ethics board approval was obtained for the study with a waiver of consent.

Statistical analysis

Data were entered into a Microsoft Excel database and analyzed using SPSS 13.0 (SPSS, Chicago, IL, USA). The patients’ results of the CPSS and the ROSIER scores, the CT and MRI scan and the final discharge diagnosis were all recorded. Compared with the final discharge diagnosis, statistical analysis was performed using the Pearson chi-square test and Kappa analysis on evaluation of the ROSIER and the CPSS. The McNemar test was used to test whether their positive rates were statistical difference or not. A P-value of less than 0.05 was considered to indicate statistical significance. Kappa statistic value was defined as follows: 0-0.20 = poor agreement; 0.21-0.40 = fair agreement; 0.41-0.60 = moderate agreement; 0.61-0.80 = good agreement; 0.81-1.00 = excellent agreement. The accuracy, sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV) and association coefficient (r) were calculated.

Results

From April 2010 to November 2011, there were 582 suspected stroke patients assessed by emergency physicians on the scene, in which 540 met the study criteria [mean age 63 years, range 18-96 years, 175 (32.41%) female]. Three hundred and eighty patients (70.37%) had a final discharge diagnosis stroke or TIA [mean age 67 years, 143 (37.63%) female, 225 (59.21%) ischemic stroke/TIA and 115 (40.79%) hemorrhagic stroke] and 160 patients were “stroke mimics” (40 vertigo, 27 seizure, 22 syncope, 20 cardiac, 15 sepsis, 10 hypoglycemia, 10 hysteria, six alcohol intoxication, four brain tumor, three demyelinative diseases, two hypokalemia and one labyrinthitis). Compared with the final discharge diagnosis, Tables 2–4 show the results of the corresponding diagnostic performance of the ROSIER and the CPSS in this prospective validation study.

Table 2.

Results of the use of the ROSIER scale in prehospital stroke assessment (n = 540)

Table 4.

The individual analysis of the ROSIER and the CPSS (n = 540)

Table 3.

Results of the use of the CPSS in prehospital stroke assessment (n = 540)

The ROSIER scale incorrectly diagnosed 65 of 540 patients (12.04%; 27 false positive, 38 false negative) in this study. The false positive group included eight vetigo, five seizure, four syncope, four brain tumor, two hypokalemia, two cardiac and two sepsis; and the false negative group included 10 TIA, 10 posterior circulation infarct (POCI), eight lacunar infarct (LACI), five partial anterior circulation infarct (PACI), three subarachnoid hemorrhage (SAH) and two intracerebral hemorrhage (ICH). By contrast, there were 92 of 540 patients misidentified by the CPSS (17.04%; 49 false positive, 43 false negative). The false positive group included 10 hypoglycemia, 10 vetigo, seven seizure, six syncope, six cardiac, four brain tumor, three sepsis, two hypokalemia and one demyelinative disease, and the false negative group included 11 TIA, nine POCI, nine LACI, seven PACI, three SAH, two ICH and two cerebellar hemorrhage.

We compared the corresponding diagnostic performance of the ROSIER scale to the CPSS using the Pearson chi-square test. Both the ROSIER and the CPSS were associated with the final discharge diagnosis (P < 0.05). The ROSIER had good corresponding diagnostic performance (Kappa value = 0.718), while the CPSS had moderate performance (Kappa value = 0.582). The ROSIER was superior to the CPSS (Se 89.97% vs. 88.77%, Sp 83.23% vs. 68.79%, PPV 92.66% vs. 87.40%, NPV 77.91% vs. 71.52% and r 0.584 vs. 0.503). However, there was no statistical significance of positive rate between the ROSIER and the CPSS using the McNemar test (P > 0.05).

Discussion

To the best of our knowledge, this is the first study to use the ROSIER to identify suspected stroke patients in the prehospital setting. The results of our study show that both the ROSIER and the CPSS have a good corresponding diagnostic performance in Chinese patients, and the ROSIER is more sensitive and specific, suggesting that the ROSIER scale could be a good stroke recognition tool for EMS providers’ use in a prehospital setting in China.

Time is brain. The early recognition and reaction to stroke warning signs is the first key step of “Stroke Chain of Survival” that links actions to maximize stroke recovery![6] As the EMS system continues to undergo rapid development, more and more suspected stroke patients are likely to arrive at the hospital earlier. It may make them who meet thrombolytic therapy indications achieve good medical treatments and outcomes possible. However, the therapeutic time window for thrombolytic therapy is narrow and precious. It is a challenge for EMS providers to make rapid stroke assessment and effective management. Until now, there is not a recognized “paramedic” profession in China. It is common for newly graduated doctors who lack clinic experience to work in prehospital emergency care.[25] In order to decrease delay, emergency physicians had better be familiar with some stroke recognition tools to help confirm their overall clinic impression of stroke.[26] At present, there has been no uniform approach to the prehospital diagnosis and assessment of suspected stroke patients in China. Some stroke recognition tools were developed based on European and American characters; therefore, they need validation among the Chinese population.

By use of regression modeling, Nor and colleagues developed the ROSIER and validated it at the Newcastle Hospital ED. In their study, early use of ROSIER has very promising results - greater sensitivity 93% (95% CI 89-97%) and similar or better specificity 83% (95% CI 77-89%), which is better than the CPSS, FAST and LAPSS.[18] It can help emergency physicians with less neurology expertise recognize stroke patients rapidly in the ED. In China, it is popular to use the physician-staffed ambulance system in the prehospital setting. Therefore, we trained our emergency physicians to perform the ROSIER scale among Chinese patients. Can the ROSIER be useful in the assessment of suspected stroke patients in the prehospital setting? However, it has not yet been studied.

Three key problems in Nor and colleagues’ study may limit the use of the ROSIER in a prehospital setting. Firstly, the paramedics referred suspected patients directly to the stroke unit, bypassing the local ED by using a rapid ambulance protocol for suspected stroke that incorporated FAST. Patients identified as having a possible acute stroke are transferred to the stroke unit and assessed by the stroke team on call. For these reasons, the ROSIER scale study was conducted in the stroke unit and not in the prehospital setting, where EMS providers routinely assess and treat patients. And, the ROSIER scale has been applied later in the diagnostic process in the ED where the level of suspicion for a diagnosis of stroke was higher.[27] Secondly, before the delivery to the stroke unit, some suspected stroke patients might be missed and “excluded” due to the FAST incorrect assessment. Thirdly, after the selection of FAST assessment, the “including” patients, with almost half having had stroke in Nor and colleagues’ study, could not necessarily be representative of the types of patients seen in the prehospital setting.

Our previous study, with a limited sample size in the prehospital setting (only 41 patients cases), shows that the ROSIER scale was a sensitive, specific stroke recognition tool in our ED.[22] Here, we increased the sample size and compared the performance of the ROSIER with the CPSS. Both of them had good corresponding diagnostic performance, but not 100%. Our data showed that if totally depending on the ROSIER, emergency physicians might miss patients with ischemic posterior or lacunar lesions (18/380, 4.74%), which indicates that a thorough examination is still necessary. And, the ROSIER could not distinguish TIA from stroke mimics in patients without neurological signs. In some cases (confusion, coma, etc.), patient's elements of history and physical examination are difficult to access. We scored patients without witness history as zero, and suggested emergency CT or MRI scan in order to exclude stroke mimics.[22]

We also found that there were no statistical significance of positive rate between the ROSIER and the CPSS (P > 0.05). A useful stroke assessment tool should be sensitive to the diagnosis of stroke (i.e., miss very few patients with a treatable disease) and sufficiently specific to ensure only appropriate patients are referred to the stroke service (or sent for emergent brain scanning). As the CPSS and the ROSIER have a very similar positive rate, and the CPSS is easier to complete, these results suggest that the simpler CPSS may be more practicable than the ROSIER for the prehospital assessment of patients with suspected stroke on the scene. Thus, when it is difficult to objectively evaluate the patients’ scores of the ROSIER, the CPSS could replace it in the prehospital setting.

Our study was limited by the small size and the single-center setting. In our study, not all stroke patients who met thrombolytic therapy indications got the thrombolytic therapy for some reasons. Thus, whether using the ROSIER can improve stroke patients outcomes is still not clear. Patients who met the study criteria were based on overall clinic impressions. And, the experience of different emergency physicians on recognition of stroke could be a bias to perform the ROSIER or not. We might have missed some patients without signs and symptoms of a stroke.

Conclusion

Based on the results of our study, we suggest that the ROSIER scale is a sensitive and specific stroke recognition tool for EMS providers’ use in a prehospital setting in China. However, the ROSIER cannot be used to confidently rule out or identify stroke as a diagnosis. The comprehensive clinical assessment and further examination on potential stroke patients are still important and irreplaceable. When it is difficult to objectively evaluate the patients’ scores of the ROSIER, the CPSS could replace it in the prehospital setting.