Diagnostic performance of prehospital large-vessel occlusion detection scales in patients with suspected acute stroke: a retrospective validation study. The Akershus Study of Ischemic Stroke and Thrombolysis

Abstract

Introduction

Prehospital stroke scales are widely used to identify patients with suspected stroke. The scales have shown potential to identify suspected large-vessel occlusion (LVO) for direct-to-thrombectomy triage, yet it remains unclear which scales perform best, and whether performance varies by patient characteristics.

Patients and methods

In this retrospective diagnostic accuracy study, we analysed 1150 consecutive adults admitted under the stroke code protocol at a primary stroke centre (January 2015–December 2017). The cohort included ischaemic stroke, transient ischaemic attack, intracerebral haemorrhage and mimics. We evaluated 55 scale–threshold combinations (46 scales) for detection of computed tomography angiography (CTA)–confirmed LVO, quantified by area under the receiver operating characteristic curve (AUC), sensitivity, specificity and predictive values.

Results

In our study population, LVO prevalence was 13.5%. AUC ranged from 0.586 (Hemiplegia) to 0.855 (FAST-ED). No scale significantly outperformed the continuous National Institutes of Health Stroke Scale (NIHSS, AUC 0.851) after Bonferroni correction, and none achieved both sensitivity and specificity above 80%. Scales combining cortical and motor items outperformed purely motor-based instruments. Accuracy was comparable between sexes but higher for left-hemispheric strokes (mean AUC: 0.82 vs 0.76) and patients aged ≥60 years (0.82 vs 0.77).

Discussion

None of the prehospital stroke scales were accurate enough to reliably detect LVO. Performance varied with stroke laterality, patient age and stroke severity—approaching chance levels at the extremes of severity and showing a consistent hemispheric bias.

Conclusion

This highlights fundamental limitations of using clinical scales alone to detect LVO in real-world stroke code populations.

Registration:  ClinicalTrials.gov, https://clinicaltrials.gov/study/NCT05378490, NCT05378490.

Graphical Abstract

Graphical Abstract.

Introduction

Prehospital stroke scales were originally developed to identify stroke symptoms and support field triage. Following the landmark endovascular thrombectomy (EVT) trials,¹ timely triage of patients eligible for this treatment became critical,^2,3 prompting the development of numerous prehospital screening tools worldwide. Yet, consensus on their use remains elusive.^4–7 The latest American Heart Association/American Stroke Association (AHA/ASA) guidelines recommend a brief prehospital stroke assessment tool to aid early large-vessel occlusion (LVO) detection, without endorsing any specific scale.⁸ In contrast, the European Stroke Organisation–European Society for Minimally Invasive Neurological Therapy (ESO–ESMINT) guidelines found insufficient evidence to recommend clinical scales for prehospital thrombectomy triage altogether.⁹

Several methodological limitations underlie this uncertainty. Most scales were derived or validated in confirmed acute ischaemic stroke (AIS) cohorts, introducing spectrum bias that limits generalisability to undifferentiated prehospital populations.^4,5,9 Validation studies have typically been small, retrospective and methodologically heterogeneous, limiting direct comparability of results across studies.^{4–6,10–12} Although sex-stratified analyses have recently emerged,^13,14 and the hemispheric bias of National Institutes of Health Stroke Scale (NIHSS)-based stroke severity scales is well recognised,^15–17 systematic evaluation of multiple scales across age, sex, stroke laterality, severity and deficit pattern remains lacking.

The aim of this study was to evaluate and compare the diagnostic accuracy of established LVO screening tools in unselected stroke code patients, using computed tomography angiography (CTA) for diagnostic verification. We further examine how cortical and motor deficit domains relate to differences in scale performance and explore whether diagnostic accuracy differs by age, sex, stroke laterality and severity.

Patients and methods

Study design and setting

This retrospective observational study is based on prospectively collected data from the Akershus Study of Ischemic Stroke and Thrombolysis 1 (ASIST-1) registry and is reported in accordance with the Standards for Reporting Diagnostic Accuracy Study (STARD) 2015 guidelines.¹⁸ ASIST-1 is a clinical stroke registry at Akershus University Hospital, a public hospital that serves both as a primary hospital and a university hospital. The hospital operates Norway’s largest stroke unit and provides care for approximately 618,000 inhabitants, corresponding to about 11% of the Norwegian population. EVT was established at Akershus University Hospital in 2019. During the study period (2015-2017), however, the hospital functioned as a primary stroke centre with respect to EVT, and patients with confirmed LVO who were eligible for EVT were transferred to a nearby comprehensive stroke centre for treatment.

Study participants

We included all consecutive adult patients (≥18 years) admitted under the stroke code protocol between 1 January 2015 and 31 December 2017. Stroke code activation required suspected focal neurological deficits and a known time of symptom onset or last known well within 4.5 hours, in line with the prevailing intravenous thrombolysis treatment window. Activation was triggered by the paramedic in mandatory telephone consultation with the on-call neurologist; patients in whom onset could not be established were therefore not admitted under the protocol. All patients within the hospital catchment area were transported directly to this hospital.

Patients were excluded if CTA was not performed and no surrogate evidence of the final diagnosis was available on non-contrast CT, defined as either a hyperdense artery sign corresponding to the clinical deficit or an alternative structural pathology (intracerebral haemorrhage [ICH], subdural haematoma and space-occupying lesion).

Test methods

Clinical assessment

On admission, and before any treatment decision, each patient’s neurological status was assessed by the on-call neurologist using the NIHSS.¹⁹

Scale reconstruction

Index tests comprised LVO detection scales identified from recent systematic reviews^6,7 and reconstructable from the ASIST-1 registry variables. Each included scale was retrospectively constructed from admission NIHSS items using original published scoring definitions and cut-offs. Full-scale names, original references and item composition are provided in Table S1. Item-by-item algorithmic conversion rules from raw NIHSS items (and prehospital vital signs and atrial-fibrillation history, where applicable) to each scale’s component and total score, including approximations required by dataset constraints, are given in Table S1b. Scales were classified by item composition into 4 categories: motor-dominant (n = 9), assessing motor deficits only; motor-cortical (n = 29, including NIHSS derived scales; 37 scale–threshold combinations), combining motor and cortical items; cortical-focused (n = 2), emphasising cortical signs such as gaze deviation, aphasia and neglect; and hybrid/anamnestic (n = 6), additionally incorporating clinical history. Category counts refer to unique scales; Table S1 reports counts per scale–threshold combination. In a small number of cases with missing NIHSS data, scores were reconstructed by an NIHSS-certified study physician from documented clinical findings.

Reference standard

The reference standard was acute symptomatic LVO on CTA, defined as occlusion of the internal carotid artery (ICA), the M1 or M2 segments of the middle cerebral artery (MCA), or the basilar artery (BA).^1,5 LVO status and final discharge diagnosis were determined by the treating neurologist and independently validated through structured chart review and radiological reassessment of CT/CTA imaging. CTA was performed immediately after NIHSS assessment as part of the acute stroke workup. No cases were excluded due to an indeterminate reference standard.

Blinding

The NIHSS was performed before vascular imaging results were available. Scale scores were computed from recorded NIHSS items using each scale’s predefined scoring rules, without knowledge of the reference standard result. The study radiologist had access to clinical information, including the presence and laterality of neurological symptoms, as is standard in clinical practice.

Missing data

Scale scores were computed using complete-case analysis. Of the 55 scale–threshold combinations, 32 were evaluable in all 1150 patients. The remaining 23 had minor reductions due to incomplete NIHSS sub-item information (19 scales, n ≤ 15 missing), missing prehospital clinical variables (3 scales, n ≤ 80), or an onset-time restriction (1 scale, n = 123). Pairwise DeLong comparisons used the intersection of complete cases for each pair; per-scale numbers are reported in Table S2.

Statistical methods

Descriptive statistics

Continuous variables were presented as mean (SD) or median (interquartile range, IQR), depending on the distribution, and categorical variables were presented as counts and percentages. Between-group comparisons used the Wilcoxon rank-sum test for continuous and ordinal variables and the χ² test or Fisher’s exact test for categorical variables, as appropriate.

Primary diagnostic accuracy measures

For each scale–threshold combination, we calculated the number of screen positive patients, sensitivity, specificity, positive and negative predictive values (PPV, NPV), positive and negative likelihood ratios (LR+, LR−), overall accuracy, Youden index and area under the receiver operating characteristic curve (AUC) with 95% confidence intervals (CI; exact Clopper-Pearson), using prespecified cut-off values from original validation studies. The AUC was estimated from each scale’s full-score distribution; for inherently binary scales (n = 7), AUC equals the average of sensitivity and specificity at the single operating point. Sensitivity–specificity pairs were displayed in scatterplots and AUCs in forest plots. Pairwise AUC comparisons against the continuous NIHSS total score were performed using DeLong’s test.²⁰ NIHSS-derived scales with continuous score distributions distinct from the full NIHSS were retained in pairwise comparisons; the 7 NIHSS-threshold variants that share the continuous score were not.

Subgroup analyses

All diagnostic accuracy measures were recalculated stratified by sex (women vs men), affected hemisphere (right vs left), age (<60 vs ≥60 years) and baseline stroke severity (mild NIHSS 0–4, moderate 5–15, severe 16–42). Subgroup AUCs were compared using DeLong’s test and displayed in forest plots. No formal interaction tests were performed; subgroup findings should be considered hypothesis-generating.

To explore which neurological deficits best discriminate LVO from non-LVO stroke, we calculated the AUC for each of the 13 NIHSS items available in the registry (bilateral motor items collapsed to worst-affected side). Items were categorised into cortical (gaze deviation, visual fields, language, neglect), motor (facial palsy, motor arm, motor leg) and other domains. Cortical and motor composite scores were derived by summing sub-items within each domain and their AUCs were compared using DeLong’s test. The presence or absence of any cortical and any motor deficit (score ≥1) was cross-tabulated against LVO status.

General analytical considerations

A two-sided P value <.05 was considered statistically significant, with Bonferroni correction applied to pairwise AUC comparisons against the continuous NIHSS (P <.001; 0.05/45) and to between-subgroup AUC comparisons within each stratification (P <.001; 0.05/45, or 0.05/39 for NIHSS severity). All consecutive registry patients meeting inclusion criteria were included without a formal sample size calculation. All analyses were performed using Stata version 18.5 (StataCorp LLC, College Station, TX, United States).

Ethics

The study was approved by the Regional Committee for Medical and Health Research Ethics, Southeast Norway (reference 2018/498), and the Data Protection Officer at Akershus University Hospital. The Norwegian Regional Committee for Medical and Health Research Ethics waived the requirement for informed consent, in compliance with Ethics Committee regulations and Norwegian medical research law.

Results

During the study period, 1339 consecutive patients were admitted under the stroke code protocol (Figure 1). Of 267 patients (19.9%) who did not undergo CTA, 189 (70.8%) were excluded because neither a hyperdense artery sign nor structural pathology was identified on non-contrast CT. The remaining 78 (29.2%) patients without CTA were retained: 2 had a hyperdense artery sign, 60 had ICH and 16 were stroke mimics with an identified structural cause on imaging. The final study population comprised 1150 patients (86% of all stroke code activations). Of these, 155 (13.5%) had CTA-confirmed LVO. Excluded patients (n = 189) were older, had longer onset-to-hospital times, lower NIHSS scores and were more often diagnosed with stroke mimics (Table S3).

Figure 1.

Study flow diagram. Flow of participants from stroke code activation to final study population, with reasons for exclusion and distribution of diagnoses by LVO status. Abbreviations: AIS = acute ischaemic stroke; CTA = CT angiography; ICH = intracerebral haemorrhage; LVO = large-vessel occlusion; NCCT = non-contrast CT; TIA = transient ischaemic attack.

Patients with LVO were older, more often had atrial fibrillation, had shorter onset-to-hospital times and presented with substantially higher NIHSS scores, with 50% in the severe range (NIHSS 16–42) compared with 5% in non-LVO patients (Table 1). The non-LVO group comprised AIS (40%, median NIHSS 4, IQR 2–7), ICH (9%, median NIHSS 10, IQR 6–17), transient ischaemic attack (TIA) (13%, median NIHSS 2, IQR 1–4) and stroke mimics (38%, median NIHSS 2, IQR 1–5); see Table S3b for full baseline characteristics by subgroup. Among the 155 LVO patients, the most proximal occlusion site was M1 in 70 (45%), M2 in 34 (22%), ICA in 42 (27%) and BA in 9 (6%). Median NIHSS varied across subtypes: 7 (IQR 3–16) for M2, 15.5 (12–21) for M1, 17 (13–22) for ICA and 18 (7–21) for BA.

Table 1.

Baseline characteristics of included patients stratified by large-vessel occlusion status.

Characteristics	Total  n = 1150	LVO  n = 155	Non-LVO  n = 995	P value
Demographics
Male sex, n (%)	622 (54)	88 (57)	534 (54)	.47
Age, years, median (IQR)	71 (60–81)	75 (65–83)	71 (59–81)	.005
Age ≥ 60 years, n (%)	864 (75)	125 (81)	739 (74)	.088
Medical history, n (%)
Stroke	221 (19)	23 (15)	198 (20)	.14
TIA	119 (10)	12 (8)	107 (11)	.25
Diabetes mellitus	162 (14)	17 (11)	145 (15)	.23
Atrial fibrillation	198 (17)	40 (26)	158 (16)	.002
Medication use, n (%)
Antiplatelets	396 (34)	53 (34)	343 (34)	.95
Anticoagulation	160 (14)	23 (15)	137 (14)	.72
Antihypertensives	614 (53)	94 (61)	520 (52)	.052
Prehospital
Onset-to-hospital time, min, median (IQR)	90 (60–142)	69 (50–102)	92 (63–147)	<.001
Systolic blood pressure, mmHg, median (IQR)	158 (139–180)	150 (132–170)	160 (140–180)	.002
Diastolic blood pressure, mmHg, median (IQR)	90 (80–102)	86 (70–99)	90 (80–104)	<.001
Hospital admission
NIHSS score, median (IQR)	4 (2–9)	16 (9–20)	3 (1–7)	<.001
Symptom severity (NIHSS), n (%)				<.001
Mild (0–4)	649 (56)	24 (15)	625 (63)
Moderate (5–15)	370 (32)	53 (34)	317 (32)
Severe (16–42)	131 (11)	78 (50)	53 (5)
Reperfusion therapy, n (%)
IVT	367 (32)	86 (55)	281 (28)	<.001
EVT	40 (3)	39 (25)	1 (0)a	<.001
Final diagnosis, n (%)				<.001
Ischaemic stroke	551 (48)	154 (99)	397 (40)
TIA	129 (11)	1 (1)	128 (13)
ICH	94 (8)	0 (0)	94 (9)
Stroke mimic	376 (33)	0 (0)	376 (38)
mRS at 3 months, median (IQR)	2 (1–3)	4 (2–6)	2 (1–3)	<.001

Data are presented as n (%) or median (IQR). P values were calculated using the Wilcoxon rank-sum test for continuous and ordinal variables and the χ² test or Fisher’s exact test for categorical variables, as appropriate.

^a Occlusion of the A1 segment of the anterior cerebral artery.

Abbreviations: EVT = endovascular thrombectomy; ICH = intracerebral haemorrhage; IQR = interquartile range; IVT = intravenous thrombolysis; LVO = large-vessel occlusion; mRS = modified Rankin Scale; NIHSS = National Institutes of Health Stroke Scale; TIA = transient ischaemic attack.

In total, 46 unique scales were evaluated across 55 scale–threshold combinations (Tables S1 and S2). Several scales were assessed at multiple cut-offs (NIHSS: 7 thresholds; G-FAST and SAVE, 2 each; Pomona and SNOW share identical items). Sensitivity ranged from 18.1% to 90.6%, specificity from 54.6% to 98.3%, PPV from 23.2% to 62.2% and NPV from 88.5% to 97.5% (Figure 2, Table S1).

Figure 2.

Sensitivity–specificity plots for 55 scale–threshold combinations evaluated for large-vessel occlusion detection. Panel A displays all scales at their published cut-off values. Dashed lines indicate 80% thresholds for sensitivity and specificity. Scales are colour-coded by diagnostic profile: Blue, high sensitivity but specificity below 80%; orange, high specificity but sensitivity below 80%; green, both sensitivity and specificity below 80%. Panel B shows the same data with the axes rescaled to the occupied range and numbered labels identifying the individual scales. Scale key: (1) FANG-D, (2) EMSA, (3) VES, (4) NIHSS ≥5, (5) FAST-VAN, (6) NIHSS ≥6, (7) MPSS (Maria), (8) SNOW, (9) GAI2AA, (10) Severe hemiparesis, (11) VAN, (12) PAST, (13) sNIHSS-EMS, (14) sNIHSS-1, (15) FACE2AD, (16) APSS, (17) G-FAST ≥3, (18) SAVE ≥2, (19) mNIHSS, (20) LARIO, (21) PASS, (22) PreSS, (23) SSTS, (24) sNIHSS-5, (25) LAMS, (26) SAFE, (27) NIHSS ≥9 (<3 hours), (28) CPSS, (29) SHIPS, (30) NIHSS ≥10, (31) ASTRAL, (32) sNIHSS-8, (33) d-CPSS, (34) FAST-ED, (35) C-STAT, (36) NIHSS ≥11, (37) mG-FAST, (38) 4I-SS, (39) NIHSS ≥12, (40) FPSS, (41) NIHSS-8 (Hunter-8), (42) Pomona, (43) RACE, (44) CG-FAST, (45) FAST-PLUS, (46) M-DIRECT, (47) NIHSS ≥14, (48) Zuckerman, (49) CPSS + C-STAT, (50) G-FAST =4, (51) CPSS + gaze, (52) 3I-SS, (53) GPS, (54) Hemiplegia, (55) SAVE =4.

No single scale achieved both sensitivity and specificity ≥80%. AUC values ranged from 0.586 (95% CI, 0.555–0.618) for Hemiplegia to 0.855 (95% CI, 0.821–0.889) for FAST-ED (Figure 3). The highest-performing scales, FAST-ED, sNIHSS-8, ASTRAL and d-CPSS, had CIs that substantially overlapped with the continuous NIHSS (AUC 0.851). After Bonferroni correction, no scale achieved a significantly higher AUC than the continuous NIHSS; however, 23 scales had significantly lower AUCs (all P <.001).

Figure 3.

Forest plot of area under the receiver operating characteristic curve (AUC) for 46 large-vessel occlusion detection scales. AUC values with 95% CIs are shown for each scale, ordered from highest to lowest. Scales are colour-coded by category. Scales labelled “(bin)” are binary or compound two-stage classifiers whose AUC reflects a single operating point. The dashed green line indicates the AUC of the full National Institutes of Health stroke scale (NIHSS) as a reference.

In sex-stratified analyses, the AUC values were similar between sexes across all scales (Figure 4A, Table S4). AUC differences were small and centred near zero (mean − 0.8 percentage points (pp), −5.6 to +4.5), with no scale reaching nominal statistical significance.

Figure 4.

Forest plots of stratified AUC differences for 46 large-vessel occlusion detection scales by sex, affected hemisphere and age. Each panel shows the AUC difference (in percentage points) with approximate 95% CIs between subgroups: (A) women minus men, (B) right- minus left-hemisphere symptoms and (C) age ≥60 minus age <60. Scales are listed alphabetically and colour-coded by category. Scales labelled “(bin)” are binary or compound two-stage classifiers. The dashed line at zero indicates no difference between subgroups; points to the right of zero indicate higher AUC in women (A), right-hemisphere symptoms (B), or age ≥60 (C).

In hemisphere-stratified analyses, restricted to patients with lateralising anterior-circulation symptoms (n = 730), the AUC values were consistently higher for left-hemispheric strokes, with 53 of 55 combinations favouring left-sided LVO detection (Figure 4B, Table S5); the 2 exceptions, Pomona and SNOW (identical items), showed marginally higher AUC for right-hemispheric strokes (+2.4 pp). The mean AUC difference (right minus left hemisphere) was −5.8 pp (range − 10.6 to +2.4), corresponding to a mean AUC of 0.76 for right-hemispheric vs 0.82 for left-hemispheric strokes. This laterality effect was most pronounced for motor-dominant scales (mean difference of −7.4 pp) and motor-cortical scales (−6.3 pp), whereas cortical-focused scales showed minimal laterality bias (+1.5 pp). Although 16 combinations were nominally significant (P <.05), none survived the Bonferroni correction. ROC curves for 6 scales stratified by affected hemispheres are provided in Figure S1.

In age-stratified analyses, the AUC values were consistently higher in patients aged ≥60 years, with 54 of 55 combinations favouring the older group (Figure 4C, Table S6; mean AUC 0.82 vs 0.77, mean difference 4.9 pp). Cortical-focused scales showed the largest age-related differences (mean 10.4 pp). After Bonferroni correction, 2 scales showed statistically significant differences: CPSS + gaze (AUC 0.56 vs 0.72, P <.001) and CPSS + C-STAT (AUC 0.58 vs 0.75, P <.001).

In severity-stratified analyses, scale discrimination varied substantially across all 3 strata and represented the largest source of subgroup heterogeneity (Figure 5, Table S7). The mean AUC was highest in the moderate subgroup (mean 0.66, range 0.50–0.74), lower in the mild subgroup (mean 0.60, range 0.44–0.74), and lowest in the severe subgroup (mean 0.53, range 0.46–0.65). Moderate-subgroup AUC exceeded severe-subgroup AUC in 54 of 55 scale–threshold combinations (mean difference 12.4 pp) and exceeded mild-subgroup AUC in 46 of 55 combinations (mean difference 6.0 pp). In the mild subgroup, 18 of 55 scale–threshold variants performed at a near-chance level (AUC 0.47–0.55), with 4 scales yielding exactly chance-level discrimination (AUC 0.50). In the severe subgroup, near-chance performance was even more prevalent (39 of 55 scale–threshold variants, AUC 0.47–0.55), and the decline was most pronounced for NIHSS-derived scales (mean AUC 0.48). Among prehospital-feasible scales in the moderate subgroup, FAST-ED (AUC 0.74) and SAFE (0.73) achieved the highest discrimination (Figure S3). After Bonferroni correction, 5 of 46 unique scales showed significant between-stratum AUC differences. Confidence intervals were wide in the extreme strata, reflecting the limited number of LVO events in the mild (n = 24) and severe (n = 78) subgroups.

Figure 5.

Forest plots of area under the receiver operating characteristic curve (AUC) for 55 large-vessel occlusion detection scale–threshold combinations (46 unique scales) stratified by stroke severity. Each panel displays AUC point estimates with 95% CIs within a single severity stratum defined by baseline NIHSS: (A) mild, NIHSS 0–4 (n = 649); (B) moderate, NIHSS 5–15 (n = 370); and (C) severe, NIHSS 16–42 (n = 131). Scales are listed alphabetically and colour-coded by category. Scales labelled “(bin)” are binary or compound two-stage classifiers. The dashed line at 0.50 indicates chance-level discrimination. AUC was calculated as the nonparametric area under the full ROC curve using each scale’s continuous or ordinal score; for strictly binary scales, the single-threshold concordance statistic is reported. AUC, area under the receiver operating characteristic curve; CI, confidence interval; NIHSS, National Institutes of Health stroke scale.

Among individual NIHSS sub-items, motor arm had the highest standalone AUC (0.778, 95% CI, 0.736–0.819), followed by facial palsy (0.766) and motor leg (0.761) (Figures S2 and S4). Cortical items ranged from 0.634 (visual fields) to 0.724 (gaze deviation). When combined into domain composites, the cortical composite (AUC 0.840) was numerically higher than the motor composite (0.812), though this difference was not statistically significant (P = .146). Both composites substantially outperformed any individual item.

Among LVO patients, 129 (83.2%) exhibited both cortical and motor deficits; 10 (6.5%) had only cortical deficits, 13 (8.4%) had only motor deficits and 3 (1.9%) had neither (Figure S5). The distribution of deficit patterns differed significantly between groups (χ²  P <.001). Among non-LVO patients, the deficit pattern was more heterogeneous: 296 (29.7%) had both cortical and motor deficits, 344 (34.6%) had isolated motor deficits, 236 (23.7%) had neither and 119 (12.0%) had isolated cortical deficits. The latter subgroup was predominantly driven by language/aphasia findings and comprised ischaemic stroke and stroke mimics in equal proportion (each 38%), followed by TIA (16%) and ICH (8%).

Discussion

In this study of 1150 consecutive stroke code patients, we evaluated 55 scale–threshold combinations representing 46 unique scales for LVO detection, exceeding the number of instruments compared in any previous single-cohort study. Three principal findings emerged: no scale achieved both sensitivity and specificity above 80%; scales incorporating cortical signs alongside motor deficits consistently outperformed purely motor-based instruments; and diagnostic accuracy was systematically lower for right-hemispheric strokes and younger patients, while discrimination varied markedly with stroke severity, peaking at moderate levels and declining at both extremes.

Overall diagnostic performance

Nearly half of all scale–threshold combinations (24/55) clustered within a narrow AUC range of 0.83–0.86, and among these, none differed significantly from each other or from the full NIHSS after Bonferroni correction. The lowest-performing scales (n = 10, AUC 0.59–0.76) were predominantly binary classifiers or compound two-stage instruments (9/10), whose single operating point constrains threshold optimisation; the single exception, sNIHSS-1, is ordinal but relies on a narrow item set. This clustering is consistent across multiple cohorts,^5–7 with no instrument clearly outperforming the NIHSS,^6,7 suggesting a diagnostic ceiling that reflects both the high degree of item overlap among scales and the limited specificity of neurological signs for vascular anatomy.

A potential limitation is the use of physician-derived in-hospital NIHSS to reconstruct prehospital scales. However, our AUCs differed by only 0.00–0.05 per scale from the largest prospective paramedic-assessed cohort,⁶ and paramedic-scored prehospital NIHSS has been shown to differ from physician scores by less than one point on average.²¹ Direct item-level comparison has shown that paramedics systematically underdetect cortical signs such as neglect and gaze deviation,²² implying that physician-derived reconstruction selectively favours scales weighting these items rather than uniformly inflating performance. This introduces a degree of circularity, as the observed advantage of cortical-containing scales may be partly attributable to more complete cortical item ascertainment by physicians. However, the deficit pattern analysis (Figure S5), which relies on the binary presence of any cortical deficit rather than graded scores, is less susceptible to this bias, and the superiority of cortical-motor scales has been independently replicated in prospective paramedic-assessed cohorts.^6,10

Cortical vs motor deficits

Although individual motor items had higher standalone AUCs than cortical items, the cortical composite (0.840) exceeded the motor composite (0.812), likely reflecting high intercorrelation among motor items, whereas cortical signs provide additive discriminative value. The deficit pattern distributions (Figure S5) confirm the clinical basis: combined cortical and motor deficits strongly characterised LVO, whereas isolated motor deficits, typical of lacunar stroke, were common among non-LVO patients, explaining the poor discrimination of purely motor-based scales.

These deficit distributions directly explain the gradient in scale performance. Purely motor-based scales performed worse (Hemiplegia AUC 0.586; Severe hemiparesis 0.734), as they cannot distinguish LVO from the large pool of non-LVO patients with overlapping motor deficits. The best motor-dominant scales (d-CPSS 0.845, MPSS 0.833) include a speech item that captures both dysarthria and aphasia, partially reflecting cortical involvement. Conversely, 6.5% of LVO patients presented with isolated cortical deficits that purely motor-based scales will systematically miss.

These findings align with reports identifying gaze deviation and aphasia as stronger LVO discriminators than motor deficits,²³ and a meta-analysis showed hemineglect as the key differentiator for higher scale accuracy.⁴ However, cortical signs are among the most challenging to assess reliably in the field.^4,22,24 Among them, gaze deviation may offer the best combination of discriminative value (AUC 0.724) and field feasibility, as it requires only observation of eye position, though prehospital detection remains challenging.²²

Sex-stratified analyses

Women more often present with non-focal stroke symptoms and are less often correctly identified as having stroke by paramedics,^14,25 raising concern about impaired LVO detection. We found no significant AUC differences between sexes for any of the 55 combinations, extending findings from a prospective paramedic-assessed cohort across 8 scales.¹³ The likely explanation is that LVO scales predominantly capture focal cortical deficits, which do not differ substantially between sexes, while non-focal symptoms are generally not included in these instruments.²⁵

Hemispheric differences

Many of the included scales are derived from NIHSS items that predominantly assess left-hemispheric cortical function, resulting in systematically lower scores for right-hemispheric strokes of equivalent lesion volume.^15,17 A key mechanism is the underdetection of hemispatial neglect, the principal cortical sign of right-hemispheric LVO, which is both difficult to assess reliably in the prehospital setting and underrepresented or absent in most scale architectures.^16,26 Although no hemisphere difference survived Bonferroni correction, the consistent direction across virtually all scales suggests a real phenomenon. Even the RACE scale, designed to balance aphasia and agnosia scoring, showed a persistent left-hemispheric AUC advantage (8.8 pp),²⁷ and only purely cortical-sign instruments (Pomona/SNOW) showed marginally higher AUC for right-hemispheric strokes.

Age-related differences

Despite age being a well-established modifier of stroke presentation and severity,²⁸ its impact on the performance of prehospital stroke scales has remained underexplored. AUC values were consistently higher in patients aged ≥60 years than in those aged <60 years; a pattern also observed in the SAFE scale validation study.²⁹ This likely reflects a clearer separation between LVO and non-LVO presentations in older patients: LVO in the elderly is more often cardioembolic with large-territory cortical infarcts producing prominent deficits, while the younger non-LVO group is known to include a higher proportion of stroke mimics that may reduce specificity.³⁰

Stroke severity-dependent discrimination

Scale discrimination peaked at moderate severity and declined towards both extremes, with a steeper decline towards severe stroke (mean AUC drop 0.13) than towards mild stroke (0.08), consistent with the first study to systematically stratify prehospital LVO scale performance by deficit severity.³¹ The mechanisms at the 2 extremes are distinct. In mild presentations, LVO deficits may be subtle, fluctuating, or confined to a single domain, rendering them insufficient to cross most detection thresholds, a pattern consistent with a recent prospective prehospital study demonstrating similarly poor diagnostic accuracy in suspected mild stroke.³² In severe presentations, both LVO and non-LVO patients present with high-scoring, overlapping deficits, compressing between-group variance such that scale scores identify any severe stroke rather than specifically LVO. However, the clinical impact of this ceiling effect is partially mitigated by the likelihood that severely affected patients will be triaged to comprehensive centres based on clinical severity alone, regardless of scale result. The moderate subgroup is of particular clinical relevance, as it encompasses the severity range in which prehospital scale-based triage decisions are most consequential: patients are severe enough to harbour LVO yet not so severe that clinical impression alone dictates transfer. This range also overlaps with the core EVT trial populations, although the lower end (NIHSS 5–9) was underrepresented in the landmark randomised trials,¹ and the severity-stratified findings should be interpreted with caution given the limited event counts in the extreme strata.

Clinical implications and future directions

No prehospital stroke scale achieved diagnostic accuracy sufficient to reliably guide direct-to-thrombectomy triage across the full clinical spectrum, and the narrow performance range among top-performing scales suggests a diagnostic ceiling inherent to bedside neurological assessment. Nevertheless, implementation of severity-based prehospital triage has reduced onset-to-puncture time by up to 70 minutes and increased direct EVT-centre arrival,^33,34 confirming that even imperfect scales confer meaningful clinical benefit. Since no instrument clearly outperformed the others, scale selection may prioritise field feasibility, training burden and integration with existing protocols over marginal differences in discrimination. With over 60 scales published to date, further refinement of scale architecture alone is unlikely to substantially improve prehospital LVO detection.

A fundamental trade-off constrains clinical utility: at sensitivity-optimised thresholds (≥80%), PPV ranged from 23%–32%, risking overburdening comprehensive centres and delaying thrombolysis at the closest stroke centre.³⁵ Conversely, specificity-optimised thresholds (≥90%) missed 36%–82% of LVO patients. The optimal operating point, therefore, depends on local transport geography and resource capacity.^36,37

Complementary strategies are needed. Mobile stroke units offer near-perfect triage accuracy and prehospital thrombolysis,³⁸ but high costs limit deployment.³⁷ The drip-and-drive approach, in which the interventionalist travels to the primary stroke centre, has shown favourable onset-to-puncture times.³⁹ Telemedicine-assisted triage using remote neurologist guidance offers a scalable alternative and safeguard against erroneous direct-to-EVT triage.³⁴ Machine learning-based decision tools and point-of-care biomarkers represent further promising avenues.⁴⁰ Biomarkers may help address a limit inherent to clinical scales: the inability to distinguish ischaemic from haemorrhagic stroke. In the RACECAT trial, direct-to-EVT triage was associated with worse outcomes in patients later found to have ICH.³⁵ Blood-based glial fibrillary acidic protein (GFAP), elevated in ICH but not ischaemic stroke, may enable prehospital haemorrhage exclusion and enhance LVO detection when combined with clinical scales.⁴¹

Strengths and limitations

This study applied 55 scale–threshold combinations from 46 unique scales to a single consecutive stroke code cohort using a uniform CTA-based reference standard, enabling direct comparisons free of between-study heterogeneity. Including all stroke code diagnoses approximates the undifferentiated population encountered by emergency medical services.

Several limitations should be considered. The retrospective, single-centre design may limit generalisability, and external validation in prospective, multi-centre prehospital cohorts is needed. The LVO prevalence of 13.5% is consistent with comparable unselected populations^10,34 but reduces positive predictive values relative to AIS-only cohorts. The high proportion of stroke mimics (33%), while representative of stroke code populations,¹¹ may influence specificity estimates, though the direction of this effect depends on mimic phenotype. Data were collected between 2015 and 2017, when EVT was being established as standard of care; subsequent changes in referral patterns and time-window criteria may limit direct applicability to current practice. CTA was performed at the discretion of the treating physician rather than as a standardised protocol for all patients; those who did not undergo vascular imaging were predominantly low-acuity cases with an obvious alternative diagnosis on non-contrast CT or clear contraindications for acute reperfusion treatment (Table S3), and their exclusion constitutes partial verification bias that may have modestly inflated specificity. Our uniform LVO definition (ICA, M1, M2, BA) may differ from those used in original derivation studies. Notably, the inclusion of M2 occlusions (n = 34/155, 22%), whose status as EVT targets remains debated, may differentially affect scale performance by including milder LVO presentations. Basilar-artery occlusions (n = 9, 6% of LVO) were included per current EVT trial criteria. While posterior-circulation LVO strokes can present with disproportionately low NIHSS,⁴³ BA cases in our cohort had a median NIHSS of 18 (IQR 7–21), comparable to anterior LVO; the small absolute number nevertheless precludes a powered subgroup analysis. Worst-side-only NIHSS motor scoring precluded true bilateral asymmetry calculations for 4 scales, potentially overestimating scores and inflating sensitivity, and grip strength was approximated from motor arm scores.

Stroke code activation required a known onset or last known well within 4.5 hours. The cohort, therefore, reflects time-window-eligible candidates rather than the full prehospital population, and excludes patients with unknown onset, wake-up stroke or extended time windows in whom NIHSS-based LVO discrimination may differ.⁴² This restriction may also explain the high intravenous thrombolysis rate (32% of stroke codes; 54% of AIS and TIA combined).

Finally, 17/155 (11%) of LVO patients in our cohort presented with NIHSS 0–3, consistent with prior reports.⁴³ Such mild presentations have been linked to favourable collateral circulation, which was not captured in the ASIST-1 registry; we therefore cannot quantify the contribution of collateral status to the performance of the LVO scales.

Conclusion

No prehospital stroke scale achieved sufficient diagnostic accuracy to reliably guide direct-to-thrombectomy triage alone, and no abbreviated scale significantly outperformed the full NIHSS. Scale selection should favour instruments incorporating both cortical and motor deficits. Stroke laterality, patient age and particularly stroke severity are important sources of variation in scale performance that should be considered in triage system design. Supplementary triage strategies are needed, particularly at both extremes of the severity spectrum.

Author contributions

Silje Holt Jahr (Conceptualization [equal], Data curation [equal], Formal analysis [lead], Investigation [equal], Methodology [equal], Software [lead], Visualisation [lead], Writing—original draft [lead], Writing—review & editing [lead]), Maiken Nordahl Selseth (Data curation [equal], Investigation [equal], Writing—review & editing [equal]), Kristin Tveitan Larsen (Data curation [equal], Investigation [equal], Writing—review & editing [equal]), Antje Sabine Reichenbach (Writing—review & editing [equal]), Marianne Altmann (Writing—review & editing [equal]), Antje Sundseth (Writing—review & editing [equal]), Vigdis Hillestad (Data curation [equal], Investigation [equal], Writing—review & editing [equal]), Helge Fagerheim Bugge (Writing—review & editing [equal]), Anne-Cathrine Braarud (Writing—review & editing [equal]), Maren Ranhoff Hov (Writing—review & editing [equal]), Kashif Waqar Faiz (Writing—review & editing [equal]), Ole Morten Rønning (Conceptualization [supporting], Methodology [supporting], Writing—review & editing [equal]), Else Charlotte Sandset (Supervision [supporting], Writing—review & editing [equal]), and Espen Saxhaug Kristoffersen (Conceptualization [lead], Data curation [lead], Funding acquisition [lead], Investigation [equal], Methodology [equal], Project administration [lead], Resources [equal], Supervision [lead], Writing—original draft [supporting], Writing—review & editing [equal])

Conflicts of interest

None declared.

Funding

This study was funded by the University of Oslo and Akershus University Hospital, Norway. The funders had no role in the study design, data collection, data analysis, data interpretation, manuscript writing, or the decision to submit for publication.

Data availability

Supporting data are available from the corresponding author upon reasonable request, subject to approval by the local ethics committee.

Ethics approval

The study was approved by the Regional Committee for Medical and Health Research Ethics, Southeast Norway (reference 2018/498) and the Data Protection Officer at Akershus University Hospital.

Informed consent

The Norwegian Regional Committee for Medical and Health Research Ethics waived the requirement for informed consent, in compliance with Ethics Committee regulations and Norwegian medical research law.

Guarantor

E.S.K.

Patient consent

Waived by the ethics committee.