A Bayesian Framework to Optimize Performance of Pre-Hospital Stroke Triage Scales

Dear Sir:

Early identification of patients with stroke due to large vessel occlusion (LVO), potentially eligible for thrombolysis and endovascular therapy.(EVT), is important for patient triage and management [1,2]—such as deciding whether to directly transport a given patient to an EVT-capable comprehensive stroke center.(CSC). Various pre-hospital triage tools for acute stroke have been developed for this purpose. Clinical evaluation scales are most commonly used, as they can be adopted by paramedics after some basic training and require no specialized equipment [3]. Below, using simulated data for the USA, we examine the strengths and weaknesses of current pre-hospital triage tools through a Bayesian lens, and discuss areas where our efforts should be directed to improve acute stroke triage.

What are we trying to accomplish with pre-hospital triage of stroke with suspected LVO?

Stroke triage tools are subject to sensitivity-specificity trade-offs like all tests. In general, a screening test needs high sensitivity, whereas a confirmatory test in positively screened patients needs high specificity. Most clinicians would think of LVO stroke as being a not-to-miss condition, in which case the primary goal of triage tools would be to maximize sensitivity. A policymaker’s perspective might differ: with scarce CSC resources, the priority may be to minimize false-positive LVO rates even if some LVOs are not flagged early for transport to CSCs. Advantages and disadvantages of both these paradigms are shown in Supplementary Table 1.

If we take the “mile-high” view of all strokes in the USA, approximately 795,000 Americans have a stroke annually [4]. Of these, around two-thirds (approximately 517,000) seek ambulance transport and are potentially eligible for pre-hospital triage [5,6]. Most of these patients are clustered towards the lower end of the severity spectrum (Figure 1), where the probability of LVO is low. Of course, the National Institutes of Health Stroke Scale (NIHSS) score is not actually known at the pre-hospital stage, but let us use it here to illustrate the spectrum of stroke severity. On a population level, the median severity for ischemic stroke patients per NIHSS is around 2 to 3 [7-9], likely shifted to around 5 among those seeking emergency medical attention [10]. But with each step up the NIHSS, the probability of LVO rises (Figure 1A), particularly getting into a moderate range of severity (NIHSS ≥10) [11]. Even though the percentage of patients with LVO may be lower at intermediate NIHSS scores, the overall number of patients in the middle range is considerably higher, and thus, the total number of EVT-eligible patients in that range will likely be higher than at the extreme of stroke severity (Figure 1B). When measuring the performance of pre-hospital triage tools, it is therefore important how they perform not only in the extremes of stroke severity, but also in the middle of the range.

Figure 1.

The occurrence of vessel occlusions in patients with ischemic stroke presenting with different degrees of stroke severity per the National Institutes of Health Stroke Scale (NIHSS) score. (A) The proportion of patients with and without large vessel occlusion (LVO), in each NIHSS score category, among all patients undergoing computed tomography or magnetic resonance angiography. These graphs are based on re-analyses of data reported from the Bernese Stroke Database [11]. (B) The estimated number of patients with ischemic stroke in the USA, seeking emergency medical attention each year, who are likely to have LVO for a given NIHSS score. These estimates were obtained by applying published data from the Bernese Stroke Database to stroke statistics from the American Heart Association [4]. In (B), the corresponding NIHSS score for different cut-offs on the Los Angeles Motor Scale (LAMS) are also shown, as roughly estimated from work by the Field Administration of Stroke Therapy–Magnesium (FAST-MAG) investigators on the distribution of vessel occlusions for different scores on the LAMS and NIHSS..

What are the characteristics and limitations of current pre-hospital triage tools?

Current prediction scales have low sensitivity (range 38% to 67%) for LVO—given their superficial focus on gross deficits—but high specificity (80% to 93%) [12-17]. A meta-analysis estimated that with a positive LVO prediction test, the LVO probability could be 50% to 60% depending on LVO prevalence in the population, but the probability with a negative test could still be ≥10% [13]. Thinking of patient-centered care, such a low sensitivity means that treatment might be withheld/delayed from patients who might still benefit from it, which is suboptimal, especially when considering that most pre-hospital tools are used in relatively wealthy, industrial countries with sufficient healthcare resources.

For instance, if we apply the Los Angeles Motor Scale (LAMS) to all emergently presenting stroke patients in the USA to decide whom to transport to CSCs (Figure 1B) [18], we would direct over one-third (n=188,000) to CSCs if using the conventional LAMS cut-off of ≥4, but more than one in five of these patients will not actually have a LVO, and we would still end up missing over one-quarter of all LVOs (n=59,000) (Supplementary Table 2). If we only consider patients with stroke severities in the intermediate range, the situation is even worse (Supplementary Table 3)—around 45% of positively screened patients would not have a vessel occlusion, but now we would be missing more than half of the patients with vessel occlusions (n=28,000).

Where might our efforts be focused to improve pre-hospital triage tools?

It can be helpful to adopt a Bayesian approach and think about how much current pre-hospital scales can shift our diagnostic confidence about whether a patient has an LVO. Although the sensitivity and specificity of pre-hospital scales for LVO likely varies by stroke severity (Table 1), most studies only report overall operating characteristics [18]. In the extremes of stroke severity, a positive test does little to shift the high pre-test probability of LVO (Figure 2), because the more disabling the stroke deficits, the more evident the diagnosis will be even to a layperson without any specialized knowledge. The patients/caregivers themselves may call in to say they are having a stroke. For example, with NIHSS ≥20, the pre-test LVO probability is around 94%, and a positive LAMS simply shifts this by around 3% to a post-test probability of 97%, meaning the test is far less useful in this range. Whereas pre-hospital scales still offer a common language for paramedics to evaluate deficits, one might argue that with such intuitively severe strokes, the best destination for the patient is almost always the CSC.

Table 1.

The value of pre-hospital triage tools likely differs by stroke severity

Symptom severity at presentation	Frequency	Predicted performance of current clinical triage tools*	Added value of a clinical triage tool for decisions	Added value of a new pre-hospital technology highly sensitive and specific for LVO	Rationale for added value or lack thereof
Severe stroke symptoms	Relatively uncommon	Moderate sensitivity	Low	Low	Pre-test probability for LVO is very high
		High specificity			Diagnosis of major stroke is obvious even to an untrained observer
Intermediate severity	Common	Poor sensitivity	High	High	Intermediate pre-test probability for LVO
		Moderate specificity			Patients with intermediate stroke severity are common
Mild stroke symptoms	Common	Poor sensitivity	Low	Potentially very High	Pre-test probability for LVO is very low
		Poor specificity			Current triage tools are likely insensitive to mild strokes
					If low-NIHSS trials of EVT are positive, then a pre-hospital technology able to identify such mild LVO-related strokes may be very valuable

LVO, large vessel occlusion; NIHSS, National Institutes of Health Stroke Scale; EVT, endovascular therapy.

^*The comparative performance of pre-hospital scales is yet to be systematically studied across different gradations of stroke severity.

Figure 2.

Pre- and post-test probability of large vessel occlusion (LVO) when using the Los Angeles Motor Scale (LAMS) with its typical cut-off of LAMS ≥4. Of course, the National Institutes of Health Stroke Scale (NIHSS) is not typically determined in the pre-hospital setting, but we show it here as a quantification of the manifest stroke severity observed by the pre-hospital team. We also show what the post-test probabilities would be with hypothetical pre-hospital technologies that achieve a slightly better positive likelihood ratio (+LR) of 3.00 (vs. 2.17 for LAMS) and a much better +LR of 19.00, including at the mildest end of the severity spectrum, to which current scales are likely insensitive. Such technologies would be especially valuable if ongoing trials of endovascular therapy (EVT) in low-NIHSS populations are positive.

In contrast, at the mildest end of stroke severity, current scales have little to offer as they will almost always be negative. This leaves us with the intermediate severity range (e.g., NIHSS 5–10). Here, the pre-test LVO probability is 35% to 40%, so a positive LAMS moves us to a post-test probability of 55% to 60%, resulting in a substantial shift of 20%. Thus, we are left with a “sweet spot” or “Goldilocks zone” in the middle of the severity spectrum where a pre-hospital tool could make a tangible difference to patient management by identifying a stroke that a human being may be uncertain about (Table 1). This is particularly important given the high overall number of patients in the middle range of symptom severity.

Looking ahead: can alternative triage technologies solve this problem?

One possibility to improve stroke triage with existing tools is a probabilistic, rather than a dichotomous approach to LVO detection: rather than classifying patients as either LVO+ or LVO–, the entire spectrum of the score could be translated into an estimated “LVO probability.” Simulation models could then individually determine the optimal triage pathway for a given patient with a certain LVO probability under local geographic circumstances in real-time, e.g., via an app, while also taking into account transport times and workflow efficiency at local primary stroke centers and CSCs [19,20]. Time from onset would be an important consideration, since patients arriving earlier are not only more likely to be thrombolysis-eligible, but also tend to have more severe deficits [21].

Furthermore, alternative pre-hospital technologies have been developed that rely on direct visualization of the occluded vessel (e.g., computed tomography angiography in mobile stroke units, transcranial Doppler) or indirect signals of impaired cerebral activity like electroencephalography [22,23]. Since these technologies do not rely on clinical deficits, they may detect LVOs in even mild strokes for which current scales are insensitive. This will become very important if ongoing EVT trials in low-NIHSS patients with LVO (like ENDOLOW, NCT04167527 and MOSTE, NCT03796468) are positive, as stroke systems will then need to accurately flag these patients (~38,000 LVOs with NIHSS ≤5 annually in USA) (Figure 1B). For example, even a slightly better technology that achieves a positive likelihood ratio (+LR) of 3.00 would boost the LVO probability from 10% to 15% for a minor stroke with NIHSS ≤3 to 30% to 35%; a much better technology achieving both sensitivity and specificity of 95% (+LR 19.00) would further boost post-test probability to 75% to 80% (Figure 2). To succeed in practice, such technologies will also need to offer an appropriate combination of costs, practicability, and ease of use.

Ethical statement

As the estimates and figures in this paper were derived from published data from prior studies, no ethical approvals or patient consents were required for this work.

Conclusion and outlook

There are many ongoing efforts to develop better technologies for pre-hospital triage of patients with suspected vessel occlusions [22], but where might our limited resources be best directed when seeking to test and improve these technologies? Firstly, future studies should focus on the ability of pre-hospital tools to identify patients with intermediate stroke severity. Secondly, stroke systems have to decide what aspect of pre-hospital triage to prioritize in their environment: sensitivity e.g., when there are sufficient resources to avoid missing EVT-eligible patients; or specificity e.g., when there is resource scarcity at CSCs. Triage tool cut-offs should be adjusted accordingly. There is a need for high-quality data on how pre-hospital scales perform across the spectrum of stroke severity to help us better understand their strengths and weaknesses when seeking to apply them in care pathways, while we continue working on alternative triage technologies. Such new technologies may have maximal impact by achieving higher sensitivity and specificity for LVO among patients with low-to-medium stroke severity, particularly if EVT begins to play a bigger role in patients with milder LVO-related strokes.

Supplementary materials

Supplementary materials related to this article can be found online at https://doi.org/10.5853/jos.2021.01312.

Supplementary Table 1.

Sensitivity/specificity trade-off in acute ischemic stroke triage depending on the symptom severity cut-off

Supplementary Table 2.

Implications of using different LAMS cut-offs for triage of all suspected acute strokes

Supplementary Table 3.

Implications of using different LAMS cut-offs for triage of suspected acute strokes of intermediate severity