Abstract
Background
Emergency medical services (EMS) compliance with recommended prehospital care for patients with acute stroke is inconsistent; however, sources of variability in compliance are not well understood. The current analysis utilizes a linkage between a statewide stroke registry and EMS information system data to explore patient and EMS agency‐level contributions to variability in prehospital care.
Methods and Results
This is a retrospective analysis of a cohort of confirmed stroke cases transported by EMS to hospitals participating in a statewide stroke registry. Using EMS information system data, the authors quantified EMS compliance with 6 performance measures derived from national guidelines for prehospital stroke care: prehospital stroke scale performance, glucose check, stroke recognition, on‐scene time ≤15 minutes, time last known well documentation, and hospital prenotification. Multilevel multivariable logistic regression analysis was then used to examine associations between patient‐level demographic and clinical characteristics and EMS compliance while accounting for and quantifying the variation attributable to agency of transport and recipient hospital.
Over an 18‐month period, EMS and stroke registry records were linked for 5707 EMS‐transported stroke cases. Compliance ranged from 24% of cases for last known well documentation to 82% for documentation of a glucose check. The other measures were documented in approximately half of cases. Older age, higher National Institutes of Health Stroke Scale, and earlier presentation were associated with more compliant prehospital care. EMS agencies accounted for more than half of the variation in EMS prehospital stroke scale documentation and last known well documentation and 27% of variation in glucose check but <10% of stroke recognition and prenotification variability.
Conclusions
EMS stroke care remains highly variable across different performance measures and EMS agencies. EMS agency and electronic medical record type are important sources of variability in compliance with key prehospital performance metrics for stroke.
Keywords: emergency medical services (EMS), healthcare qualtiy assessment, medical record linkage, prehospital, quality improvement, registries, stroke
Subject Categories: Intracranial Hemorrhage, Ischemic Stroke, Health Services, Quality and Outcomes
Nonstandard Abbreviations and Acronyms
- EMR
electronic medical record
- LKW
last known well
- MASR
Michigan's Acute Stroke Registry
- MI‐EMSIS
Michigan's EMS Information System
- NEMSIS
National EMS Information System
- NIHSS
National Institutes of Health Stroke Scale
- PSS
prehospital stroke scale
Clinical Perspective.
What Is New?
This analysis quantifies emergency medical services compliance with recommendations for the prehospital care of patients with stroke and explores sources of variability in care using linked data from statewide emergency medical services and stroke registries.
Compliance with key metrics was highly variable overall, and a substantial portion of this variability is attributable to agency‐level variation in the content or documentation of care.
What Are the Clinical Implications?
Knowledge of variability in emergency medical services performance may help stroke programs design and target interventions to improve prehospital stroke care.
The need for data‐driven optimization of prehospital care for time‐dependent emergencies such as acute stroke has long been recognized. 1 , 2 , 3 , 4 , 5 However, accomplishing this goal requires both a common nomenclature to describe emergency medical services (EMS) encounters as well as a mechanism to compile EMS data from its many segmented sources in order to describe and monitor care. 6 To accomplish this, a Uniform Prehospital Dataset 6 , 7 has been developed and used to create state‐level and National EMS Information Systems (NEMSIS). 5 , 8 However, while these data have been used for a variety of surveillance functions, relatively little research has examined its content and reliability. Furthermore, because of the separation of EMS and in‐hospital data describing patient outcomes, NEMSIS has yet to fully deliver on its promise of providing data to drive quality improvement. 9 Recognition of this shortcoming has led to calls for systematic integration of EMS data to other health data sources, a goal identified as a top priority by the Prehospital Guideline Consortium. 10
Acute stroke is a medical emergency of particular interest to EMS systems because the efficacy and safety of stroke treatments are highly time dependent. 11 As the first point of contact for more than half of patients with stroke, 12 EMS providers are in a unique position to expedite stroke care. Observational evidence has confirmed that, compared with those who arrive by other means, EMS‐transported patients with stroke arrive in the emergency department earlier in the course of their symptoms, 13 , 14 receive brain imaging faster, 13 , 14 and are treated with alteplase more frequently 14 and more quickly. 15 Nevertheless, these benefits are not universally achieved for all EMS‐transported patients with stroke but are linked in part to the quality of prehospital care provided by EMS. 16 In particular, accurate EMS stroke recognition 17 , 18 , 19 and hospital prenotification 19 , 20 , 21 are pivotal in translating EMS utilization into favorable emergency department stroke responses. To promote EMS stroke recognition, expedient transport, and activation of hospital‐based stroke systems, several prehospital performance metrics have been developed for stroke 11 , 22 ; however, compliance with these practices is variable, 16 and data regarding outcomes for EMS‐transported patients with stroke are often lacking. 23 , 24
Little has been done to establish the sources of variation in EMS care. Patient‐level characteristics such as age, time since last known well (LKW), and whether the onset of symptoms was witnessed presumably contribute to patient‐level differences. Another potential source of variability is EMS agency‐level factors stemming from different practice patterns or different electronic medical record (EMR) systems. We address these gaps using a recently established linkage between Michigan's EMS Information Systems (MI‐EMSIS) and MASR (Michigan's Acute Stroke Registry), a member of the Paul Coverdell National Acute Stroke Registry. 25 This linkage allows for examination of current EMS stroke care performance in a state‐wide sample of EMS‐transported patients with strokes and describes the degree and sources of variability in EMS performance compliance at the patient and EMS agency level.
METHODS
Data Sources and Linkage
The current study is a retrospective, observational analysis of a cohort of EMS patient encounters who ultimately received a hospital‐based diagnosis of an acute cerebrovascular event (ischemic stroke, hemorrhagic stroke, or transient ischemic attack) admitted to one of 38 MASR participating hospitals between January 1, 2018, and June 30, 2019. The data set arises from a probabilistic linkage between the MASR and MI‐EMSIS previously described. 25 In it, we used probabilistic matching software (LinkPlus, Centers for Disease Control) followed by an iterative cleaning process to link records from 2 deidentified data sets. Of 8828 stroke cases in the MASR coded as arrived by EMS, 5985 (67.8%) were successfully matched to a corresponding EMS record. The matched cohort included data from 38 of the 104 acute care hospitals in Michigan and 147 of the 281 transporting EMS agencies in Michigan. The MASR contains 56.7% of all stroke admissions in the state. For this analysis, cases with illogical door‐to‐computed tomography times (<0 minutes), prolonged door‐to‐computed tomography (>360 minutes), or that did not arrive to the emergency department were excluded from analysis. This quality improvement analysis of deidentified data received an exempt determination from the Michigan Department of Health and Human Services institutional review board and, thus, informed consent was not required. The data that support the findings of this study are available from the corresponding author upon reasonable request.
EMS Performance Measures
Compliance with 6 quality measures derived from stroke clinical guidelines and consensus statements 11 , 22 , 26 were examined (Table 1). Each measure represents an action included in the state EMS stroke protocol, which serves as the basis for local EMS protocols in Michigan. 27 Five of the measures were abstracted directly from relevant fields in MI‐EMSIS, which reflect documentation by EMS providers. The sixth measure (prenotification) was derived from hospital‐based documentation in MASR because it was not reliably available in MI‐EMSIS. The proportion of EMS‐transported stroke cases with documented compliance for each measure was calculated across all cases in the linked data set. Phi correlation coefficients 28 were calculated to quantify the correlation in compliance between each pair of quality measures. We then calculated the mean, SD, median, and 90th percentile agency‐level compliance rates for each measure. Agency‐level performance was also examined after excluding agencies with ≤10 stroke transports during the study period.
Table 1.
Prehospital Stroke Performance Measure Definitions*
| Measure | Definition |
|---|---|
| Prehospital stroke scale | EMS documentation of a validated stroke screen |
| Glucose check | Documented glucose level |
| EMS stroke recognition | EMS primary or secondary impression of stroke or TIA |
| On‐scene time ≤15 min | Time from EMS scene arrival to beginning transport to hospital is ≤15 min |
| LKW time documentation | Cases with EMS documentation of LKW date/time |
| Hospital prenotification† | Documentation of prenotification in MASR database |
EMS indicates emergency medical services; LKW, last known well; MASR, Michigan Acute Stroke Registry; and TIA, transient ischemic attack.
For each measure, the compliance rate is calculated as the number of compliant cases divided by all EMS‐transported stroke cases.
Data obtained from Michigan's EMS Information System except for prenotification, which was derived from the MASR database.
Statistical Analysis
Descriptive statistics were used to characterize the demographic and clinical characteristics of the study sample using proportions with 95% CIs for counts, means with SDs for normally distributed continuous variables, and medians with quartiles for nonnormally distributed continuous variables. All statistical analyses were conducted using Stata version 15 (StataCorp LLC).
To identify factors independently associated with variation in EMS compliance, a multilevel logistic regression model was developed. In it, patient‐level factors including age, race, sex, stroke type, stroke severity, and time from LKW to hospital arrival were treated as fixed effects. Additionally, hospital‐level and agency‐level random intercepts (crossed random effects) were included to account for clustering within each group (see Data S1). Intraclass correlation coefficients were calculated to estimate the proportion of total variance in EMS compliance attributable to EMS agency‐level and hospital‐level variation. 29 , 30 The model was run with each of the 6 quality measures as a binary outcome. We also hypothesized that EMR structure may account for some interagency variation as each EMS agency uses one of 19 different EMR software vendors. Therefore, for metrics where a substantial portion of variance was attributable to EMS agencies and very little to destination hospital, we examined the contribution of different EMS EMR software by including EMR type as a random effect.
RESULTS
During the 18‐month study period, 5707 EMS‐transported stroke cases met inclusion criteria (Figure 1). A summary of demographic and clinical characteristics of these clinically confirmed EMS‐transported patients with acute stroke is provided in Table 2. Patients were mostly older than 60 years with a slight female predominance. Acute ischemic stroke or transient ischemic attack made up >80% of cases. About one‐third arrived within 120 minutes of symptom onset, and 18.5% arrived within 60 minutes. Transportation was provided by 147 EMS agencies to the 38 MASR hospitals. EMS agencies delivered patients to a median of 2 hospitals (quartile 1–quartile 3, 2–4; minimum–maximum, 1–10) and hospitals received patients from a median of 10 different agencies (quartile 1–quartile 3, 7–13; minimum–maximum, 1–23).
Figure 1. Flow diagram of included patients.
CT indicates computed tomography; DTCT, door‐to‐computed tomography; EMS, emergency medical services; MASR, Michigan Acute Stroke Registry; and MI‐EMSIS, Michigan's EMS Information System.
Table 2.
Characteristics of the 5707 EMS‐Transported Patients With Stroke in the MASR
| Characteristic | All patients (N=5707) |
|---|---|
| Age category, y | |
| <60 | 1081 (18.9) |
| 60–69 | 1197 (21) |
| 70–79 | 1383 (24.2) |
| 80–89 | 1436 (25.2) |
| >89 | 610 (10.7) |
| Women | 2971 (52.1) |
| Race | |
| Non‐Hispanic White | 4039 (70.8) |
| Non‐Hispanic Black | 1222 (21.4) |
| Hispanic ethnicity (any race) | 167 (2.9) |
| Asian | 47 (0.8) |
| American Indian/Alaska Native | 19 (0.3) |
| Native Hawaiian/Pacific Islander | 2 (0.0) |
| Unknown | 209 (3.7) |
| Year | |
| 2018 (12 mo) | 3684 (64.5) |
| 2019 (6 mo) | 2023 (35.5) |
| Stroke type | |
| IS/TIA/stroke NOS | 4732 (82.9) |
| SAH | 169 (3.0) |
| ICH | 806 (14.1) |
| NIHSS | |
| 0–5 | 2609 (45.7) |
| 6–11 | 1093 (19.2) |
| 12–20 | 808 (14.2) |
| >20 | 532 (9.3) |
| Missing | 665 (11.7) |
| Onset‐to‐door time, min | |
| 0–60 | 1055 (18.5) |
| 61–120 | 839 (14.7) |
| 121–360 | 885 (15.5) |
| 136–720 | 645 (11.3) |
| >720 | 1344 (23.6) |
| Missing | 939 (16.5) |
| DTCT ≤25 min | 3214 (56.3) |
| Median DTCT (quartile 1–quartile 3) | 21 (10–55) |
| Alteplase treated | 959 (19.9) |
| DTN ≤45 (%) | 469 (49.0) |
| Median DTN (quartile 1–quartile 3) | 46 (36–64) |
| EMS agency volume (transports) (n=147) | |
| >300 | 4 (2.7) |
| 151–300 | 4 (2.7) |
| 76–150 | 10 (6.8) |
| 11–75 | 54 (36.7) |
| 1–10 | 75 (51.0) |
| Hospital certification (n=38) | |
| CSC | 10 (26.3) |
| PSC | 25 (65.8) |
| ASR/no designation | 3 (7.9) |
| Hospital stroke volume (n=38) | |
| >300 | 5 (13.1) |
| 150–299 | 12 (31.6) |
| 75–149 | 11 (28.9) |
| 25–74 | 5 (13.1) |
| 1–24 | 5 (13.1) |
Values are expressed as number (percentage) unless otherwise indicated. ASR indicates acute stroke ready; CSC, comprehensive stroke center; DTCT, door‐to‐computed tomography; DTN, door‐to‐needle; EMS, emergency medical services; ICH, intracerebral hemorrhage; IS, ischemic stroke; MASR, Michigan Acute Stroke Registry; NIHSS, National Institutes of Health Stroke Scale; NOS, not otherwise specified; PSC, primary stroke center; SAH, subarachnoid hemorrhage; and TIA, transient ischemic attack.
Overall transport‐level quality measure compliance is provided in Figure 2. Glucose documentation had the highest compliance (82%) and LKW documentation the lowest (24%). Prehospital stroke scale (PSS) documentation, on‐scene time ≤15 minutes, and prenotification were all documented in approximately half of the cases. At the patient level, compliance with any one measure was only weakly correlated with compliance on any other measure (Table S1).
Figure 2. Percentage of 5707 EMS–transported stroke cases with documented prehospital stroke performance metric compliance with 95% CIs.
A, Overall EMS performance at the patient level. B, Patient‐level EMS performance by agency stroke transport volume. EMS, emergency medical services; LKW, last known well; OST, on‐scene time; and PSS, prehospital stroke scale.
Mean and median agency‐level quality are provided in Table 3. Point estimates for median EMS agency‐level compliance ranged from 12.5% for LKW documentation to 86.1% for glucose documentation; however, the range between quartile 1 and quartile 3 was large for these estimates. Mean and median agency‐level compliance was ≈50% for PSS documentation, EMS stroke recognition, on‐scene time ≤15 minutes, and prenotification. When limiting the analysis to agencies with >10 cases, interquartile range estimates (quartile 3–quartile 1) were smaller, but mean and median agency‐hospital–level performance estimates were similar with one exception: LKW documentation demonstrated an unchanged mean but higher median performance (Table 3). The full distribution of agency‐level compliance rates is provided in Figure S1.
Table 3.
Mean, Median, and 90th Percentile EMS Agency Performance Across All EMS Agencies (n=147) and Those With >10 Stroke Transports (n=72)
| All EMS agencies (n=147)* | Agencies with >10 stroke transports (n=72)† | |||||
|---|---|---|---|---|---|---|
| Performance measure | Mean agency‐level compliance (SD) | Median agency‐level compliance (quartile 1–quartile 3) | 90th percentile | Mean agency‐level compliance (SD) | Median agency‐level compliance (quartile 1–quartile 3) | 90th percentile |
| PSS documentation | 47.2 (36.2) | 53.6 (0.0–76.5) | 100.0 | 50.9 (29.7) | 59.1 (26.7–71.9) | 87.5 |
| Glucose check | 76.7 (27.4) | 86.1 (66.7–100) | 100.0 | 79.0 (18.9) | 84.9 (73.8–91.6) | 96.2 |
| EMS stroke recognition | 52.5 (30.1) | 55.2 (35.0–70.0) | 100.0 | 52.9 (16.8) | 54.5 (46.7–61.7) | 72.7 |
| LKW documented | 26.6 (32.2) | 12.5 (0.0–46.9) | 75.0 | 26.9 (24.0) | 22.7 (4.8–45.9) | 62.7 |
| OST ≤15 min | 51.0 (28.0) | 50.0 (37.5–67.9) | 100.0 | 52.8 (14.7) | 52.2 (41.7–62.3) | 75.0 |
| Prenotification | 56.2 (30.9) | 57.9 (38.9–81.8) | 100.0 | 59.5 (22.4) | 59.1 (45.7–79.8) | 86.7 |
Agency‐level performance was defined as the proportion of compliant cases transported by an agency divided by the total number of cases transported by that agency. EMS indicates emergency medical services; LKW, last known well; OST, on‐scene time; and PSS, prehospital stroke scale.
N=5707 encounters.
N=5448 encounters.
Output from multilevel, multivariable random‐effects models is presented in Table 4. Older age demonstrated statistically significant associations with most measures. Female sex was associated with slightly higher odds of glucose check and slightly lower odds of EMS stroke recognition and on‐scene time ≤15 minutes. On the other hand, patient race was not associated with any EMS performance measure in adjusted models. The most consistent associations were observed for stroke severity, with moderate to severe cases (National Institutes of Health Stroke Scale [NIHSS] >6) consistently having the highest odds of receiving compliant care. Patients who presented earlier following symptom onset also tended to have higher odds of EMS compliance, while patients with subarachnoid hemorrhage tended to have significantly lower odds of compliance than those with ischemic stroke.
Table 4.
Demographic and Clinical Characteristics Associations With EMS Performance Measure Compliance Among 5707 EMS‐Transported Stroke Cases in Unadjusted and Adjusted ORs Generated From Multivariable Crossed Random‐Effects Logistic Regression Models.
| PSS documentation | Glucose check | EMS stroke recognition | ||||
|---|---|---|---|---|---|---|
| Covariate | Unadjusted | Adjusted | Unadjusted | Adjusted | Unadjusted | Adjusted |
| Age, y | ||||||
| <60 | Reference | Reference | Reference | Reference | Reference | Reference |
| 60–69 | 1.37 (1.16–1.61)† | 1.4 (1.14–1.71)† | 1.1 (0.89–1.36) | 1.19 (0.93–1.52) | 1.26 (1.07–1.48)† | 1.3 (1.08–1.57)† |
| 70–79 | 1.37 (1.17–1.61)† | 1.31 (1.08–1.6)† | 1.08 (0.88–1.33) | 1.22 (0.96–1.56) | 1.15 (0.98–1.35) | 1.12 (0.93–1.35) |
| 80–89 | 1.34 (1.14–1.57)† | 1.24 (1.01–1.51)† | 1.14 (0.93–1.4) | 1.28 (1–1.64) | 1.31 (1.11–1.53)† | 1.21 (1–1.46)† |
| ≥90 | 1.38 (1.13–1.68)† | 1.33 (1.03–1.71)† | 1.22 (0.93–1.58) | 1.4 (1.02–1.92) | 1.30 (1.06–1.58)† | 1.23 (0.97–1.55) |
| Women | 0.94 (0.85–1.04) | 0.93 (0.81–1.05) | 1.15 (1–1.31) | 1.18 (1.02–1.38)† | 0.82 (0.74–0.91)† | 0.79 (0.70–0.89)† |
| Race | ||||||
| White | Reference | Reference | Reference | Reference | Reference | Reference |
| Black | 0.64 (0.56–0.72)† | 0.96 (0.79–1.17) | 1.24 (1.04–1.47)† | 1.22 (0.96–1.55) | 0.64 (0.57–0.73)† | 0.88 (0.73–1.06) |
| Other/missing | 1.02 (0.81–1.28) | 1.04 (0.78–1.38) | 1.06 (0.79–1.42) | 1.09 (0.78–1.54) | 0.67 (0.55–0.87) | 0.76 (0.58–1) |
| Stroke subtype | ||||||
| IS/TIA | Reference | Reference | Reference | Reference | Reference | Reference |
| SAH | 0.33 (0.24–0.47)† | 0.57 (0.38–0.86)† | 0.79 (0.54–1.15) | 0.98 (0.64–1.51) | 0.24 (0.16–0.35)† | 0.37 (0.24–0.57)† |
| ICH | 0.7 (0.6–0.81)† | 1.12 (0.91–1.37) | 1.1 (0.9–1.35) | 1.18 (0.93–1.51) | 0.85 (0.73–0.98)† | 1.11 (0.91–1.34) |
| NIHSS | ||||||
| 0–6 | Reference | Reference | Reference | Reference | Reference | Reference |
| 6–11 | 1.52 (1.31–1.76)† | 1.69 (1.41–2.02)† | 1.28 (1.06–1.54)† | 1.33 (1.08–1.65)† | 2.32 (2.00–2.68)† | 2.41 (2.05–2.83)† |
| 12–20 | 1.47 (1.25–1.73)† | 1.67 (1.36–2.04)† | 1.75 (1.39–2.21)† | 1.82 (1.41–2.37)† | 2.72 (2.30–3.22)† | 2.9 (2.41–3.49)† |
| >20 | 1.04 (0.86–1.26)† | 1.03 (0.82–1.3) | 1.8 (1.36–2.38)† | 1.79 (1.31–2.44)† | 2.14 (1.76–2.59)† | 2.15 (1.73–2.66)† |
| Missing | 0.36 (0.3–0.44)† | 0.4 (0.32–0.51)† | 0.78 (0.64–0.96)† | 0.83 (0.64–1.08)† | 0.34 (0.28–0.41)† | 0.48 (0.37–0.61)† |
| LKW‐to‐door | ||||||
| 0–120 | Reference | Reference | Reference | Reference | Reference | Reference |
| 121–360 | 0.9 (0.77–1.07) | 0.81 (0.66–0.99)† | 1.08 (0.86–1.35) | 1.03 (0.8–1.32) | 0.85 (0.72–1.0) | 0.8 (0.67–0.96)† |
| 361–720 | 0.74 (0.62–0.89)† | 0.69 (0.55–0.86)† | 0.84 (0.66–1.06) | 0.9 (0.69–1.17) | 0.66 (0.55–0.80)† | 0.65 (0.53–0.79)† |
| >720 | 0.57 (0.49–0.66)† | 0.52 (0.44–0.62)† | 0.86 (0.71–1.04) | 0.89 (0.72–1.1) | 0.37 (0.32–0.43) | 0.38 (0.32–0.44)† |
| Missing | 0.34 (0.29–0.4)† | 0.28 (0.23–0.34)† | 0.59 (0.48–0.71)† | 0.58 (0.47–0.73)† | 0.16 (0.14–0.19)† | 0.17 (0.14–0.21)† |
| Agency ICC | 0.52* | 0.55 | 0.26* | 0.27 | 0.09* | 0.10 |
| Hospital ICC | 0.01* | 0.01 | 0.02* | 0.02 | 0.01* | 0.01 |
| LKW documentation | On‐scene ≤15 min | Prenotification | ||||
|---|---|---|---|---|---|---|
| Covariate | Unadjusted | Adjusted | Unadjusted | Adjusted | Unadjusted | Adjusted |
| Age, y | ||||||
| <60 | Reference | Reference | Reference | Reference | Reference | Reference |
| 60–69 | 1.11 (0.91–1.36) | 1.2 (0.92–1.57) | 0.80 (0.68–0.94)† | 0.8 (0.67–0.95)† | 0.99 (0.84–1.17) | 1.01 (0.83–1.24) |
| 70–79 | 1.20 (0.99–1.45) | 1.11 (0.86–1.43) | 0.77 (0.66–0.91)† | 0.73 (0.62–0.87)† | 1.12 (0.95–1.32) | 1.14 (0.94–1.4) |
| 80–89 | 1.29 (1.07–1.53) | 1.12 (0.86–1.44) | 0.71 (0.61–0.84)† | 0.65 (0.54–0.77)† | 1.07 (0.91–1.25) | 0.98 (0.8–1.19) |
| ≥90 | 1.33 (1.06–1.68) | 1.15 (0.84–1.58) | 0.61 (0.50–0.74)† | 0.57 (0.45–0.71)† | 0.92 (0.76–1.13) | 0.89 (0.69–1.15) |
| Women | 0.91 (0.80–1.02) | 0.91 (0.77–1.07) | 0.83 (0.75–0.92)† | 0.87 (0.78–0.97)† | 0.87 (0.79–0.97)† | 0.89 (0.78–1.01) |
| Race | ||||||
| White | Reference | Reference | Reference | Reference | Reference | Reference |
| Black | 0.42 (0.35–0.50)† | 0.88 (0.68–1.15) | 0.83 (0.73–0.94)† | 0.89 (0.75–1.05) | 0.69 (0.61–0.79)† | 1.05 (0.86–1.27) |
| Other/missing | 1.08 (0.84–1.38) | 0.96 (0.68–1.36) | 0.79 (0.63–1.00) | 0.85 (0.66–1.09) | 1.36 (1.04–1.73) | 0.97 (0.72–1.3) |
| Stroke subtype | ||||||
| IS/TIA | Reference | Reference | Reference | Reference | Reference | Reference |
| SAH | 0.40 (0.25–0.65)† | 0.4 (0.22–0.71)† | 0.77 (0.57–1.05) | 0.77 (0.55–1.08) | 0.56 (0.41–0.76)† | 0.63 (0.43–0.92)† |
| ICH | 0.81 (0.68–0.98) | 1.01 (0.78–1.31) | 1.02 (0.88–1.18) | 1.01 (0.85–1.2) | 0.98 (0.84–1.15) | 1.13 (0.93–1.38) |
| NIHSS | ||||||
| 0–6 | Reference | Reference | Reference | Reference | Reference | Reference |
| 6–11 | 1.35 (1.15–1.58) | 1.58 (1.28–1.96)† | 1.22 (1.06–1.41)† | 1.21 (1.04–1.4)† | 1.59 (1.37–1.85)† | 1.61 (1.35–1.91)† |
| 12–20 | 1.53 (1.28–1.82)† | 1.81 (1.43–2.29)† | 1.55 (1.32–1.82)† | 1.53 (1.29–1.81)† | 1.72 (1.45–2.04)† | 1.77 (1.45–2.16)† |
| >20 | 1.18 (0.95–1.46) | 1.6 (1.2–2.14)† | 1.32 (1.10–1.59)† | 1.36 (1.11–1.66)† | 1.59 (1.31–1.94)† | 1.54 (1.22–1.95)† |
| Missing | 0.63 (0.50–0.79) | 0.89 (0.63–1.26) | 0.81 (0.68–0.96) | 0.96 (0.78–1.18) | 0.62 (0.52–0.73) | 0.7 (0.55–0.89) |
| LKW‐to‐door | ||||||
| 0–120 | Reference | Reference | Reference | Reference | Reference | Reference |
| 121–360 | 0.84 (0.71–1.00)† | 0.78 (0.63–0.98)† | 0.62 (0.53–0.73)† | 0.62 (0.53–0.74)† | 0.96 (0.81–1.14)† | 0.91 (0.74–1.11)† |
| 361–720 | 0.81 (0.67–0.98)† | 0.81 (0.63–1.04) | 0.52 (0.43–0.62)† | 0.52 (0.43–0.62)† | 0.84 (0.70–1.02) | 0.78 (0.63–0.97)† |
| >720 | 0.54 (0.46–0.64)† | 0.52 (0.43–0.64)† | 0.50 (0.43–0.57)† | 0.5 (0.43–0.58)† | 0.51 (0.44–0.59)† | 0.48 (0.41–0.57)† |
| Missing | … | … | 0.39 (0.33–0.46)† | 0.41 (0.34–0.49)† | 0.45 (0.39–0.53)† | 0.44 (0.37–0.54)† |
| Agency ICC | 0.56* | 0.59 | 0.05* | 0.06 | 0.03* | 0.03 |
| Hospital ICC | 0.00* | 0.00 | 0.01* | 0.01 | 0.40* | 0.41 |
EMS indicates emergency medical services; ICC, intraclass correlation coefficient; ICH, intracerebral hemorrhage; IS, ischemic stroke; LKW, last known well; NIHSS, National Institutes of Health Stroke Scale; OR, odds ratio; PSS, prehospital stroke scale; SAH, subarachnoid hemorrhage; and TIA, transient ischemic attack.
Unadjusted intraclass correlation coefficient (ICC) is derived from the unconditional means model containing no fixed effects.
†
P<0.05.
In examining the relative contribution of transporting agency and destination hospital to variability in EMS quality measure compliance, we observed evidence of a substantial agency‐level effect for PSS and LKW documentation. For both measures, intraclass correlation coefficient values indicated that more than half of variation was attributable to the EMS agency (Table 4). The only measure for which destination hospital contributed to a large portion of overall variation was prenotification (intraclass correlation coefficient=0.41). Because of the extremely high degree of variation attributable to agency for PSS and LKW documentation, we repeated the crossed random‐effects models for these variables substituting agency EMR type for destination. In these models, 20% of variance in PSS performance and 31% of LKW documentation were attributable to EMR type after accounting for patient‐level factors and EMS agency‐level effects (Table S2).
DISCUSSION
To minimize the harm associated with stroke, rapid diagnosis and treatment are critical. 11 EMS providers are in a unique position to expedite care through early recognition of stroke, efficient transport to appropriate facilities, and prehospital activation of hospital‐based stroke teams. Clinical guidelines have outlined recommended prehospital actions that target each of these goals. 11 , 22 , 26 However, sources of variation in EMS stroke care are not well described. Our statewide sample of EMS‐transported stroke cases offers a unique opportunity to examine this question across a variety of practice settings.
Perhaps the most novel element of this analysis is our ability to quantify the contribution of agency‐level and hospital‐level variation to overall EMS variability in performance. We found that variation in each measure could be attributed in part to the transporting EMS agency. For measures such as on‐scene time and EMS stroke recognition, this contribution was relatively small. On the other hand, >50% of variation in PSS and LKW documentation was attributable to the transporting agency. While this may reflect true practice variation between agencies, other potential sources for this variation include differences in EMS documentation practices or problems with mapping of data elements to the statewide registry. These latter possibilities are highly probable given the fact that the hospital prenotification, the only metric derived from hospital data, demonstrated little EMS‐level variation. Ultimately, the quantification of group‐level variation is primarily beneficial in prompting further investigation to determine the underlying causes. Based on analysis of measures with substantial EMS agency‐level variation, roughly half of that variation was attributable to EMR software type. These findings suggest that quality and accuracy of EMS data in MI‐EMSIS may be improved by more closely examining data collection and upload processes from lower‐performing agencies and EMR software.
As has been demonstrated in previous studies, 16 , 17 , 18 , 31 , 32 , 33 , 34 we also identified that documentation of EMS quality indicators varied greatly by measure. Among currently recommended EMS practices for stroke care, the metric most consistently documented was a glucose check. While this finding is encouraging, 17.5% of all stroke transports were still missed. As hypoglycemia is a relatively common stroke mimic 35 , 36 that is correctable in the prehospital setting, and checking glucose is consistently required by stroke protocols, 37 this remains a logical target for quality improvement efforts. On the other hand, the documentation of LKW was disproportionately low compared with every other measure, which may imply that data capture issues are driving this finding. MI‐EMSIS contains separate fields for date/time of symptom onset and for date/time LKW. We suspect that EMS providers either document symptom onset time in lieu of LKW or record LKW in fields that fail to map properly to the designated LKW field in the MI‐EMSIS system. Further investigation of this issue is needed.
For the metrics PSS documentation, EMS stroke recognition, and prenotification, compliance rates averaged ≈50% to 60%. These rates are in keeping with previous studies of EMS performance. 16 , 38 Given the strong linkages observed between PSS documentation and EMS stroke recognition, 17 , 34 , 39 and between recognition, prenotification, and favorable emergency department stroke evaluation and treatment, 16 , 18 , 21 , 40 , 41 , 42 these findings strongly support the need to target these measures through prehospital stroke quality improvement programs. This is further underscored by the National EMS Quality Alliance, which recently endorsed a quality metric focused on prehospital stroke scale performance. 43 Previous studies have suggested that EMS compliance with these metrics can be improved through education and feedback, 31 , 44 , 45 although the impact of these efforts in the long term is less certain. 44
Demographic characteristics demonstrated only modest associations with EMS performance measure compliance. Advancing age demonstrated a particularly strong association with EMS scene times. This might be driven by mobility difficulties in older adults requiring greater levels of assistance from EMS providers, or higher rates of nursing home residence where ingress and egress from facilities may cause delays. Advancing age was positively associated with both PSS documentation and EMS recognition of stroke, which is not surprising given the well‐known association between advanced age and stroke risk. Nevertheless, LKW documentation and prenotification were unrelated to age, suggesting that once a case is recognized as a suspected stroke, prehospital providers proceeded without regard to the age of the patient. Female sex was associated with lower odds of both EMS stroke recognition in adjusted analysis. The lower odds of EMS stroke recognition among women has been described in at least one other previous study. 46 One possible explanation for this might be higher rates of atypical symptoms among women as has been reported in hospital‐based studies 47 ; however, further research is needed to investigate the potential causes for this finding. Finally, although Black race was associated with lower odds of EMS compliance for several measures in unadjusted analysis, these associations became nonsignificant following adjustment in multivariable models, suggesting that other factors account for much of the crude differences observed by race. Given the known racial disparities in stroke treatment and outcomes generally, 48 these findings are somewhat reassuring with respect to equity in prehospital stroke care.
Clinical characteristics often had the strongest associations with EMS compliance. Strokes with an NIHSS >6, ischemic strokes, and patients who presented earlier were at higher odds of EMS compliance for almost every measure. We suspect that more “obvious” stroke cases (severe strokes and those with symptoms such as unilateral weakness) are more likely to be correctly identified and treated appropriately by EMS, as previous studies demonstrate. 17 , 39 , 49 This phenomenon also likely explains the low odds of compliance among patients with subarachnoid hemorrhage, who often present with headache or altered mental status. Overall, these results are encouraging that patients most likely to receive optimal prehospital care are also the patients most likely to be candidates for acute ischemic stroke treatments. Targeting EMS education toward recognizing patients with stroke who have atypical presentations and low NIHSS may also be beneficial.
Limitations
Hospitals that participate in MASR tend to be larger, located in urban/suburban settings rather than rural areas, and are mostly certified primary or comprehensive stroke centers. As such, our findings should be applied with caution to less populated areas. As with all observational studies, the associations in this analysis should not be considered as causal relationships, and it is possible that additional unmeasured factors confound some of the associations we observed. Moreover, our data are limited to cases with confirmed stroke and are thus not able to address EMS quality of care among stroke mimics. However, our previous work has found that when EMS compliance was examined only among suspected stroke cases, compliance rates tended to be higher, 16 thus estimates of compliance we report are likely lower than what would be observed among all EMS‐suspected stroke cases. Another important consideration for the estimates of EMS compliance is that this analysis is entirely reliant on documentation by EMS that successfully mapped to a statewide database. Therefore, it is possible that outcomes such as performance measure compliance may not reflect actual care delivered in the field. For the stroke recognition metric, lower rates may be partially explained by EMS providers feeling reluctance to commit to the impression even if clinical suspicion is high enough to prompt otherwise appropriate prehospital stroke care and prenotification. Furthermore, although we calculated compliance using all cases as the denominator, this is not necessarily meant to imply that all these measures should have 100% compliance for all patients. For example, prenotification may not be critically important for the transport of a patient whose symptoms are known to be present for several days.
CONCLUSIONS
In this large state‐wide sample, EMS compliance with quality measures was variable. Practices such as obtaining a point of care glucose test were frequently performed; however, there are substantial opportunities for improvement in EMS stroke screening, recognition, and prenotification. Future studies are needed to investigate the different sources of variation between EMS agencies. Identifying agencies and EMR types with low rates of performance measure compliance may be a productive strategy to address known issues with missing data in MI‐EMSIS 50 by locating readily fixable data mapping problems that would result in more consistent and accurate estimates of actual EMS performance for stroke and other conditions.
Sources of Funding
The current publication was supported, in part, by DP‐21‐2102 Paul Coverdell National Acute Stroke Program, grant # 6 NU58DP006949–01‐02, from the Centers for Disease Control and Prevention.
Disclosures
The authors have no significant conflicts of interest to report.
Supporting information
Supplemental Material is available at https://www.ahajournals.org/doi/suppl/10.1161/JAHA.122.026834
See Editorial by Cash and Richards
For Sources of Funding and Disclosures, see page 10.
REFERENCES
- 1. Sayre MR, White LJ, Brown LH, McHenry SD. The National EMS research strategic plan. Prehosp Emerg Care. 2005;9:255–266. doi: 10.1080/10903120590962238 [DOI] [PubMed] [Google Scholar]
- 2. Pepe PE, Zachariah BS, Sayre MR, Floccare D. Ensuring the chain of recovery for stroke in your community. Acad Emerg Med. 1998;5:352–358. doi: 10.1111/j.1553-2712.1998.tb02721.x [DOI] [PubMed] [Google Scholar]
- 3. Glickman SW, Kit Delgado M, Hirshon JM, Hollander JE, Iwashyna TJ, Jacobs AK, Kilaru AS, Lorch SA, Mutter RL, Myers SR, et al. Defining and measuring successful emergency care networks: a research agenda. Acad Emerg Med. 2010;17:1297–1305. doi: 10.1111/j.1553-2712.2010.00930.x [DOI] [PubMed] [Google Scholar]
- 4. Kaji AH, Lewis RJ, Beavers‐May T, Berg R, Bulger E, Cairns C, Callaway C, Camargo CA Jr, Carcillo J, DeBiasi R, et al. Summary of NIH medical‐surgical emergency research roundtable held on April 30 to May 1, 2009. Ann Emerg Med. 2010;56:522–537. doi: 10.1016/j.annemergmed.2010.03.014 [DOI] [PubMed] [Google Scholar]
- 5. The History of NEMSIS . NHTSA's Office of EMS. Washington, DC: Office of EMS, NHTSA. https://nemsis.org/what‐is‐nemsis/history‐of‐nemsis/. Accessed April 21, 2021. [Google Scholar]
- 6. Hedges JR. Beyond utstein: implementation of a multisource uniform data base for prehospital cardiac arrest research. Ann Emerg Med. 1993;22:41–46. doi: 10.1016/s0196-0644(05)80248-3 [DOI] [PubMed] [Google Scholar]
- 7. Spaite D, Benoit R, Brown D, Cales R, Dawson D, Glass C, Kaufmann C, Pollock D, Ryan S, Yano EM. Uniform prehospital data elements and definitions: a report from the uniform prehospital emergency medical services data conference. Ann Emerg Med. 1995;25:525–534. doi: 10.1016/s0196-0644(95)70271-7 [DOI] [PubMed] [Google Scholar]
- 8. Dawson DE. National emergency medical services information system (NEMSIS). Prehosp Emerg Care. 2006;10:314–316. doi: 10.1080/10903120600724200 [DOI] [PubMed] [Google Scholar]
- 9. Becknell J, Simon L. Beyond EMS Data Collection:Envisioning an Information‐Driven Future for Emergency Medical Services. Vol 812. Washington, DC: Administration NHTS; 2016:361. [Google Scholar]
- 10. Richards CT, Fishe JN, Cash RE, Rivard MK, Brown KM, Martin‐Gill C, Panchal AR. Priorities for prehospital evidence‐based guideline development: a modified Delphi analysis. Prehosp Emerg Care. 2022;26:286–304. doi: 10.1080/10903127.2021.1894276 [DOI] [PubMed] [Google Scholar]
- 11. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, Biller J, Brown M, Demaerschalk BM, Hoh B, et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2019;50:e344–e418. doi: 10.1161/STR.0000000000000211 [DOI] [PubMed] [Google Scholar]
- 12. Asaithambi G, Tong X, Lakshminarayan K, Coleman King SM, George MG, Odom EC. Emergency medical services utilization for acute stroke care: analysis of the paul coverdell national acute stroke program, 2014‐2019. Prehosp Emerg Care. 2022;26:326–332. doi: 10.1080/10903127.2021.1877856 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Gache K, Couralet M, Nitenberg G, Leleu H, Minvielle E. The role of calling EMS versus using private transportation in improving the management of stroke in France. Prehosp Emerg Care. 2013;17:217–222. doi: 10.3109/10903127.2012.755584 [DOI] [PubMed] [Google Scholar]
- 14. Ekundayo OJ, Saver JL, Fonarow GC, Schwamm LH, Xian Y, Zhao X, Hernandez AF, Peterson ED, Cheng EM. Patterns of emergency medical services use and its association with timely stroke treatment: findings from get with the guidelines‐stroke. Circ Cardiovasc Qual Outcomes. 2013;6:262–269. doi: 10.1161/circoutcomes.113.000089 [DOI] [PubMed] [Google Scholar]
- 15. Kim DG, Kim YJ, Shin SD, Song KJ, Lee EJ, Lee YJ, Hong KJ, Park JO, Ro YS, Park YM. Effect of emergency medical service use on time interval from symptom onset to hospital admission for definitive care among patients with intracerebral hemorrhage: a multicenter observational study. Clini Exp Emerg Med. 2017;4:168–177. doi: 10.15441/ceem.16.147 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Oostema JA, Nasiri M, Chassee T, Reeves MJ. The quality of prehospital ischemic stroke care: compliance with guidelines and impact on In‐hospital stroke response. J Stroke Cerebrovasc Dis. 2014;23:2773–2779. doi: 10.1016/j.jstrokecerebrovasdis.2014.06.030 [DOI] [PubMed] [Google Scholar]
- 17. Oostema JA, Konen J, Chassee T, Nasiri M, Reeves MJ. Clinical predictors of accurate prehospital stroke recognition. Stroke. 2015;46:1513–1517. doi: 10.1161/strokeaha.115.008650 [DOI] [PubMed] [Google Scholar]
- 18. Abboud ME, Band R, Jia J, Pajerowski W, David G, Guo M, Mechem CC, Messe SR, Carr BG, Mullen MT. Recognition of stroke by EMS is associated with improvement in emergency department quality measures. Prehosp Emerg Care. 2016;20:729–736. doi: 10.1080/10903127.2016.1182602 [DOI] [PubMed] [Google Scholar]
- 19. Mosley I, Nicol M, Donnan G, Patrick I, Kerr F, Dewey H. The impact of ambulance practice on acute stroke care. Stroke. 2007;38:2765–2770. doi: 10.1161/strokeaha.107.483446 [DOI] [PubMed] [Google Scholar]
- 20. Nielsen VM, DeJoie‐Stanton C, Song G, Christie A, Guo J, Zachrison KS. The association between presentation by EMS and EMS prenotification with receipt of intravenous tissue‐type plasminogen activator in a state implementing stroke systems of care. Prehosp Emerg Care. 2019;24:319–325. doi: 10.1080/10903127.2019.1662862 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Lin CB, Peterson ED, Smith EE, Saver JL, Liang L, Xian Y, Olson DM, Shah BR, Hernandez AF, Schwamm LH, et al. Emergency medical service hospital prenotification is associated with improved evaluation and treatment of acute ischemic stroke. Circ Cardiovasc Qual Outcomes. 2012;5:514–522. doi: 10.1161/CIRCOUTCOMES.112.965210 [DOI] [PubMed] [Google Scholar]
- 22. Acker JE III, Pancioli AM, Crocco TJ, Eckstein MK, Jauch EC, Larrabee H, Meltzer NM, Mergendahl WC, Munn JW, Prentiss SM, et al. Implementation strategies for emergency medical services within stroke systems of care: a policy statement from the American Heart Association/American Stroke Association expert panel on emergency medical services systems and the stroke council. Stroke. 2007;38:3097–3115. doi: 10.1161/strokeaha.107.186094 [DOI] [PubMed] [Google Scholar]
- 23. Hodell E, Hughes SD, Corry M, Kivlehan S, Resler B, Sheon N, Govindarajan P. Paramedic perspectives on barriers to prehospital acute stroke recognition. Prehosp Emerg Care. 2016;20:415–424. doi: 10.3109/10903127.2015.1115933 [DOI] [PubMed] [Google Scholar]
- 24. Li T, Munder SP, Chaudhry A, Madan R, Gribko M, Arora R. Emergency medical services providers' knowledge, practices, and barriers to stroke management. Open Access Emerg Med. 2019;11:297–303. doi: 10.2147/oaem.S228240 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Oostema JA, Nickles A, Reeves MJ. A comparison of probabilistic and deterministic match strategies for linking prehospital and in‐hospital stroke registry data. J Stroke Cerebrovasc Dis. 2020;29:105151. doi: 10.1016/j.jstrokecerebrovasdis.2020.105151 [DOI] [PubMed] [Google Scholar]
- 26. Jauch EC, Schwamm LH, Panagos PD, Barbazzeni J, Dickson R, Dunne R, Foley J, Fraser JF, Lassers G, Martin‐Gill C, et al. Recommendations for regional stroke destination plans in rural, suburban, and urban communities from the prehospital stroke system of care consensus conference: a consensus statement from the American Academy of Neurology, American Heart Association/American Stroke Association, American Society of Neuroradiology, National Association of EMS Physicians, National Association of state EMS officials, Society of NeuroInterventional surgery, and Society of Vascular and Interventional Neurology: endorsed by the neurocritical care society. Stroke. 2021;52:e133–e152. doi: 10.1161/strokeaha.120.033228 [DOI] [PubMed] [Google Scholar]
- 27. Michigan State Protocol 3.2: Stroke/Suspected Stroke . https://www.michigan.gov/documents/mdhhs/Section_3_Adult_Specific_Treatment_613177_7.pdf Bureau of EMS T, and Preparedness; 2018.
- 28. Yule GU. On the methods of measuring association between two attributes. J R Stat Soc. 1912;75:579–642. doi: 10.1111/j.2397-2335.1912.tb00463.x [DOI] [Google Scholar]
- 29. Austin PC, Merlo J. Intermediate and advanced topics in multilevel logistic regression analysis. Stat Med. 2017;36:3257–3277. doi: 10.1002/sim.7336 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. David W Hosmer SL, And Sturdivant Rodney X.. Applied Logistic Regression. 3rd ed. Hoboken, NJ: John Wiley & Sons, Inc.; 2013. [Google Scholar]
- 31. Choi B, Tsai D, McGillivray CG, Amedee C, Sarafin J‐A, Silver B. Hospital‐directed feedback to emergency medical services improves prehospital performance. Stroke. 2014;45:2137–2140. doi: 10.1161/strokeaha.114.005679 [DOI] [PubMed] [Google Scholar]
- 32. Li T, Cushman JT, Shah MN, Kelly AG, Rich DQ, Jones CMC. Prehospital time intervals and management of ischemic stroke patients. Am J Emerg Med. 2020;42:127–131. doi: 10.1016/j.ajem.2020.02.006 [DOI] [PubMed] [Google Scholar]
- 33. Andersson E, Bohlin L, Herlitz J, Sundler AJ, Fekete Z, Andersson HM. Prehospital identification of patients with a final hospital diagnosis of stroke. Prehosp Disaster Med. 2018;33:63–70. doi: 10.1017/s1049023x17007178 [DOI] [PubMed] [Google Scholar]
- 34. Mould‐Millman NK, Meese H, Alattas I, Ido M, Yi I, Oyewumi T, Colman M, Frankel M, Yancey A II. Accuracy of prehospital identification of stroke in a large stroke belt municipality. Prehosp Emerg Care. 2018;22:734–742. doi: 10.1080/10903127.2018.1447620 [DOI] [PubMed] [Google Scholar]
- 35. Okano Y, Ishimatsu K, Kato Y, Yamaga J, Kuwahara K, Okumoto K, Wada K. Clinical features of stroke mimics in the emergency department. Acute Med Surg. 2018;5:241–248. doi: 10.1002/ams2.338 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Ramanathan RS, Wisco D, Vela‐Duarte D, Zafar A, Taqui A, Winners S, Buletko AB, Hustey F, Reimer A, Russman A, et al. Pre‐hospital diagnosis in mobile stroke unit. J Stroke Cerebrovasc Dis. 2021;30:105801. doi: 10.1016/j.jstrokecerebrovasdis.2021.105801 [DOI] [PubMed] [Google Scholar]
- 37. Glober NK, Sporer KA, Guluma KZ, Serra JP, Barger JA, Brown JF, Gilbert GH, Koenig KL, Rudnick EM, Salvucci AA. Acute stroke: current evidence‐based recommendations for prehospital care. West J Emerg Med. 2016;17:104–128. doi: 10.5811/westjem.2015.12.28995 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Lin CB, Peterson ED, Smith EE, Saver JL, Liang L, Xian Y, Olson DM, Shah BR, Hernandez AF, Schwamm LH, et al. Patterns, predictors, variations, and temporal trends in emergency medical service hospital prenotification for acute ischemic stroke. J Am Heart Assoc. 2012;1:e002345. doi: 10.1161/jaha.112.002345 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Gropen TI, Gokaldas R, Poleshuck R, Spencer J, Janjua N, Szarek M, Brandler ES, Levine SR. Factors related to the sensitivity of emergency medical service impression of stroke. Prehosp Emerg Care. 2014;18:387–392. doi: 10.3109/10903127.2013.864359 [DOI] [PubMed] [Google Scholar]
- 40. Tennyson JC, Michael SS, Youngren MN, Reznek MA. Delayed recognition of acute stroke by emergency department staff following failure to activate stroke by emergency medical services. West J Emerg Med. 2019;20:342–350. doi: 10.5811/westjem.2018.12.40577 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Sheppard JP, Mellor RM, Greenfield S, Mant J, Quinn T, Sandler D, Sims D, Singh S, Ward M, McManus RJ. The association between prehospital care and in‐hospital treatment decisions in acute stroke: a cohort study. Emerg Med J. 2015;32:93–99. doi: 10.1136/emermed-2013-203026 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Rostanski SK, Shahn Z, Elkind MSV, Liberman AL, Marshall RS, Stillman JI, Williams O, Willey JZ. Door‐to‐needle delays in minor stroke: a causal inference approach. Stroke. 2017;48:1980–1982. doi: 10.1161/strokeaha.117.017386 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Alliance NEQ . National EMS Quality Measure Set. Mc Lean, VA: National EMS Quality Alliance. https://www.nemsqa.org/nemsqa‐measures. 2021. Accessed April 31, 2022. [Google Scholar]
- 44. Oostema JA, Chassee T, Baer W, Edberg A, Reeves MJ. Brief educational intervention improves emergency medical services stroke recognition. Stroke. 2019;50:1193–1200. doi: 10.1161/strokeaha.118.023885 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Oser CS, McNamara MJ, Fogle CC, Gohdes D, Helgerson SD, Harwell TS. Educational outreach to improve emergency medical services systems of care for stroke in Montana. Prehosp Emerg Care. 2010;14:259–264. doi: 10.3109/10903120903572335 [DOI] [PubMed] [Google Scholar]
- 46. Govindarajan P, Friedman BT, Delgadillo JQ, Ghilarducci D, Cook LJ, Grimes B, McCulloch CE, Johnston SC. Race and sex disparities in prehospital recognition of acute stroke. Acad Emerg Med. 2015;22:264–272. doi: 10.1111/acem.12595 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Bushnell C, Howard VJ, Lisabeth L, Caso V, Gall S, Kleindorfer D, Chaturvedi S, Madsen TE, Demel SL, Lee SJ, et al. Sex differences in the evaluation and treatment of acute ischaemic stroke. Lancet Neurol. 2018;17:641–650. doi: 10.1016/s1474-4422(18)30201-1 [DOI] [PubMed] [Google Scholar]
- 48. Virani SS, Alonso A, Benjamin EJ, Bittencourt MS, Callaway CW, Carson AP, Chamberlain AM, Chang AR, Cheng S, Delling FN, et al. Heart Disease and Stroke Statistics: 2020 update: a report from the American Heart Association. Circulation. 2020;141:e139–e596. doi: 10.1161/CIR.0000000000000757 [DOI] [PubMed] [Google Scholar]
- 49. Brandler ES, Sharma M, McCullough F, Ben‐Eli D, Kaufman B, Khandelwal P, Helzner E, Sinert RH, Levine SR. Prehospital stroke identification: factors associated with diagnostic accuracy. J Stroke Cerebrovasc Dis. 2015;24:2161–2166. doi: 10.1016/j.jstrokecerebrovasdis.2015.06.004 [DOI] [PubMed] [Google Scholar]
- 50. Abir M, Taymour RK, Goldstick JE, Malsberger R, Forman J, Hammond S, Wahl K. Data missingness in the Michigan NEMSIS (MI‐EMSIS) dataset: a mixed‐methods study. Int J Emerg Med. 2021;14:22. doi: 10.1186/s12245-021-00343-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. Tibaldi FS, Verbeke G, Molenberghs G, Renard D, Van den Noortgate W, de Boeck P. Conditional mixed models with crossed random effects. Br J Math Stat Psychol. 2007;60:351–365. doi: 10.1348/000711006x110562 [DOI] [PubMed] [Google Scholar]
- 52. Meyers JL, Beretvas SN. The impact of inappropriate modeling of cross‐classified data structures. Multivar Behav Res. 2006;41:473–497. doi: 10.1207/s15327906mbr4104_3 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.