A low-cost, tablet-based option for prehospital neurologic assessment

Abstract

Objectives:

In this 2-center study, we assessed the technical feasibility and reliability of a low cost, tablet-based mobile telestroke option for ambulance transport and hypothesized that the NIH Stroke Scale (NIHSS) could be performed with similar reliability between remote and bedside examinations.

Methods:

We piloted our mobile telemedicine system in 2 geographic regions, central Virginia and the San Francisco Bay Area, utilizing commercial cellular networks for videoconferencing transmission. Standardized patients portrayed scripted stroke scenarios during ambulance transport and were evaluated by independent raters comparing bedside to remote mobile telestroke assessments. We used a mixed-effects regression model to determine intraclass correlation of the NIHSS between bedside and remote examinations (95% confidence interval).

Results:

We conducted 27 ambulance runs at both sites and successfully completed the NIHSS for all prehospital assessments without prohibitive technical interruption. The mean difference between bedside (face-to-face) and remote (video) NIHSS scores was 0.25 (1.00 to −0.50). Overall, correlation of the NIHSS between bedside and mobile telestroke assessments was 0.96 (0.92–0.98). In the mixed-effects regression model, there were no statistically significant differences accounting for method of evaluation or differences between sites.

Conclusions:

Utilizing a low-cost, tablet-based platform and commercial cellular networks, we can reliably perform prehospital neurologic assessments in both rural and urban settings. Further research is needed to establish the reliability and validity of prehospital mobile telestroke assessment in live patients presenting with acute neurologic symptoms.

The ability to initiate rapid treatment of acute stroke is limited by prehospital barriers, including delays in hospital arrival, inaccuracies in stroke screening, and proximity to centers with neurologic expertise.^1–3 Numerous initiatives, including the American Heart Association/American Stroke Association (AHA/ASA) Target: Stroke program, have identified the need for innovative approaches to prehospital stroke care.^4,5 Given recent evidence supporting endovascular therapy in addition to thrombolysis for select stroke patients, there is an even greater need for timely prehospital diagnosis and triage based on stroke severity.^6,7

Early studies have explored the concept of ambulance-based telemedicine to facilitate rapid and accurate triage of acute stroke in the field. Although promising, these pilot studies demonstrated technical challenges including earlier-generation broadband, limiting technical feasibility.^8–10 More recent studies incorporating modern cellular connectivity have shown greater reliability but involve costly telemedicine endpoints and a limited geographic scope primarily confined to urban areas.^11,12

The objective of our study, Improving Treatment with Rapid Evaluation of Acute Stroke via Mobile Telemedicine (iTREAT), is to demonstrate the reliability and technical feasibility of bidirectional, ambulance-based videoconferencing using a low cost, portable mobile telestroke system. Utilizing fourth-generation (4G) long-term evolution (LTE) commercial broadband, we developed a tablet-based platform for emergency medical services (EMS) transport and assessed feasibility using simulated stroke scenarios. We hypothesized that (1) mobile telestroke assessments are clinically reliable as determined by correlation of the NIHSS score of ≥0.90 between bedside and remote evaluations, and (2) 80% of iTREAT test runs could be completed without prohibitive technical interruption.

METHODS

Setting.

The simulation study was conducted in 2 distinct EMS systems comprising rural and urban catchments at the University of Virginia (UVA) and University of California, San Francisco (UCSF) Health Systems, respectively. In preparation for the study, investigators mapped cellular connectivity, distance, and transport times along ambulance routes triaging to the receiving hospital, UVA Medical Center.¹³ To ensure optimization of mobile broadband during ambulance transport, we partnered with Verizon Wireless, the largest supplier of 4G LTE coverage in the region. We targeted 9 minutes of continuous connectivity to allow for completion of a focused neurologic examination using the NIHSS in the field. Less formal mapping was undertaken at UCSF (Bay Area) because of saturation of cellular coverage in the study region.

In central Virginia, the Thomas Jefferson Emergency Medical Services (TJEMS) Council, Inc., comprises 6 mainly rural counties and more than 35 ambulance agencies. Partnering TJEMS agencies serve a patient population of more than 250,000 in geographic regions designated as rural and medically underserved by the Health Resources and Services Administration (HRSA) and the Federal Communications Commission.

In the Bay Area, our study setting was the City of Berkeley served by the EMS division of the Berkeley Fire Department. The land area is about 11 square miles with an estimated population of 116,768 (2013). The total number of medical calls received by the fire department is approximately 8,000/y. The first responders include fire fighter/paramedics, and the ambulance personnel are paramedics who are responsible for stabilizing and transporting to the medical facility.

The geographic distribution of these different EMS populations allows 2 necessary components of a mobile telestroke encounter: (1) adequate transport time for videoconferencing to obtain relevant clinical information, e.g., initial stroke history, the NIHSS examination score, initial eligibility for thrombolysis or thrombectomy, and (2) transport within a reliable wireless cellular network. To account for initial EMS screening and time to initiate a videoconferencing call before the mobile telestroke encounter, we designated ambulance routes with a minimum 10 to 15 minutes of emergency transport time.

Simulation scenarios.

At the Virginia site, 4 scripted stroke scenarios (2 middle cerebral artery ischemic strokes, one right posterior cerebral artery ischemic stroke, one brainstem hemorrhage) and 2 scripted stroke-mimic scenarios (postictal state and brachial plexopathy) were portrayed by standardized patient actors (i.e., medical students provided a script and trained in the components of the NIHSS). Each of the 6 scenarios was assigned to 1 of 6 primary ambulance routes to UVA Medical Center in order to capture connectivity and audiovisual (AV) quality along each route. Each scenario was portrayed 4 times: twice in-person at the bedside and twice remotely from the back of the ambulance using our mobile telestroke system. To calculate both intra- and interrater correlation between methods, 2 blinded vascular neurologists determined the NIHSS score for both the bedside and mobile telestroke encounters. Raters alternated the order of performing the bedside and remote evaluations to reduce bias. The NIHSS takes 5 to 8 minutes to complete and is reliable across a remote videoconferencing link including use of both the iPhone and iPad for bedside testing.^14–16

At the Bay Area site, we used 15 scripted scenarios, including acute stroke syndromes and stroke mimics. The scenarios were portrayed by standardized patients and evaluated concurrently by a vascular neurologist in the ambulance and 2 remote vascular neurologists via the mobile telestroke platform.

Study personnel.

All raters were NIHSS certified and independently blinded to the other assessments. In addition to performing the NIHSS, remote examiners provided feedback on the audio/video quality with each scenario in order to optimize performance. Prehospital providers and study investigators assisted with remote assessments as tele-presenters for the NIHSS, similar to a bedside emergency room nurse or technician in conventional telestroke encounters.

Mobile telestroke system.

Initial system development and feasibility testing have been presented elsewhere.¹³ During the simulation study, the system included an Apple iPad with retina display (2,048-by-1,536 resolution at 264 pixels per inch), cradled mount, 4G LTE CradlePoint modem with Verizon 4G Mini SIM (Subscriber Identification Module) card, and externally mounted cellular antennae. For live video streaming, we used the Cisco Jabber (Movi) videoconferencing application, which enables encrypted bidirectional telecommunication and meets security standards for transmission of protected health information—HIPAA (Health Insurance Portability and Accountability Act) secure and firewall protected through a virtual private network. This application is currently in use for clinical practice within the UVA Center for Telehealth and Stroke Telemedicine and Tele-education (STAT) Program and the University of California, San Francisco Telehealth program.

All system components can be securely stored for compact mobility and ambulance readiness in a standard Pelican case. Depending on ambulance design, the tablet cradle can be mounted to the ambulance gurney or roll bar using a portable clamp, or to a window, sidewall, or under a shelf using a suction mount depending on optimal patient view (figure 1).

Figure 1. iTREAT ambulance setup with cradled iPad and suction mounting.

The neurologist is communicating directly with a simulated patient to perform the NIH Stroke Scale while a local emergency medical services provider assists as tele-presenter. iTREAT = Improving Treatment with Rapid Evaluation of Acute Stroke via Mobile Telemedicine.

The total hardware cost of the system, excluding commercial connectivity fees and data packaging, is approximately $1,650.¹³

Data collection.

During technical feasibility testing at the UVA site, we developed a connectivity map by recording AV quality during live videoconferencing along each of our 6 primary ambulance routes. AV quality was recorded using an adapted 6-point scale⁹ as measured by 2 independent raters from both the vehicle and hospital. We also recorded the time of continuous, uninterrupted video transmission along each route. During initial connectivity testing in central Virginia, a total of 31 test runs were conducted along 6 primary ambulance routes with 93% achieving a prespecified minimum of 9 minutes of continuous live video transmission. Mean continuous mobile connectivity time was 18 minutes.¹³

For convenience of study personnel and EMS resources, all scenarios were conducted during regular daytime hours. In addition to quality ratings and NIHSS scoring, data were collected via an online form including basic information on test run characteristics such as ambulance location, weather, time of day, and any additional technical limitations.

Data analysis.

Our primary analysis was correlation of the NIHSS score between the bedside and remote examiners. We used a mixed-effects regression model accounting for scenario and rater as random effects and order of exposure and face-to-face vs mobile assessment as fixed effects. As the NIHSS is an ordinal scale, we chose to measure intraclass correlation of the total NIHSS scores between bedside and remote encounters with 95% confidence interval (CI) using the SAS icc9 macro (SAS Institute, Cary, NC).¹⁷ We then examined Bland-Altman (BA) plots,¹⁸ which plot the difference in scores between bedside and remote evaluations vs the average of the same 2 quantities. These plots show discrepancies between the 2 measurements as they depend on the severity of the scenario. Comparisons included both the total NIHSS scores and discrepancies on individual NIHSS components.

In the combined analysis for both centers, we modeled the total NIHSS scores using a mixed-effects model. We treated the scenario as a random effect and adjusted for site, mobile vs face-to-face, and effect modification by site or method of evaluation to assess whether or not the differences between bedside and remote scores were similar between the central Virginia and Bay Area sites.

We defined clinical reliability as correlation of the NIHSS score ≥0.90 between bedside and remote evaluations. As a secondary outcome measure, we prespecified technical feasibility as the ability to complete an evaluation for 80% of stroke scenarios without prohibitive technological interruption. Statistical analyses were conducted with SAS version 9.4 (SAS Institute) and R version 3.2.

Standard protocol approvals, registrations, and patient consents.

This feasibility study was conducted with protocol approval from both local institutional review boards. All study investigators completed Collaborative Institutional Training Initiative Certification in accordance with ethical standards for human subjects research.

RESULTS

All simulated mobile telestroke encounters were completed without prohibitive technical interruption (n = 27). NIHSS scores were very similar between bedside (face-to-face) and remote (video) assessments: intraclass correlation for the Virginia site (UVA) was 0.98 (95% CI 0.94–1.00), for the Bay Area (UCSF) 0.94 (95% CI 0.87–0.98), and for both sites combined was 0.96 (95% CI 0.92–0.98) (figure 2).

Figure 2. Correlation graph: Combined analysis.

Points in the plot above represent average video scores (remote) plotted vs average face-to-face scores (bedside) for each simulation scenario. UCSF = University of California, San Francisco (Bay Area); UVa = University of Virginia (central Virginia).

For the BA plot, which graphs the difference in scores vs the average, the average difference between bedside (face-to-face) and remote (video) NIHSS scores was 0.25 (95% CI 1.00 to −0.50) (figure 3). Similarly, the BA plot for the itemized NIHSS components showed fairly even distribution across the x-axis with no significant differences on individual items (figure 4). Of note, outlier differences for motor arm/leg at the UCSF site (4-point difference) were likely attributable to left/right confusion on the scoring sheet for one of the scenarios.

Figure 3. Bland-Altman plot demonstrating discrepancies between mobile video and bedside evaluations by stroke severity.

The vertical axis represents the difference between the video (remote) and the face-to-face (bedside) NIHSS scores. The horizontal axis represents the average of the scores for each scenario. All averages are weighted to account for differences in number of face-to-face and video evaluations. NIHSS = NIH Stroke Scale; UCSF = University of California, San Francisco (Bay Area); UVa = University of Virginia (central Virginia).

Figure 4. Bland-Altman plot for itemized NIHSS components.

The vertical axis represents the difference between video (remote) and face-to-face (bedside) NIHSS scores for individual items. The horizontal axis represents the average of the item scores for each scenario. Outlier differences for motor arm/leg left/right (4-point difference) were likely attributable to left/right confusion on the scoring sheet for one of the scenarios at the UCSF site. Loc = level of consciousness; Loc C = level of consciousness commands; Loc Q = level of consciousness questions; NIHSS = NIH Stroke Scale; UCSF = University of California, San Francisco (Bay area); UVa = University of Virginia (central Virginia).

In the mixed-effects regression model, there were no statistically significant differences accounting for bedside vs remote evaluation, differences in site, or interaction between method of evaluation and site.

DISCUSSION

These initial results suggest the technical feasibility and clinical reliability of a low-cost, utilitarian mobile telemedicine system for bidirectional video communication between hospital-based neurologists and ambulance-based providers during prehospital transport. We achieved similar correlation of the NIHSS score between remote and ambulance evaluations during simulated testing in both a rural and urban EMS system. Similar to previous studies, we found that maintaining continuous, high-quality AV connectivity to facilitate mobile telestroke encounters depends on the integrity of 4G broadband along ambulance routes.

Extrapolating from current hub-and-spoke models of outreach telemedicine, mobile telestroke potentiates face-to-face neurologic expertise for EMS providers and acute stroke patients in the prehospital setting. A 2009 AHA/ASA scientific statement reviewing the evidence for telestroke systems of care determined that provision of stroke expertise to a prehospital unit in transport via videoconferencing could “increase diagnostic accuracy, provide earlier resource mobilization, and increase appropriate triage”; however, the statement reasonably questioned both practicality and utility of EMS-based telemedicine and called for further research.¹⁹ The TeleBAT study was the first to publish data showing that ambulance-based mobile telestroke was potentially feasible to evaluate stroke deficits on the NIHSS using a commercially available wireless network and a customized telemedicine platform.²⁰ However, agreement on most NIHSS items was modest, and transmissions were constrained by relatively low bandwidth compared to today's standards.⁸ More recent mobile telestroke demonstrations in Europe have shown conflicting but promising feasibility results, highlighting the importance of 4G vs 3G mobile broadband to achieve reliable continuous connectivity for uninterrupted communication.^9–11

In the United States, investigators in Houston, TX, reported successful feasibility and reliability of telemedicine in a mobile stroke unit using the RPXpress portable telemedicine system from InTouch Technologies, Inc. (Santa Barbara, CA).¹² Similar to our study, they utilized patient actors performing simulated stroke scenarios during ambulance transport and were able to demonstrate excellent intraclass correlation (0.997, 0.97–0.998) for the NIHSS score between real-time and scripted scenarios. Ambulance connectivity was achieved through a Verizon 4G LTE mobile hotspot allowing successful teleconsultation in 34 of 40 scenarios (85%) without major technical interruption. Similar to previous ambulance-based telemedicine studies in Europe, the geographic radius was limited to the surrounding metropolitan area.

Our results expand on ambulance-based mobile telestroke by offering a tool that is low cost and easy to use with readily available components. In addition, our results further the generalizability of mobile telestroke by successful testing in different EMS systems, including both urban and rural networks. Similar to conventional telestroke applications, rural and low-access areas may benefit the most from ambulance-based mobile telemedicine, particularly given the geographic disparity in proximity to neurologic expertise and longer transport times from stroke onset to hospital arrival.

Several observational studies have described the benefits of early recognition and prehospital notification on outcomes. The Helsinki group demonstrated that through use of multiple concurrent strategies, including EMS education and prenotification implemented over a decade, the rate of IV tissue plasminogen activator treatment in their hospital increased 10-fold with a concomitant reduction in the median door-to-treatment time to 20 minutes.²¹ Target: Stroke, a national quality-improvement initiative by the AHA/ASA, and other studies have shown an improvement in treatment rates and time targets after implementation of prehospital notification aimed at better coordination and streamlining of acute stroke care.^4,5,22,23 However, universal acceptance of prehospital notification has been limited by regional and institutional variability, with a high number of false-positive activations resulting in system fatigue at many stroke centers.^1,2,24–31 We believe that an interactive and bidirectional evaluation, facilitated by mobile telestroke, can improve the accuracy of prehospital diagnosis and lead to better stroke outcomes without overburdening the system. Furthermore, with recent trials showing a clear benefit for early recognition and triage of stroke patients with a large vessel occlusion, remote vascular neurology evaluation using telemedicine could help identify the subset of patients who may benefit from rapid triage to comprehensive/endovascular-capable stroke centers.⁶ In this way, an ambulance-based tele-neurologic examination could inform the prehospital management of acute stroke similar to an ECG guiding prehospital management in acute coronary syndrome.^32,33 A translatable mobile telemedicine system also has applicability to prehospital stroke research, such as the FAST-MAG (Field Administration of Stroke Therapy–Magnesium) Trial, which demonstrated the potential for bidirectional video communication for ambulance-based informed consent.³⁴

Our study has several limitations. This was a simulation trial in the prehospital setting, without sirens or other unanticipated distractions, and may not fully reflect the evaluation of emergency stroke scenarios by prehospital providers. While NIHSS assessments were performed independently among trained examiners, raters may have been biased toward completing the examination and achieving good results with the mobile telemedicine platform. That said, we saw no significant differences between site or method of evaluation in our mixed-effects model. We noted at least one discrepancy likely attributable to rater error in scoring individual components on the NIHSS, but noted no clinically relevant differences on itemized testing overall. Further study may be needed to validate whether individual NIHSS components are more or less reliable using our method. We are also unable to fully assess differences in performance by stroke severity, particularly at higher NIHSS scores with limited data points. We anticipate a more diverse case mix with prospective study in live patient scenarios.

In addition, EMS providers may be reluctant to maintain a video call with a remote neurologist while stabilizing an acute stroke patient, which could be challenging in real-world scenarios. In anticipation of future testing in live patient encounters, we would exclude patients meeting certain criteria such as cardiac or vital sign instability, respiratory distress, or major trauma. We also plan to incorporate more formal training and feedback on the protocol and efficient use of the mobile telemedicine equipment.

We benefited greatly from the technical support and infrastructure of established telemedicine programs. While telemedicine technology and clinical practice continue to evolve, resources remain limited in many geographically underserved areas. However, dissemination of mobile telemedicine in the United States is buttressed by federal programs eager to support novel applications of telemedicine, such as the Federal Communications Commission's Connect America Fund,³⁵ and HRSA's regional funding of Telehealth Resource Centers.³⁶

Generalizability of our mobile telemedicine platform to other health systems would require a refined appreciation for area-specific cellular connectivity, in particular the ability to maximize 4G broadband along EMS routes. In the Houston study, for example, investigators noted several technically unsuccessful teleconsultations reflective of poor wireless connectivity in certain areas with weak coverage; however, utilizing multiple service providers could enhance these dead zones, depending on the network map. We also tested on preselected ambulance routes. While this may reduce the external validity of our data, we were able to demonstrate correlative results in 2 geographically distinct EMS systems, both urban and rural.

Lastly, this was a small pilot study and we are not able to account for or investigate all possible variables affecting the performance of an ambulance-based neurologic assessment. While we captured basic data on weather and time of day, further testing will require correlating results during after-hours, varying periods of cellular traffic, and changing weather states. In addition, different ambulance environments will both influence the telemedicine encounter and the mounting and positioning of our tablet-based platform. Having documented more than 50 test runs in our investigations thus far, we have learned that the portability and adaptability of our apparatus is essential to successful implementation in varying prehospital settings.

This simulated study demonstrates that a low-cost, tablet-based mobile telemedicine system can facilitate prehospital neurologic assessment during ambulance transport. The clinical reliability of our platform in both rural and urban environments supports the generalizability of mobile telestroke in any EMS system with adequate transport time and broadband coverage. Future research is needed in live patient scenarios to further verify feasibility and investigate measures of efficacy for translation to clinical practice.

GLOSSARY

AHA

American Heart Association

ASA

American Stroke Association

AV

audiovisual

BA

Bland-Altman

CI

confidence interval

EMS

emergency medical services

4G

fourth generation

HRSA

Health Resources and Services Administration

iTREAT

Improving Treatment with Rapid Evaluation of Acute Stroke via Mobile Telemedicine

LTE

long-term evolution

NIHSS

NIH Stroke Scale

TJEMS

Thomas Jefferson Emergency Medical Services

UCSF

University of California, San Francisco

UVA

University of Virginia

AUTHOR CONTRIBUTIONS

S.N. Chapman Smith, MD: contributed to design and conceptualization of the study, analysis and interpretation of the data, and drafting/revising the manuscript for intellectual content. P. Govindarajan, MD, MAS: contributed to design and conceptualization of the study, analysis and interpretation of the data, and drafting/revising the manuscript for intellectual content. M.M. Padrick, MD: contributed to design and conceptualization of the study, analysis and interpretation of the data. J.M. Lippman, BA: contributed to design and conceptualization of the study, analysis and interpretation of the data. T.L. McMurry, PhD: contributed to design and conceptualization of the study, analysis and interpretation of the data, and provided statistical support. B.L. Resler, MD: contributed to design and conceptualization of the study, analysis and interpretation of the data. K. Keenan, MD: contributed to design and conceptualization of the study, analysis and interpretation of the data. B.S. Gunnell, BA: contributed to design and conceptualization of the study, analysis and interpretation of the data, and provided telemedicine support. P. Mehndiratta, MD: contributed to design and conceptualization of the study. C.Y. Chee, MD: contributed to design and conceptualization of the study. E.A. Cahill, MD: contributed to design and conceptualization of the study. C. Dietiker, MD: contributed to design and conceptualization of the study. D.C. Cattell-Gordon, MDiv, MSW: contributed to design and conceptualization of the study. W.S. Smith, MD, PhD: contributed to design and conceptualization of the study. D.G. Perina, MD: contributed to creating prehospital protocols, coordinating prehospital deployment, revising the manuscript for intellectual content. N.J. Solenski, MD: contributed to design and conceptualization of the study. B.B. Worrall, MD, MSc: contributed to design and conceptualization of the study, analysis and interpretation of the data, and revising the manuscript for intellectual content. A.M. Southerland, MD, MSc: contributed to design and conceptualization of the study, analysis and interpretation of the data, drafting/revising the manuscript for intellectual content, and provided study oversight.

STUDY FUNDING

Study design, implementation, and analysis was supported by a grant from the Health Resources and Services Administration (HRSA GO1RH27869-01-00; principal investigator [PI] Solenski), American Heart Association (AHA) Grant-In-Aid award from the Western States Affiliate Program (PI Govindarajan), Virginia Alliance of Emergency Medicine Research Award (PI Chapman Smith), and the University of Virginia (UVA) Neuroscience Center of Excellence (PI Southerland). Student effort was supported by the AHA Student Scholarship in Cerebrovascular Diseases and Stroke and the UVA Medical Student Summer Research Program (M.M.P., J.M.L.).

DISCLOSURE

S. Chapman Smith: research support from the Virginia Alliance of Emergency Medicine Research Award; provisional US patent 61/867,477. P. Govindarajan: research support from the American Heart Association Grant-In-Aid award from the Western States Affiliate Program. M. Padrick: financial support from the American Heart Association Student Scholarship in Cerebrovascular Diseases and Stroke. J. Lippman: financial support from an American Heart Association Student Scholarship in Cerebrovascular Diseases and Stroke. T. McMurry, B. Resler, and K. Keenan report no disclosures relevant to the manuscript. B. Gunnell: provisional US patent 61/867,477. P. Mehndiratta, C. Chee, E. Cahill, C. Dietiker, D. Cattell-Gordon, W. Smith, and D. Perina report no disclosures relevant to the manuscript. N. Solenski receives research support from HRSA GO1RH27869-01-00. B. Worrall: serves as associate editor for Neurology®. A. Southerland: provisional US patent 61/867,477; research support from HRSA GO1RH27869-01-00 (PI Solenski) and the UVA Neuroscience Center of Excellence; speaker for America's Essential Hospitals; Dr. Southerland also serves as deputy section editor for the Neurology Podcast. Go to Neurology.org for full disclosures.