Pre-hospital Utility of Rapid Stroke evaluation Using In-ambulance Telemedicine (PURSUIT): A Pilot Feasibility Study

Abstract

Background and Purpose

: Pre-hospital evaluation using telemedicine (TM) may accelerate acute stroke treatment with tissue plasminogen activator (t-PA). We explored the feasibility and reliability of using TM in the field and ambulance to help evaluate acute stroke patients.

Methods

: Ten unique, scripted stroke scenarios, each conducted four times, were portrayed by trained actors retrieved and transported by Houston Fire Department emergency medical technicians (EMT) to our stroke center. The vascular neurologists (VN) performed remote assessments in real-time, obtaining clinical data points and NIH Stroke-Scale (NIHSS) using the In-Touch RP-Xpress TM device. Each scripted scenario was recorded for a subsequent evaluation by a second blinded VN.

Study feasibility was defined by the ability to conduct 80% of the sessions without major technological limitations. Reliability of video interpretation was defined by a 90% concordance between the data derived during the real-time sessions and the scripted scenarios.

Results

In 34/40 (85%) scenarios, the teleconsultation was conducted without major technical complication. The absolute agreement for intra-class-correlation (ICC) was 0.997 (95% CI: 0.992-0.999) for the NIHSS obtained during the real-time sessions and 0.993 (95% CI: 0.975-0.999) for the recorded sessions. Inter-rater agreement using κ-statistics showed that for live-raters, 10/15 items on the NIHSS showed excellent agreement, and 5/15 showed moderate agreement. Matching of real-time assessments occurred for 88% (30/34) of NIHSS scores by ±2 points, and 96% of the clinical information.

Conclusions

Mobile TM is reliable and feasible in assessing actors simulating acute stroke in the pre-hospital setting.

INTRODUCTION

Treatment with intravenous (IV) t-PA is an effective treatment for acute ischemic stroke within the first few hours of symptom onset.^{1, 2} Earlier treatment is associated with a higher likelihood of better outcome. ^{3, 4} This finding has led to national guidelines setting target door to treatment times at 60 minutes for eligible acute ischemic stroke (AIS) patients.⁵ Unfortunately, less than one-third of patients treated with IV t-PA received the treatment in this recommended 60 minute time window from hospital arrival.⁶

Emergency Medical Technicians (EMT) are the first point of contact for the majority of acute stroke patients in Houston. Their initial assessment and recognition of stroke symptoms play a pivotal role in ensuring that the patient is triaged to the appropriate hospital facility. Studies have shown that with rigorous EMT education, diagnostic accuracy and rapid transport times can be achieved.⁷ Prenotification by EMT has also been demonstrated to improve AIS treatment times and increase use of lytics. ⁸

A potential approach to further reduce treatment times is to utilize telemedicine (TM) in the field and ambulance to assess and rapidly triage acute stroke patients. TM technology allows for two way high quality video conferencing using cellular bandwidth. Enabling a stroke specialist to assess the patient in collaboration with EMS staff during the pick-up and transport could potentially reduce the time spent at hospital arrival to obtain a history and examine the patient. In addition, pre-hospital identification and registration of the potential thrombolytic candidate, and advance notification to the receiving hospital to prioritize imaging and laboratory processes, could lead to reduced time spent in initial evaluation at the hospital. Lastly, pre-hospital TM consultation may help to maintain homeostasis in the vulnerable stroke patient, such as hypotension, cardiac arrhythmias, or dysglycemia which may also help to protect the ischemic penumbra prior to thrombolysis.

Before TM can be applied in the pre-hospital setting for acute stroke evaluation, it is first necessary to determine the feasibility and reliability of mobile two way audio-visual TM technology using the internet while patients are being transported from the field to the hospital. We therefore conducted the PURSUIT study, a pilot feasibility study that used actors performing pre-scripted stroke scenarios of varying stroke severity to simulate live acute stroke assessments in the field and the ambulance. The objective of our study was to explore the feasibility and reliability of using mobile TM technology in the pre-hospital setting to help identify, triage, and evaluate AIS patients.

RESEARCH DESIGN AND METHODS

Overview

Ten unique, scripted stroke scenarios with various levels of stroke severity ranging from 0 to 27 on the NIHSS and various stroke syndromes (left and right middle cerebral artery (MCA) syndromes, and cortical and subcortical syndromes, see figure 1 for sample case) were portrayed by trained actors retrieved and transported by EMT to our stroke center. Specifically, two scenarios were transient ischemic attacks (NIHSS = 0), three cases were LMCA cortical syndromes (NIHSS = 10, 23, 27), two were RMCA cortical syndromes (NIHSS = 19, 17), two cases were subcortical syndromes (NIHSS = 3, 5) and one case was a posterior circulation syndrome (NIHSS = 7). The case mix was selected to reflect the typical case presentation we see at our facility, and we only selected one posterior circulation stroke scenarios as those cases can be difficult to examine even in person.⁹ Each scenario was conducted 4 times for a total of 40 scenarios. Each scenario began when both the VN and EMT were simultaneously notified of the potential stroke patient, from a “virtual” dispatch center, continued with EMT arrival on scene and a vascular neurologist (VN) simultaneously logged on to the TM device remotely, and ended once the ambulance arrived at our emergency department. Each scenario was recorded for a subsequent evaluation by a second blinded VN. Clinical data points were gathered either onsite or during transport but all neurological assessments were conducted during transportation.

Figure 1.

Sample case scenario.

Trained Actors and Case Scenarios

The actors were all members of the UT Houston Stroke team (stroke research nurses/coordinators and senior neurology residents) and were trained by one of the authors, a vascular neurologist, on the script and the portrayal of stroke deficits to be detected on the NIH Stroke Scale (NIHSS). The actors rehearsed the script with one of the authors and memorized the key portions of the history and clinical scenario. All of the actors were NIHSS certified. The script consisted of 12 clinical data points including the name of the patient, gender, age, time of onset, past medical history, medications, vitals and basic clinical situation and NIHSS. Each scenario location was chosen with varying distances ranging from three to 10 miles from the ED and in different environments (four indoor locations, one outdoor suburban location, and two outdoor downtown locations). Locations were chosen without prior knowledge or consultation of 4G/LTE network coverage maps.

Vascular Neurologists

All vascular neurologists (VNs) are NIHSS certified and were blinded to all scripted scenarios performed by the actors. The group was comprised of one stroke attending and nine stroke fellows. VNs were trained by one of the authors on the use of the telemedicine software and remote control of the telemedicine device; and all had experience in performing acute teleneurology consultations in the emergency room setting. The first group of VNs performed remote assessments of the scripted scenarios in real-time, obtaining clinical data points and NIHSS, and provided feedback regarding audio/video quality after completion of each real-time session scenario. The second group of VNs abstracted the clinical data points and NIHSS from the recorded session of each scripted scenario captured on TM. All of the examiners were separated when conducting the assessments and were blinded to assessments conducted by prior examiners.

EMT Personnel

Five active EMT personnel were trained by two of the authors on the use of the TM equipment, and were provided elements of the case such as vital signs and finger-stick glucose values. They were instructed to provide their standard assessments and in the field protocols including the use of sirens during transportation for each scenario. EMT Personnel were not specifically trained to perform the NIHSS, but assisted in the neurological exam with instructions from the remote VN. Vital signs and blood glucose levels were transmitted orally to the remote VN during the evaluation. This process is similar to our current practice when our stroke team is pre-notified by the Houston Fire Department about an incoming stroke patient and vitals and blood glucose level are communicated via telephone.

Technology

We utilized existing portable telemedicine units available from In Touch Health (Santa Barbara, CA; www.intouchhealth.com). The RP-Xpress System is a mobile, robotic communications platform that allows physicians bidirectional communication over remote distances via the Internet comprised of audio and video utilizing a 175° field of view fisheye camera capable of 6× zoom, and a high quality hyper-cardioid microphone and full range speaker integrated into the portable device. The device is FDA-approved for patient monitoring and connection to diagnostic medical devices. The established connection is HIPPA compliant and encrypted. VNs connected to the device from a desktop computer located at the University of Texas at Houston Medical School Building. Connections are encrypted using a combination of RSA public/private key and 256-bit AES symmetric encryption to ensure confidentiality of patient information transmitted.

The RP-Xpress was mounted on the EMT stretchers and brought with the EMT personnel to the patient's location (Figure 2). This allowed full view of the patient from head to toe and the zooming function allowed for adequate resolution to examine eye movements and facial asymmetry. The C-arm attachment also allowed the EMT personnel to move and reposition the device if needed during evaluation. Verizon Jetpack 4G LTE mobile hotspot (4620LE) was used to provide a 4G LTE WWAN connection, and was tethered to RP-Xpress via a private, encrypted WLAN giving cable free coverage in and around the ambulance, and able to follow RP-Xpress into buildings via attachment to the stretcher. The mobile hotspot was placed on the dashboard of the ambulance for optimal connection.

Figure 2.

A) Remote vascular neurologist performing stroke evaluation; B) RP-Xpress mounted on stretcher in ambulance; C) Remote assessment in ambulance with EMS assistance; D) Remote assessment on scene in patient home with EMS.

Data Collection

The following objective data points were collected: time from EMT/VN notification to start of simulation, total length of simulation, length of TM consultation (total camera time), ambulance transport time (departure from site to ED), NIHSS, and clinical data elements obtained from the VN. Subjective data was gathered from surveys completed by the EMT and VNs, rating the audio and video quality and ease of use of the technology and determining potential impairments to normal work flow from the technology.

Data Analysis

Study feasibility was defined by the ability to conduct and complete 80% of the scenarios without major technological limitations. Eighty percent was chosen as the feasibility threshold, since studies showed that there is around an 18% transmission failure/error rate of pre-hospital EKG transmission.¹⁰ Reliability of video interpretation was defined by a 90% concordance between the data derived from the real time/recorded sessions and the scripted scenarios. The intra-class correlation coefficient along with 95% confidence interval has been estimated for absolute agreement between the raters on the NIHSS, separately for real time and recorded sessions, using two way mixed effects model as per published methods.¹¹ All statistical analyses were conducted using STATA version 13 (StataCorp. 2013. Stata Statistical Software: Release 13, College Station, TX: Stata Corp LP).

RESULTS

In 34 of 40 (85%) scenarios, the teleconsultation was conducted and completed without major technical complication. The absolute agreement for ICC using the two-way mixed-effects model was 0.997 (95% CI 0.97-0.998) for the NIHSS obtained during the real time sessions (Table 1) and 0.993 (95% CI 0.91-0.997) for the recorded sessions. Of note, only 33 recorded scenarios were available for review due to a recording error of the video clip, on one recording. Of the 34 real time assessments, 30 (88%) matched the NIHSS by ±2 points, and 96% of the clinical information obtained by the remote VN matched those provided from the script. Inter-rater agreement (Table 2) using Kappa statistics showed that for live raters, 10/15 items on the NIHSS showed excellent agreement (κ statistic, 0.75–1.0), and 5/15 showed moderate agreement (κ statistic, 0.4–0.75). Recorded raters had less agreement, with 3 items (extinction, visual fields, and facial palsy) showing poor agreement (κ statistic, <0.4). The mean total length of the scenarios was 21 minutes ranging from 16 to 30 minutes and the mean time of the teleconsultation (total camera time) was 10 minutes ranging from five to 12 minutes (Table 3).

Table 1.

Data Analysis

	Real Time Assessments (n = 34)	Recorded Assessments (n=33)
RELIABILITY
Intra-class correlation (95% CI) for NIHSS	0.997 (0.992-0.999)	0.993 (0.975-0.999)
VALIDITY
Matched Scripted NIHSS ± 2 points	88%	70%
Clinical Data points Obtained (12 items)	96%	96%

Table 2.

Inter-rater agreement for NIHSS items

NIHSS Component	n*	Kappa for 4 Live Raters	Kappa Classification	Kappa for 4 Recorded Raters	Kappa Classification
1. LOC	7	0.703	Moderate (0.4 –0.75)	0.46	Moderate (0.4 – 0.75)
2. LOC Questions	7	1.0	Excellent (0.75 – 1)	0.832	Excellent (0.75 – 1)
3. LOC Commands	7	0.84	Excellent (0.75 – 1)	0.467	Moderate (0.4 – 0.75)
4. Best Gaze	7	0.862	Excellent (0.75 – 1)	0.689	Moderate (0.4 – 0.75)
5. Visual Fields	6	0.605	Moderate (0.4 – 0.75)	0.36	Poor (< 0.4)
6. Facial Palsy	6	0.709	Moderate (0.4 – 0.75)	0.37	Poor (< 0.4)
7. Motor Right Arm	6	1.0	Excellent (0.75 – 1)	0.616	Moderate (0.4 – 0.75)
8. Motor Right Leg	6	1.0	Excellent (0.75 – 1)	0.616	Moderate (0.4 – 0.75)
9. Motor Left Arm	5	0.77	Excellent (0.75 – 1)	0.715	Moderate (0.4 – 0.75)
10. Motor Left Leg	6	0.845	Excellent (0.75 – 1)	0.704	Moderate (0.4 – 0.75)
11. Limb Ataxia	6	1.0	Excellent (0.75 – 1)	1.0	Excellent (0.75 – 1)
12. Sensory	6	0.606	Moderate (0.4 – 0.75)	1.0	Excellent (0.75 – 1)
13. Best Language	5	1.0	Excellent (0.75 – 1)	1.0	Excellent (0.75 – 1)
14. Dysarthria	5	1.0	Excellent (0.75 – 1)	0.819	Excellent (0.75 – 1)
15. Extinction	5	0.494	Moderate (0.4 – 0.75)	0.348	Poor (< 0.4)

^*n denotes number of subjects for whom complete data was available i.e. all 4 raters.

Table 3.

Time Metrics

Mean time from Stroke Alert to VN logon1	2 minutes 55 seconds
Mean total tele-consultation time2	9 minutes 46 seconds
Mean total ambulance transport time3	9 minutes 16 seconds
Mean total scenario time4	21 minutes 18 seconds

¹Time from stroke alert page to VN logon to RP-Xpress

²Total time VN on camera performing stroke evaluation

³Total time for patient transportation from scene departure to arrival to ED

⁴Total time from stroke alert page to arrival to ED.

The EMT personnel commented that the telemedicine technology and remote consultation did not delay transportation of the patient. The TM device was subjected to vibrations and road noise of the ambulance during transportation but it was not reported by any of the remote VNs that their ability to conduct assessments was compromised.

Six scenarios were unable to be completed due to technical complications. In three of the six scenarios, failure was most likely due to poor WIFI connection of the 4G-hotspot as indicated by the lack of signal bars on the device, which necessitated multiple reconnections and poor quality of the teleconsultation. This is likely due to poor coverage in that location and potentially could be improved with another service provider that has better coverage in this area. In two early scenarios, the RP-Xpress had an error (automatic software updates) that required rebooting of the device that led to significant delay of the teleconsultation, so that evaluation was not completed prior to arrival of the ambulance to the ED. This error should have been avoided with proper testing of the equipment from the vendor before initiation of the pilot trial. In one scenario, the WIFI hotspot was out of battery as it was being charged with an improper charger and no internet connection was established.

DISCUSSION

Five recent studies have investigated the feasibility and reliability of telemedicine for pre-hospital evaluation of stroke patients, and results thus far have been variable^12-16. Of the five studies, only two were conducted in the United States (US) where physicians are not dispatched with the EMT team such as in some European countries, and in some studies the remote neurological evaluations were not performed real-time but were viewed in a store-forward manner (reviewed recordings).^{13, 14} Liman et al. showed that due to technical failure, they were only able to perform 12 of the planned 30 stroke scenarios.¹⁵ Bergrath et al. in Aachen, Germany reported that real-time audio/video teleconsultation on patients with suspected stroke is feasible but that there were no differences in time metrics between the pre-hospital teleconsultation group versus the traditional EMT group.¹² The Bergrath study, however, did not have vascular neurologists performing the teleconsultation; instead, a physician was part of the dispatched EMT team which differs from EMT protocols in the US. Recently, Van Hooff et al. demonstrated that remote assessment of stroke severity, using the unassisted telestroke scale (UTSS) in Belgium, is both feasible and reliable.^{16, 17}

The results of our pilot feasibility study showed that 85% of the pre-hospital stroke scenarios were completed without any major technical failure. This compared to 40% success from the Liman study¹⁵ which used 3G connectivity and only one technical issue out of 41 stroke scenarios (98% success) in the Van Hooff study¹⁶, which was the first to utilize 4G technology for pre-hospital TM. Our study confirms that improved telemedicine technology and especially use of a robust 4G/LTE cellular network allows for a more stable internet connection. As reported by other studies, we also experienced occasional (one to three per scenario) connection drops during ambulance transport as coverage transitioned between cell towers but reconnection times were quick and did not significantly impact the flow of the consultations. We also demonstrated that remote assessments using mobile telemedicine yielded reliable neurological assessments with strong intra-class correlation in the full NIHSS, >95% concordance of the clinical data points.

Our data show that it may only take an average of 10 minutes to perform the teleconsultation, which is less than half of the time from dispatch to ED arrival. This key time point has not been reported in prior studies but is of critical importance. A potential hurdle for the use of pre-hospital teleconsultation is the lack of time to perform the consultation prior to ED arrival. In the Van Hooff study, they reported a mean examination time of the UTSS of 3.1 minutes¹⁶, which involved only a neurological assessment. In order to reduce in-hospital stroke evaluation and decrease decision time, clinical information such as time of onset and other potential t-PA /inclusion/exclusion criteria are needed in addition to the neurological assessment. We demonstrated that experienced VNs were able to remotely perform a high yield pre-hospital neurological assessment, including the full NIHSS exam with assistance from EMT personnel, and that remote evaluation by TM might decrease in-hospital stroke evaluation by roughly 10 minutes, which could potentially impact treatment times and clinical outcomes. Pre-hospital TM evaluation coupled with pre-hospital registration of patients and notification of the receiving facilities has the potential to further reduce treatment times and false activation of local stroke teams. Although the UTSS can be performed without assistance from a physician, we chose to use the NIHSS because of its acceptance as well-known tool for acute stroke assessment. Importantly, nearly all AIS trials utilize the NIHSS as the standard deficit scale, which would be useful to extend to the pre-hospital setting if pre-hospital stroke trials are to be conducted. It is possible, however, that as the UTSS may become more widely adopted in the pre-hospital setting, and potentially lead to further reductions in time metrics.

All of the EMT personnel commented that the teleconsultations did not interfere with their assessments and protocols and did not cause any delay in transportation of the patient, however, the personnel did not obtain actual vital functions or deliver any other type of care in these simulations. Our study relied heavily on the assistance of EMT personnel during the NIHSS assessment. , While the EMT personnel were inexperienced in performing the NIHSS, the VN was able to conduct the NIHSS remotely by simply instructing the EMT to help with certain aspects of the NIHSS assessment. Furthermore, we compared time metrics by distance to ED and NIHSS severity and found no significant interaction difference. However, the inexperience with the NIHSS by the EMT may have led to inaccuracies in the NIHSS obtained. It would be optimal in future studies to fully certify EMTs in the NIHSS or substitute the UTSS for the NIHSS. In real patient situations, it might be difficult to involve EMT to this degree and maintain their motivation and cooperation; however, our results suggest the need for EMTs to become better trained on certain components of the exam, such as visual fields, sensory, and extinction. Our results also showed that the agreement in NIHSS was poorer for the recorded raters. The discrepancies in agreement may be because the recorded raters were not conducting the NIHSS themselves with the EMT and lacked the ability to repeat if unsure of findings during parts of the exam. The experience in stroke care was similar in the two groups, and the visual and audio quality were comparable in the live and recorded sessions. While our approach differs from the Van Hooff study utilizing the UTSS which does not require an experienced professional at the bedside to conduct assessment of stroke severity,¹⁶ we feel that by engaging EMT in the pre-hospital evaluation of the stroke patient, they may grow to accept this practice as a new standard. In our community, EMT personnel greatly embrace involvement in the care of the stroke patient after delivery to our center. Pre-hospital evaluation may be another way to engage interest of EMT personnel in acute stroke treatment.

There are other issues to consider when working with EMT to apply TM in the pre-hospital setting. We noted that during the assessments in transport, the belt buckles for the legs had to be loosened in order to test leg strength and ataxia. As a result, the NIHSS assessment required a deviation for the standard EMS transport protocol in which patients are to be secured for the entire duration of transport. This issue would need to be clarified before testing in live patients for safety concerns, and potential modifications to the protocol may be needed, such as testing for leg strength and ataxia prior to departing the scene for the hospital while the ambulance is still stationary. In addition, on multiple occasions, the TM device had to be repositioned in order for the remote VN to have the best view of the patient during assessments, again, highlighting the crucial involvement of the EMT personnel in our study. In some scenarios, head positions of the actors were upright and some were laid flat on the stretcher. Head positioning of the actors was not predetermined and was left up to the discretion of the EMS personnel, recognizing that patients with suspected strokes should not be transported in the upright position. During some sessions, verbal communication had to be repeated multiple times by the remote VN to the EMT, but it still allowed for adequate communication. At our institution, we perform TM in multiple EDs in our region and it is not infrequent that we need to repeat our questions to the patients. This appears to be a common phenomenon but does not impact the care of the patient. Lastly, while the EMT personnel orally communicated vital signs in our study, which is standard of care, we look forward to direct digital communication from devices would likely be more advantageous in pre-hospital stroke care.

There are several limitations of our study. We relied on the use of healthy actors who are medical professionals, simulating stroke scenarios in a controlled environment which did not completely represent acute emergent situations. The virtual setting of our trial may have led to artificially low tele-consultation times which may be longer in real patient situations where patients often do not remember which medications they take or other pertinent past medical history that may be critical to the assessment. We also conducted the scenarios during the day time which likely decreases the VN response times. It is possible that at night, there might be a delay in response by the VN hence limiting the ability to fully assess the patient in the pre-hospital setting. With the repetitive nature of the scenarios, it is possible that times and accuracy of the NIHSS assessment were enhanced due to a learning effect of the actors and EMT personnel. Furthermore, it is difficult to assess accurately in this artificial setting the ability of the EMT personnel to deliver standard or emergency medical care especially with their additional responsibility to interact with the remote VN. Lastly, our pilot trial was conducted using only one telemedicine device from one vendor and one wireless carrier; our results may have varied if other telemedicine devices or wireless carriers were used and wireless coverage may not be as strong in other regions.

In conclusion, remote pre-hospital evaluation of actors simulating AIS patients in a controlled setting using mobile TM technology with 4G/LTE capability is feasible and reliable. Current TM and cellular wireless technology is sufficient to support pre-hospital evaluation for potential stroke patients. Pre-hospital evaluation could potentially be used in the future to triage stroke patients to appropriate facilities that match the level of stroke severity; for example, patients with severe strokes may benefit from comprehensive stroke center level of care as opposed to patients with minor stroke who may only need primary stroke center level of care¹⁸. Another potential application for this technology is the ability to conduct pre-hospital clinical trials and orchestrate pre-hospital treatment in ambulances equipped with appropriate diagnostic technology.^{19, 20} Further studies testing the technical feasibility, reliability, and clinical usefulness in a real world patient setting are warranted.