Human Bedside Evaluation versus Automatic Responsiveness Testing in Epilepsy (ARTiE)

Abstract

Evaluation of behavioral impairment during epileptic seizures is critical for medical decision-making, including accurate diagnosis, recommendations for driving and presurgical evaluation. We investigated the quality of behavioral testing during inpatient video-EEG monitoring at an established epilepsy center, and introduce a technical innovation that may improve clinical care. We retrospectively reviewed video-EEG data from 152 seizures in 33 adult or pediatric patients admitted for video-EEG monitoring. Behavioral testing with questions or commands was performed in only 50% of seizures ictally, 73% of seizures postictally, and 80% with either ictal or postical testing combined. Further, the questions or commands were highly inconsistent and were performed by non-medical personnel in about a quarter of cases. In an effort to improve this situation we developed and here introduce Automatic Responsiveness Testing in Epilepsy (ARTiE), a series of video-recorded behavioral tasks automatically triggered to play in the patient’s room by computerized seizure detection. In initial technical testing using pre-recorded or live video-EEG data we found that ARTiE is initiated reliably by automatic seizure detection. With additional clinical testing we hope that ARTiE will succeed in providing comprehensive and reliable behavioral evaluation during seizures for people with epilepsy to greatly improve their clinical care.

Introduction

Behavioral impairment during epileptic seizures seriously impacts patients’ quality of life. Loss of consciousness as well as motor, visual, and cognitive deficits can cause injury, car accidents, and social stigmatization^1–3. Accurate evaluation of behavior during and following seizures is crucial for clinical decisions about driving privileges, to diagnose epileptic verses non-epileptic seizures, and to help determine seizure severity as well as localization for epilepsy surgery. Behavioral evaluation in outpatients typically relies on report by patients or family, which can be unreliable^{4; 5}. Improved information can be obtained with inpatient video-EEG monitoring, but still depends on the training and bedside-availability of hospital personnel or family members. An important, unknown question is—how reliable is behavioral testing in a typical inpatient video-EEG monitoring unit and can this testing be improved?

We retrospectively reviewed behavioral testing performed on our video-EEG monitoring unit for adult and pediatric patients over a four to five month period and found that testing was only performed at all during about half of seizures, and even in these seizures the content of testing was inconsistent. To begin to improve this situation, we built on our prior experience with standardized prospective behavioral testing^6–9 to develop a system providing fully Automatic Responsiveness Testing in Epilepsy (ARTiE).

Methods

Patients

We reviewed video-EEG recordings for all seizures from inpatients monitored at Yale-New Haven Hospital during 4 months (adult patients) or 5 months (pediatric patients) beginning October 2013. Patients with non-epileptic or subclinical seizures were excluded, as were days in which patients experienced more than 10 seizures, because consistent behavioral testing is not always expected under these conditions. For pilot technical testing of ARTiE we also used adult and pediatric epilepsy inpatients. All procedures were approved by the Yale University Human Investigations Committee and informed consent was obtained from each subject.

Behavioral Testing and Review

All nurses working at the Yale Inpatient Video-EEG Monitoring Unit undergo 11–12 weeks of comprehensive education and orientation for inpatient care of neurology patients, including specific instructions on how to evaluate epilepsy patients during seizures based on a standard 1-page protocol that has been established at our center for over 15 years. As a reminder, this protocol is also posted in the rooms of all patients undergoing Video-EEG monitoring. Prior to beginning work on the floor, all nurses are observed by more experienced staff as the new personnel perform behavioral evaluations on at least 1–2 patients during seizures and feedback is provided to ensure the evaluations are performed correctly.

Video-EEG recordings were reviewed by three reviewers (EK, AP, EJ) for all seizures previously identified by the clinical team. The reviewers first viewed several seizures together to ensure that they were using the same criteria. Analysis of approximately one third of the seizures was then conducted independently by each reviewer. For any seizure in which results were ambiguous, the reviewers then met again to determine final results by consensus. Behavioral testing was considered to have occurred if there were any questions or commands directed towards the patient ictally or postictally (within 5 minutes of seizure offset). Video-EEG review was also used to determine who interacted with the patient (medical personnel or visitor), time of day, patient waking state, seizure type (focal or generalized), whether or not a pushbutton alert occurred, and the specific questions or commands given to the patient. Seizure onset times were defined by the onset EEG or behavioral changes (whichever was first); and offset times by the end of EEG or behavioral changes (whichever was last). Results comparing frequency of behavioral testing under different conditions were analyzed by chi-squared test with significance level p < 0.05.

ARTiE

ARTiE is a multi-device behavioral testing system designed for the clinical environment of the epilepsy monitoring unit (Figure 1). The two main components are: 1) the ARTiE Controller, consisting of software installed on the video-EEG collection station in the Monitoring Room; and 2) the ARTie Tester, consisting of an all-in-one PC with video display software in the Patient Room.

Figure 1. Main components and configuration of ARTiE in the video-EEG monitoring unit.

During video-EEG recordings, detection of a seizure by Persyst triggers the ARTiE Controller software installed on the video-EEG collection computer in the monitoring room. A signal is sent via Ethernet to the ARTiE Tester in the patient room which displays the behavioral testing video. The patient’s responses are recorded by the video-EEG system and analyzed offline during clinical review.

The ARTiE Controller is a Windows Form application written in Visual Basic (Microsoft) that hosts a secondary copy of Persyst 12’s MagicMarker Bedside (Persyst) for EEG seizure detection. Installed in the hospital’s monitoring room (Figure 1), the program reads real-time COM seizure events generated by Persyst’s trends and connects over the hospital’s Ethernet network to the ARTiE Tester.

The ARTiE Tester consists of a multithreaded Python (Python Software Foundation) application running on an All-in-One PC in the patient’s room (Figure 1). The main thread handles ARTiE video playback via PsychoPy (Psychology software in Python), while the background thread listens for network communications from the ARTie Controller and signals the main thread to start or interrupt the video.

The video behavioral testing sequence for ARTiE was modified slightly from the most recent version of the Responsiveness in Epilepsy Scale (RES)^6–9. Based on prior work with this scale the interval between onset of consecutive items was set at approximately 9 seconds. The RES-ARTiE testing sequence is shown in Supplementary Data S1 along with suggested scoring (although scoring was not done for the present work, aimed only at pilot technical testing). A video example of the testing sequence is shown in Supplementary Video S2. Total duration of the video behavioral test is 7 min and 5 s which should cover both the ictal and postictal periods for typical seizure durations. As in prior versions of RES^6–9, to provide a unique memory item for each seizure the patient is asked to remember the exact time (hour: minute); this time is supplied automatically by ARTiE. The program can be interrupted manually for false alarms, and can be started manually to allow baseline testing in non-seizure (interictal) periods.

ARTiE underwent technical testing with two approaches. Before introduction to the video-EEG monitoring unit we tested ARTiE in an isolated local network using pre-recorded video-EEG data from 4 patients. Next we installed ARTiE in the epilepsy monitoring unit for testing during live recordings of video-EEG from two patients (one adult and one pediatric). For both the pre-recorded and live data we reviewed the number of Persyst seizure detection events that did or did not successfully initiate ARTiE using data logging incorporated into the ARTiE software.

Results

Conventional behavioral testing

Video-EEG data were reviewed from a total of 152 seizures in 33 patients (14 male and 19 female; mean age 25.6 years), 3 patients during intracranial EEG and the remaining 30 during scalp EEG. Mean seizure duration was 105 ± 218 s (mean ± SD).

Behavioral testing was performed in only 50% of seizures ictally, 73% postictally, and 80% either ictally or postictally (Table 1). On average, behavioral testing began with a delay of 69 ± 64 s (mean ± SD) after seizure onset. For seizures in which testing was done, this was performed by medical personnel in 76% of seizures; family or other visitors did the testing in the remaining seizures. Testing was done significantly more often when seizures occurred while patients were awake at onset, during the daytime, or when the event button was pushed, but was no different in focal verses generalized seizures (Table 1). Even allowing much more lenient criteria for behavioral testing by including any verbal or non-verbal stimulus that would normally elicit a response (for example shaking patient’s shoulder or talking loudly to them) only led to testing in an additional 13% of seizures in the ictal period and 14% postictally.

Table 1.

Frequency of Behavioral Testing with Questions or Commands on the Epilepsy Monitoring Unit

	Ictal Testing Frequency1	Statistics	Postictal Testing Frequency1	Statistics	Combined Ictal or Postical Testing1	Statistics
Overall	76/152 (50%)		109/149 (73%)2		122/152 (80%)
Awake at Onset	58/96 (60%)	X2=11.31	78/94 (83%)2	X2=12.515	88/96 (92%)	X2=21.390
Asleep at Onset	18/56 (32%)	p < 0.001	31/55 (56%)2	p < 0.001	34/56 (61%)	p<0.0001
Daytime Onset (7 AM – 11PM)	53/93 (56%)	X2=4.682	67/92 (73%)2	X2=0.013	76/93 (82%)	X2=0.321
Nighttime Onset (11 PM – 7AM)	23/59 (39%)	p = 0.03	42/57 (74%)2	p = 0.91	46/59 (78%)	p=0.5709
Focal Seizure	65/126 (52%)	X2=0.742	99/126 (79%)	X2=1.331	99/126 (79%)	X2=1.331
Generalized3 Seizure	11/26 (42%)	p =0.39	23/26 (88%)	p = 0.40	23/26 (88%)	p= 0.2487
Pushbutton Pressed4	68/114 (60%)	X2=16.99	95/111 (86%)2	X2=34.248	107/114 (94%)	X2=53.214
Pushbutton not Pressed	8/38 (21%)	p<0.001	14/38 (37%)	p < 0.001	15/38 (39%)	p<0.0001

¹Number of seizures tested / total number of seizures

²Number of seizures analyzed were reduced slightly in these cases (by at most 3 seizures) due to inadequate postictal video data for analysis.

³Generalized seizures were all focal onset with secondary generalization. No primary generalized seizures occurred in the study sample.

⁴Bedside event pushbutton was pressed by patient, family or staff to indicate seizure occurrence.

Questions or commands given during seizures were highly inconsistent. The most common questions ("Are you OK?” or “How are you feeling?") constituted only 50% of seizure questions ictally, and 56% postictally. The next most common questions were asked even less often, (24% ictally and 36% postictally for “What’s your name?”; and 18% ictally, 44% postically for “Where are you?”), and the majority of other questions were asked for only a few (< 14% each) of seizures.

ARTiE

In technical testing using pre-recorded files ARTiE processed 46 hours of video-EEG data from 4 patients. The program handled 86 seizure pushbutton events and 14 Persyst seizure detection events, successfully triggering the behavioral testing videos in each instance without any false negatives or false positives.

In testing during active video-EEG recordings, ARTiE processed 67 hours of EEG data from 2 patients, with a total of 44 seizure detection events. 13 occurred within 7 minutes of a previous seizure, and successfully did not re-initiate ARTiE, as it was already playing. 100% of the remaining seizures correctly initiated ARTiE. In addition, ARTiE was never initiated at a time when Persyst did not detect a seizure.

Discussion

This study demonstrates that behavioral testing during and after seizures by hospital personnel, even in the setting of a comprehensive inpatient epilepsy program, is done infrequently and inconsistently. Only 50% of seizures occurring during the study period had behavioral testing. Further, only three quarters of seizures were tested by medical personnel, and behavioral testing was not standardized, evidenced by the fact that the most commonly asked question was only used in half of cases in which patients were tested. Behavioral impairment accounts for many of the hardships faced by epilepsy patients, including injury, car accidents, and memory loss^{1; 2}, and is an integral part of seizure localization and surgical decision-making. Currently, much valuable information for clinical decision-making is not being gathered despite admission for specialized epilepsy care. Although these findings only reflect the quality of behavioral testing at our center and may vary substantially at other epilepsy programs, the fact that we do invest substantial effort to ensure that standard behavioral testing protocols are in place raises the possibility that testing quality at least at some centers might be even worse than reported here.

ARTiE represents an opportunity to potentially improve consistent and standardized testing of patients with epilepsy. Feasibility testing with pre-recorded and live video-EEG data shows that Persyst seizure detection reliably initiates ARTiE, and that ARTiE is not falsely initiated without Persyst seizure detection. ARTiE did not interfere with the normal operation of the clinical video-EEG monitoring system, and also did not restart in the event of additional seizures occurring while it was running, important for practical applicability. In addition, ARTiE can be manually stopped in case of false detections, limiting any disruption to the patient or staff, and can be initiated manually to obtain baseline interictal testing.

To our knowledge, no other system exists that can replicate this functionality. Previous attempts at standardized responsiveness testing batteries relied on the presence of specially trained personnel, making them infeasible for consistent round-the-clock use. In addition, despite extensive training, there were still inconsistencies in testing⁶. ARTiE is available at all times of the day, does not require additional personnel, and is completely standardized. This approach could potentially greatly enhance the clinical knowledge gained when patients are admitted for video-EEG monitoring. In addition, by maximizing the information gathered, ARTiE could enhance the speed of reliable diagnosis and reduce patients’ length of stay.

The current study provides the clinical motivation for ARTiE and initial technical feasibility testing, however it does not yet fully test the system in a series of patients, a crucial next step. This further testing will be essential to confirm the usefulness, sensitivity and specificity of ARTiE. By relying on Persyst seizure detection algorithms, ARTiE is limited by the real-time sensitivity and specificity of Persyst which may require additional customization to be fully optimized. At present ARTiE has not yet been integrated with seizure detection software other than Persyst, and future work implementing ARTiE in other popular seizure detection systems would be needed. In addition ARTiE is currently designed only for inpatient video-EEG monitoring, although in principle a similar system could eventually be developed for use in the outpatient setting.

In conclusion, we have demonstrated that behavioral testing occurs inconsistently and in a non-standardized manner in the setting of continuous video-EEG epilepsy monitoring. We have shown that ARTiE is reliably initiated by Persyst seizure detection software and represents a feasible solution to increase the rate and consistency of behavioral testing in this patient population. This report represents a preliminary introduction of the system. We now hope to collect further data on the success of clinical and electrographic seizure detection with ARTiE, and on the response of patients and staff to ARTiE in the epilepsy monitoring unit.

Supplementary Material

Supp Material Data S1. Supplementary Data S1. Responsiveness in Epilepsy Scale used for Automatic Responsiveness Testing in Epilepsy (RES-ARTiE).

Testing sequence is modified slightly from previous versions of the Responsiveness in Epilepsy Scale (RES) used for prospective standardized testing of patients in the epilepsy monitoring unit.

Supp Material Video S1. Supplementary Video S2. Example of RES-ARTiE testing sequence.

Video demonstrates the sequence of test items presented by the ARTiE Tester upon detection of a seizure. In actual use, the time given at the beginning is the real-world time at onset of testing (arbitrary time given here for demonstration purposes). MPEG video codec is H.264 (MPEG-4 AVC).