Seizure Detection by Critical Care Providers using Amplitude-Integrated EEG and Color Density Spectral Array in Pediatric Cardiac Arrest Patients

Abstract

Objective

Determine the accuracy and confidence of CCM providers to identify seizures using amplitude-integrated EEG (aEEG) versus aEEG combined with Color Density Spectral Array electroencephalography (aEEG+CDSA)

Design

tutorial and questionnaire.

Subjects

Pediatric critical care providers (attendings, fellows, nurses).

Interventions

A standardized powerpoint tutorial on aEEG and CDSA followed by classification of 100 aEEG images and 100 aEEG combined with CDSA as displaying seizures or not displaying seizures.

Measurements and Main Results

EEG tracings were obtained from children monitored with continuous EEG after cardiac arrest. The gold standard for seizure identification was continuous EEG interpretation by a pediatric electroencephalographer. The same EEG tracings were used to generate images containing only aEEG or aEEG+CDSA. Twenty-three CCM providers underwent a 30-minute tutorial on aEEG and CDSA interpretation. They were then asked to determine if there were seizures on 100 aEEG images and 100 aEEG+CDSA. aEEG seizure detection sensitivity was 77% (95%CI: 73%–80%), specificity of 65% (95%CI: 62%–67%), negative predictive value (NPV) of 88% (95%CI: 86%–90%) and positive predictive value (PPV) of 46% (95%CI: 43%–49%). For aEEG+CDSA, sensitivity was 77% (95%CI: 74%–81%), specificity of 68% (95%CI: 66%–71%), NPV of 89% (95%CI: 87%–90%) and PPV of 49% (95%CI: 46%–52%). Sensitivity for status epilepticus detection was 77% (95%CI: 71%–82%) with aEEG and 75% (95%CI: 69%–81%) with aEEG+CDSA. The addition of CDSA to aEEG did not improve seizure detection. However, 87% of CCM providers qualitatively felt that combining both modalities increased their ability to detect seizures.

Conclusions

aEEG and aEEG+CDSA offer reasonable sensitivity and NPV for seizure detection by CCM providers. aEEG+CDSA did not improve seizure detection over aEEG alone, although CCM providers felt more confident using both tools combined. aEEG and CDSA require further evaluation as a tool for screening for seizures and should only be used in conjunction with professional cEEG review.

Introduction

Electrographic seizures are common in critically ill children (1–6) and are associated with worse clinical outcomes (7–12). Electrographic seizures occur in 47% of children after cardiac arrest (3). Continuous electroencephalographic monitoring (cEEG) is the current gold-standard for seizure diagnosis (13), but it requires a high degree of expertise and is time consuming. The majority of Intensive Care Units (ICUs) in the United States and Canada do not have electroencephalographers or EEG technologists available to continually interpret cEEG. Further, when available, cEEG is often only screened for seizures twice daily, which can lead to delays in seizure identification(14).

Frequent screening of cEEG to detect seizures by critical care medicine (CCM) providers in a timely fashion could lead to earlier seizure detection and treatment. To develop a program with bedside CCM clinician quantitative EEG (qEEG) interpretation, it is important to determine the optimal qEEG panels to present with the goal of seizure detection accuracy and CCM provider confidence. We previously determined the accuracy of compressed density spectral array (CDSA) interpreted by CCM providers for seizure detection as having a 70% sensitivity and 86% negative predictive value (NPV) after a brief 30 minute training session (15). Because amplitude-integrated EEG (aEEG) has been more extensively utilized with neonates and has reported seizure detection rates by electroencephalographers and neonatologists of 22–80%, we aimed to compare seizure detection rates of CCM providers using aEEG and the combination of aEEG and CDSA (aEEG+CDSA) to gold standard of electroencephalographer interpretation of full montage EEG in children following cardiac arrest(16–23).

aEEG and CDSA are qEEG techniques that allow more facile review of large volumes of data and might be interpretable by non-electroencephalographers. aEEG is a processed, filtered and time-compressed electroencephalogram that presents amplitude (y-axis) over time (x-axis). It displays peak-to-peak amplitude values of filtered and rectified EEG. CDSA uses Fourier transformation to present EEG power (amplitude²/Hz) (color) and frequency (y-axis) over time (x-axis). They both have the advantage of displaying several hours of compressed EEG data as a single image (15, 19, 24)(Figure 1).

Fig. 1.

(A) Amplitude-integrated EEG (aEEG) and (B) Color Density Spectral Array (CDSA) image (B) for two hours.

(A): aEEG: The red tracing represents the right hemisphere, the blue tracing represents the left hemisphere, and the pink tracing represents both hemispheres. The y-axis represents amplitude (0–100uV). The arrows correspond to seizures. Seizures start with a sharp ramp-up in amplitude tracing (upper and lower margins) and are followed by a rapid decline in amplitude tracing.

(B): CDSA: The top panel displays the left hemisphere and the lower panel displays the right hemisphere. The y-axis represents frequency (0–20 Hz). Power (amplitude²/Hz) is displayed as color. Red and white represent high power, green is moderate power and blue represents low power. The arrows correspond to seizures. Background is low power (blue) in all frequencies. Seizures start with an increase in power in the low frequency range (green) followed by an increase in power in both high and low frequencies (red and white). After the seizure, the power decreases back to baseline.

If CCM providers can reliably detect or exclude electrographic seizures with these qEEG transformations when access to cEEG interpretation by electroencephalographers and EEG technologists is limited, future patient care may be impacted. We aimed to determine the sensitivity of CCM providers at detecting seizures using aEEG or aEEG+CDSA, both as a group as well as CCM provider role (attending, fellow, nurse). We hypothesized that 1) aEEG interpreted by CCM providers would have a sensitivity >80% for seizure detection, 2) CCM providers would have a higher sensitivity and NPV using aEEG+CDSA than aEEG alone and 3) attendings, fellows and nurses would have similar seizure detection rates with these tools. Because status epilepticus post cardiac arrest has been associated with worse outcomes, we did a post hoc evaluation of CCM providers’ sensitivity for detecting status epilepticus.

Materials and Methods

Parents/guardians consent to collect clinical data was obtained as part of an ongoing ICU EEG monitoring study. Collection of post cardiac arrest cEEG tracings was approved by The Children’s Hospital of Philadelphia (CHOP) Institutional Review Board. Evaluation of CCM providers’ ability to review aEEG and CDSA images was exempt from review.

cEEG Acquisition, aEEG and CDSA Image creation

cEEG monitoring was performed as part of standard clinical care in critically ill children after cardiac arrest to identify electrographic seizures, consistent with recent guidelines and consensus statements (12, 13). One hundred cEEG tracings were obtained from 39 children admitted to our pediatric intensive care unit post-cardiac arrest. Patients were monitored for about 72 hours. Multiple two hour tracings could be made from the same patient. A pediatric electroencephalographer categorized these images as having no seizures, seizures or status epilepticus (15). Seizures were defined as abnormal paroxysmal events lasting more than 10 seconds or shorter if associated with a clinical change, with a temporal-spatial evolution in morphology, amplitude and frequency. Status epilepticus was defined as either a single prolonged seizure lasting ≥ 30 minutes or recurrent seizures totaling ≥ 30 minutes within a one-hour epoch (7).

Methods for cEEG tracings acquisition and conversion to aEEG and CDSA tracings are described in our group’s previous published work (15). The CDSA tracings and corresponding aEEG tracings were the same as those used in our previous study (15). Twenty eight percent of tracings displayed seizures. This percentage is similar to the seizure prevalence in children post-cardiac arrest (3, 15).

Participants

Participants included pediatric CCM attending and fellow physicians and nurses working in the pediatric intensive care unit. We recruited participants by email, and they received a gift card following the completion of participation. We collected demographic data including role in the pediatric intensive care unit, previous knowledge of aEEG, CDSA and cEEG interpretation, and years of experience in critical care.

We presented each participant with a 30 minute standardized slide tutorial by a research assistant. Our group had developed this tutorial for a previous study on seizure detection using CDSA, and it was modified for this study with the addition of aEEG training components (15). This tutorial presented: 1) basics of cEEG, 2) basics of aEEG and CDSA interpretation 3) seizure detection examples, and 4) difficult pattern examples including artifacts such as movement artifact. The participants were allowed to ask questions and review the tutorial for as long as needed.

Following the tutorial, we presented participants with a questionnaire that required the classification of aEEG and aEEG+CDSA image as having seizure(s) or no seizure. Each participant reviewed 200 images. These included 100 images of aEEG and the same 100 images of aEEG+CDSA obtained from identical portions of cEEG. Each aEEG image question was followed by the same aEEG image combined with the matching CDSA image. Participants were asked to determine whether seizures were present or absent on each image. Participants were not asked to distinguish individual seizures or determine if status epilepticus was present.

Participants did not have access to cEEG tracings. The images were displayed and answers collected using survey monkey (http://www.surveymonkey.com). Participants were not allowed to ask questions during the survey completion. At the end of the survey, we asked participants about their level of confidence in interpreting aEEG and aEEG+CDSA images and whether they believed combining both tools increased their ability to detect seizures.

Statistics

Our primary aim was to evaluate the ability of CCM providers to identify seizures using either aEEG or aEEG+CDSA. Sensitivity, specificity, positive predictive value (PPV) and NPV were measured and presented as percentages (95% confidence intervals). We have calculated the inter-rater reliability of attendings, fellows and nurses for aEEG and aEEG+CDSA. The reliability of agreement among participants (attending, fellows, and nurses) was examined using Fleiss Kappa. The raters package in R was used to produce Kappa statistics, SD and 95% confidence intervals.(25)

Post hoc, we determined the sensitivity for status epilepticus detection by measuring the frequency with which participants identified seizures on a cEEG tracing with status epilepticus. Test characteristics were calculated for each participant and by group (i.e., attending physicians, fellows and nurses). Statistical analyses were performed using SAS 9.3 (SAS Cary, NC).

Results

CCM providers included 6 attending physicians, 12 fellows and 5 nurses. One participant was both a neurologist and critical care physician, and he had prior cEEG training. Two other participants had prior CDSA training, none had previous aEEG training. Seventy percent of participants had less than 5 years of critical care experience.

aEEG

CCM providers had an overall sensitivity of 77% (95%CI: 74%–80%), specificity of 65% (95%CI: 63%–67%), NPV of 88% (95%CI: 86%–90%) and PPV of 46% (95%CI: 43%–49%) at detecting seizures using aEEG (Table 1). The false-positive and false-negative rates were 35% and 23% respectively. Table 2 presents results by group (attendings, fellows, nurses). There was no difference between provider groups using aEEG. Post-hoc sensitivity for status epilepticus detection was 77 % (95%CI: 71%–82%).

Table 1.

Sensitivity, Specificity, Negative Predictive Value and Positive Predictive Value of PCC Providers at Detecting Seizures Using aEEG and aEEG+CDSA.

	aEEG		aEEG+CDSA
200 questions (n= 2300)	Seizure on cEEG	No Seizure on cEEG	Seizure on cEEG	No Seizure on cEEG
Seizure on	495	579	502	530
aEEG/aEEG+CDSA
No Seizure on	150	1076	141	1126
aEEG/aEEG+CDSA
Measure	Value	Exact 95%CI	Value	Exact 95%CI
Sensitivity	77%	75–80%	78%	74–81%
Specificity	65%	63–67%	68%	66–71%
Negative predictive value	88%	86–90%	89%	87–91%
Positive predictive value	46%	43–49%	49%	46–52%

aEEG; amplitude-integrated EEG, CDSA; color density spectral array, cEEG; conventional electroencephalogram

Table 2.

Sensitivity, Specificity, Negative Predictive Value, Positive Predictive Value and Inter-rater reliability (Kappa) of PCC Providers at Detecting Seizures Using aEEG and aEEG+CDSA by group

Measures		aEEG			aEEG+CDSA
		Value	Exact 95% CI	Value	Exact 95% CI
Sensitivity	Attendings	81%	74–87%	81%	74–87%
	Fellows	74%	69–78%	75%	70–80%
	Nurses	81%	74–87%	81%	73–87%
Specificity	Attendings	62%	57–66%	63%	58–68%
	Fellows	66%	62–69%	71%	68–74%
	Nurses	67%	62–72%	69%	63–73%
Negative predictive value	Attendings	89%	85–93%	90%	85–93%
	Fellows	86%	84–89%	88%	85–90%
	Nurses	90%	86–94%	90%	86–93%
Positive predictive value	Attendings	45%	39–51 %	46%	40–52%
	Fellows	45%	41–50%	50%	46–55%
	Nurses	49%	42–56%	50%	43–57%
Inter-rater reliability (Fleiss Kappa)	Attendings Fellows Nurses	0.41 0.40 0.37	0.32–0.50 0.33–0.48 0.28–0.47	0.33 0.40 0.33	0.25–0.42 0.35–0.50 0.23–0.44

aEEG; amplitude-integrated EEG, CDSA; color density spectral array, cEEG; conventional electroencephalogram

aEEG+CDSA

CCM providers had an overall sensitivity of 77% (95%CI: 74%–81%), specificity of 68% (95%CI: 66%–71%), NPV of 89% (95%CI: 87%–90%) and PPV of 49% (95%CI: 46%–52%) at detecting seizures using aEEG+CDSA (Table 1). The false-positive and false-negative rates were 32% and 23% respectively. Table 2 presents results by group. There was no difference between provider groups using aEEG+CDSA. Detection of status epilepticus by CCM provider had a sensitivity of 75% (95%CI: 69%–81%).

There was no difference in seizure detection test characteristics between CCM provider interpretation of aEEG and aEEG+CDSA. Sixty one percent of CCM providers felt confident interpreting aEEG slides alone, 43% CDSA and 83% aEEG+CDSA. Eighty seven percent of CCM providers felt that combining both modalities increased their ability to detect seizures.

Discussion

This study shows that after a brief training, pediatric CCM providers are able to achieve a moderate sensitivity and high NPV for seizure detection using either aEEG or aEEG+CDSA. Sensitivity and NPV using aEEG were 77% and 88% respectively and 78% and 89% using aEEG+CDSA. Of note, despite the lack of difference in accuracy, clinician confidence for seizure detection was higher using both aEEG and CDSA combined compared to either modality alone.

Electrographic seizures are common in critically ill patients with acute encephalopathy and are associated with worse outcomes (1, 2, 4, 6–9, 26). Every hour increase in seizure burden has been associated with neurologic decline (26). Delay in treatment can lead to refractory seizure control and need for increased therapy (10–12). Seizures are often clinically silent and can only be diagnosed by cEEG monitoring (4, 27–29). Unfortunately, access to cEEG monitoring is limited in the majority of North American PICUs. Up to 57% of centers have access to cEEG reviews less than 3 times per day and 21% obtain less than one cEEG written report per day. Twenty one percent of centers do not have 24/7 in house cEEG technicians available and 36% of centers in Canada do not have remote access to cEEG(14).

In order to develop a bedside neuromonitoring program with the goal of CCM providers detecting seizures using qEEG, it is important that providers are both accurate and confident. Quantitative trend analysis devices in the ICU allow bedside providers to have access to continuous, more easily interpretable, real-time monitoring with the goal of detecting seizures more rapidly. These devices display several hours of EEG tracings in a single image. In our current study, sensitivity was encouraging at 77 % for aEEG and 78% for aEEG+CDSA. This means that up to 77% of images with seizures were correctly classified by CCM providers. We previously published a study of CCM providers using CDSA for seizure detection with a sensitivity of 70% (95%CI: 67%–73%), specificity of 68% (95%CI: 67%–70%), NPV of 86% and PPV 46% at detecting seizures using CDSA. (15). Thirty-nine CCM providers were tested using the same 30 minutes CDSA tutorial and the same CDSA slide classification as our current study. The purpose of this current study was to determine if we could improve upon our previous results, with addition of aEEG. In fact, we found a 7% absolute increase in sensitivity of aEEG and aEEG+CDSA compared to CDSA.

Status epilepticus has been associated with worse outcomes in children post cardiac arrest (7, 30,31). Early identification and management could potentially improve outcomes. In our study, physicians were able to identify 77% of images with status epilepticus using aEEG and 75% using aEEG+CDSA. These detection rates did not differ with these different modalities. In our study, physicians were not asked identify images with seizures versus status epilepticus. Identifying either seizures or status epilepticus should both prompt the clinical team to alert the electrophysiologist and ask for a confirmatory read with cEEG. However, there is a building body of evidence that the higher seizure burden of status epilepticus is associated with worse outcomes as well as higher rates of treatment failure. Therefore, identification of status epilepticus, as opposed to isolated seizures may increase clinicians’ sense of urgency and affect time to diagnosis and management. It would be of interest for future studies to look into the detection of status epilepticus as opposed to that of single seizures.

The NPV for seizures was high at 88% and 89% for aEEG and aEEG+CDSA respectively, whereas PPV was low at 46% and 49%, respectively. This means that CCM providers at times falsely identified seizures when there were none, but were not likely to be falsely reassured by a negative read. CCM providers should have access to confirmatory cEEG interpretation from neurophysiology prior to administration of anti-seizure medications, thus a lower sensitivity would not result in treatment of false positives. A high NPV, while dependent on the prevalence of seizures, is more reassuring, because when the CCM providers think there are no seizures, they are often correct, thus decreasing the concern for false negatives. In practice, this could be optimized as CCM bedside providers can screen qEEG for seizures more frequently than standard screening by the neurophysiology team. While there are concerns that seizures may be missed, cEEG interpretation by professional readers should detect these missed seizures. If qEEG is used, it should be as an adjunctive screening tool not as a diagnostic one. cEEG should always be part of patients’ monitoring and CCM providers should always be able to confirm their impression with neurophysiology prior to treatment.

Prior studies have reported similar sensitivity rates using aEEG or CDSA (15,16,32–35). However, the majority of these studies evaluated the ability of neurophysiologists or neurologists to detect seizures as opposed to that of non EEG trained bedside providers. One study that included 5 neuroscience nurses who received a 15 minute training tutorial had a sensitivity for seizure detection on CDSA and aEEG of 87%, but with a specificity of 61.6%(35). Furthermore, a study including 2 neurointensive care nurses who underwent a 5–6 hour training had a sensitivity of 92–99% and specificity of 89–90% for seizure detection on aEEG and CDSA used together (36). While our study has a lower sensitivity for seizure detection, it remains unclear as to whether or not duration of training impacts accuracy. There are several reasons why participants may have had difficulties identifying seizures, specifically 1) difficulty differentiating artifacts from seizures, 2) shorter duration and intensity of training 3) lack of patient clinical factors and 4) not having access to patient EEG interpretation prior to the provided 2 hour images. Despite these challenges, our study confirms that with minimal training, bedside providers are able to detect more than half of seizures. Importantly, CCM nurses, the most available bedside providers, had similar seizure detection rates as CCM fellows and CCM attendings.

The inter-rater agreements in each group were fair with kappas of 0.41 for attendings, 0.40 for fellows and 0.37 for nurses for aEEG and kappas of 0.33 (attendings), 0.40 (fellows) and 0.33 (nurses) for aEEG+CDSA. We believe these results should improve once these tools are introduced into clinical practice and that CCM providers start using them regularly and receive appropriate feedback from electroencephalographers. Increased exposure should in the long-term improve CCM providers’ understanding and confidence when detecting seizures.

There was no difference between CCM provider accuracy using aEEG or aEEG+CDSA; however, our previous study had a lower CCM provider sensitivity for seizure detection with CDSA alone despite using the same images. It is unclear why CDSA would be less sensitive than aEEG or aEEG+CDSA. Our results suggest that aEEG might be overall easier to interpret which could explain why combining aEEG with CDSA appeared to improve seizure detection rates over CDSA alone. aEEG displays seizures as changes in peak-to-peak amplitude (19, 24) whereas CDSA displays seizures as high power through a Fourier transformation of both amplitude and frequency(15). CDSA provides a different dimension to seizures with color displays of power. A study by Stewart et al. compared electroencephalographers’ ability to detect seizures using either aEEG and CDSA and showed no difference; both modalities had sensitivities of 82% and 83% respectively (33). aEEG seizures are displayed as increases in amplitude from baseline whereas with CDSA seizures are displayed as increases in power (changes of color in higher frequency ranges). Despite the lack of difference in accuracy between aEEG and aEEG+CDSA, CCM providers felt more confident using both tools combined. For implementation of a CCM provider interpreted qEEG program this is important because provider confidence may lead to increased frequency of qEEG interpretation and neurophysiology confirmation.

Our study has limitations. First, the number of participants was small which could have affected our ability to detect differences between modalities and between groups. Second, the tutorial was short, only 30 minutes, and was immediately followed by the classification test. Participants did not receive any feedback as they were classifying images. Memory and retention were not measured. In real time, CCM providers would receive regular feedback from electroencephalographers and have access to cEEG. Finally, CCM providers did not have access to the clinical context of the patients from which the images were obtained. Patient age may impact sensitivity and specificity, however, we did not have access to that information. One of the challenges associated with qEEG is differentiating artifacts from true seizures. Knowing the clinical context would certainly have help CCM providers differentiating artifacts from true seizure and increase overall specificity. All these factors could improve CCM providers’ ability to detect seizures as well as improve inter-rater agreement. Developing a training program that provides sufficient teaching, is easily transmittable to large groups of critical care providers, and is not overly time-consuming is key for the successful clinical implementation and usefulness of these monitoring tools. Future directions to improve seizure detection may be a longer more intensive training program with feedback to providers real-time and then refresher training when a patient is monitored with qEEG. Furthermore, training “super-users” such as charge nurses or nurse practitioners could be a more feasible and sustainable approach than training hundreds of nurses, many of whom may only be able to apply their knowledge during very sporadic and limited periods.

Despite various published approaches, the optimal teaching strategy for bedside electrographic seizure detection is not clear.(32,33,35) A 30 minute tutorial seems insufficient and a future direction will be to evaluate whether more intensive training can improve seizure detection rates. aEEG and CDSA are potential seizure detection tools that may help CCM providers detect some seizures they would otherwise have been unaware of because of lack of cEEG accessibility. Based on our data these qEEG modalities should be used in addition to cEEG, not instead of stand neurology interpreted cEEG. Once optimization of education and improved reliability of these modalities has occurred. qEEG would need to be evaluated to see if it might improve bedside provider seizure detection when used in conjunction with professionally interpreted cEEG.

Conclusions

CCM providers are able to detect electrographic seizures with a moderate sensitivity and NPV using either aEEG or aEEG+CDSA. CCM providers felt qualitatively more confident at detecting seizures using aEEG and CDSA together. aEEG and CDSA require further evaluation as a tool for screening for seizures and should only be used in conjunction with professional cEEG review.