Detection of Electrographic Seizures by Critical Care Providers Using Color Density Spectral Array After Cardiac Arrest is Feasible

Abstract

Objective

To determine the accuracy and reliability of electroencephalographic seizure detection by critical care providers using color density spectral array (CDSA) electroencephalography (EEG).

Participants

Critical care providers (attending physicians, fellow trainees and nurses.)

Interventions

A standardized powerpoint CDSA tutorial followed by classification of 200 CDSA images as displaying seizures or not displaying seizures.

Measurements and Main Results

Using conventional EEG recordings obtained from patients who underwent EEG monitoring after cardiac arrest, we created 100 CDSA images, 30% of which displayed seizures. The gold standard for seizure category was electroencephalographer determination from the full montage conventional EEG. Participants did not have access to the conventional EEG tracings. After completing a standardized CDSA tutorial, images were presented to participants in duplicate and in random order. Twenty critical care physicians (12 attendings and 8 fellows) and 19 critical care nurses classified the CDSA images as having any seizure(s) or no seizures. The 39 critical care providers had a CDSA seizure detection sensitivity of 70% [95% CI: 67%, 73%], specificity of 68% [95% CI: 67%, 70%], positive predictive value of 46%, and negative predictive value of 86%. The sensitivity of CDSA detection of status epilepticus was 72% [95% CI: 69%, 74%].

Conclusion

Determining which post-cardiac arrest patients experience electrographic seizures by critical care providers is feasible after a brief training. There is moderate sensitivity for seizure and status epilepticus detection and a high negative predictive value.

Introduction

Acute symptomatic electroencephalographic (EEG) seizures are common in children who experience a cardiac arrest, and are associated with worse short term survival (1). In larger studies of critically ill children with heterogeneous acute encephalopathy etiologies, EEG seizures occur in 10–40% of monitored patients, and there is increasing evidence that high seizure burdens are associated with worse outcomes (2–5). Furthermore, status epilepticus treatment delays are associated with reduced medication efficacy for status epilepticus termination (6, 7). The majority of EEG seizures in critically ill children have no clinical correlate and therefore detection requires EEG monitoring (2, 5, 8–15). Many institutions do not have access to continuous EEG monitoring services. Additionally, even when EEG “monitoring” is performed, data review is generally intermittent, leading to delays between seizure onset and detection (16).

Color density spectral array (CDSA) is a quantitative EEG technique that uses Fourier transformation to present EEG power (amplitude²/Hz, by color) and frequency (y-axis) over time (x-axis). Up to several hours of EEG can be displayed as a single image (Figure 1). Most EEG seizures are characterized by increases in frequency and amplitude compared to the baseline EEG, and these changes are displayed in CDSA as changes in color (power increase) compared to the baseline, or shifts of power into higher frequency ranges (upward arches of color) (Figure 1). Not all seizures involve frequency and power increases and some may be too short to be displayed clearly due to the time compression leading to CDSA false negatives. In contrast, some rhythmic artifacts may produce frequency and power increases, leading to CDSA false positives. Studies of CDSA use have demonstrated moderate sensitivity and specificity for seizure detection but have all evaluated use by electroencephalographers (17, 18). Critical care providers have expertise at screening multiple monitoring modalities. If critical care providers are able to use CDSA accurately, bedside EEG seizure detection could occur more promptly.

Figure 1.

Color Density Spectral Array (CSDA) Image. The top panel represents the left hemisphere and the bottom panel represents the right hemisphere. The x-axis represents 2 hours of time. The y- axis for each panel represents frequencies from 0 to 20 Hz. Power (amplitude²/Hz) is represented on a color scale, with blue representing low power, green moderate power, and red/white high power. The four arrows denote the four seizures. Seizures are clearly distinguished from the baseline (A), which shows high power in the low frequency range (red and white) and little power in the high frequency range (blue). Seizures (B) begin with increase in power in the higher frequency range (green) followed by increase in power in both high and low frequencies (green high, red in low). At seizure termination the power is low and mostly blue again.

We aimed to determine the accuracy of CDSA EEG patterns interpreted by critical care practitioners to detect the presence of full montage EEG confirmed seizures among patients resuscitated after cardiac arrest. We hypothesized that CDSA read by critical care providers (attendings, fellows, and nurses) would be at least 80% sensitive for detection of EEG seizures when compared to the gold standard of electroencephalographer's interpretation of full montage raw EEG tracings. Further, we hypothesized that the negative predictive value would be >80% (i.e., when seizures were not identified by critical care providers on CDSA EEG, seizures were not observed by electroencephalographers on full montage EEG).

Methods

The Children's Hospital of Philadelphia Institutional Review Board approved the collection of EEG tracings. Parents/guardians provided consent for collection of clinical data including EEG tracings as part of an on-going intensive care unit (ICU) EEG Monitoring Study (3). The portion of the study involving critical care provider review of CDSA images was deemed exempt from review.

Continuous EEG (cEEG) Tracings

Continuous EEG tracings were obtained from 39 patients who were successfully resuscitated from a cardiac arrest and then underwent clinically indicated EEG monitoring in the ICU at The Children's Hospital of Philadelphia. Continuous EEG was performed using a Grass-Telefactor (West Warwick, RI) video-EEG system, recorded at a sampling rate of 200 samples/second. Gold-over-silver scalp surface electrodes were positioned according to the international 10–20 system and affixed with collodion adhesive. Full EEG tracings were saved for investigational use. A pediatric electroencephalographer blinded to the CDSA images categorized continuous EEG tracings as having no seizures, seizures, or status epilepticus. EEG seizures were defined as abnormal paroxysmal events that were different from the background, lasted longer than ten seconds (or shorter if associated with a clinical change), and had a temporal-spatial evolution in morphology, frequency, and amplitude with a plausible electrographic field. Two hour epochs were classified as status epilepticus if they contained at least 30 minutes of EEG seizure(s) as a single ≥30 minute seizure or recurrent seizures totaling ≥30 minutes (3).

Color Density Spectral Array Images

The CDSA images were derived from EEG epochs with and without seizures. Conventional EEG was converted to CDSA images using Persyst Insight II Magic Marker (Version 11, Persyst Development Corporation, San Diego, CA). EEG power values were calculated from bipolar montages in the left (F3-C3, C3-P3, P3-O1, F7-T3, T3-T5, T5-O1) and right (F4-C4, C4-P4, P4-O2, F8-T4, T4-T6, T6-O2) hemispheres via fast Fourier transformation with the following parameters: 2 second window, 8 windows/epoch with overlap. Standard images displayed 2 hours of CDSA in frequencies 0 to 20 Hz independently over the right and left hemispheres. (Figure 1) One hundred CDSA images were created with a seizure prevalence of 30% to represent the seizure prevalence in a post cardiac arrest ICU population.

Participants

Invitations to participate in this study were sent by email to critical care attendings, critical care fellows, and critical care nurses. A total of 39 participants (12 attending physicians. 8 fellows, and 19 nurses) were recruited to this study.

Each participant met with the same research assistant who presented them with a 15 minute standardized powerpoint slide tutorial addressing (1) EEG basics, (2) description of CDSA components, (3) interpretation of CDSA, (4) recognition of seizures on CDSA, and (5) recognition of challenging CDSA patterns such as artifacts and burst suppression. The participants had as much time as needed to review the tutorial and ask questions. Participants were then presented with unique questions and answers that had not previously been seen by the participants. Demographic data was obtained for each participant including role in the ICU, years of clinical practice (including critical care training) and previous experience reading EEG or CDSA.

CDSA images were presented in replication so that each participant was presented with a total of 200 images in random order. Participants were instructed to answer “yes” if they thought there was at least one seizure on the image or “no” if they thought there were no seizures on the image. They were not asked to distinguish seizures from status epilepticus or to identify individual seizures. Participants did not have access to the conventional EEG tracings. The images were displayed and answers collected using survey monkey (surveymonkey.com). Participants could not ask questions or receive feedback as they reviewed and scored the images during the testing.

Statistics

The primary aim of this study is to assess the accuracy of seizure detection on CDSA readings compared to the gold standard of expert electroencephalographer detection of seizures on full-array conventional EEG. Measures of diagnostic accuracy including sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), accuracy rates, and the area under (AUC) on the receiver operating characteristic (ROC) curve were calculated for each image-reader combination. Post hoc we determined the sensitivity for status epilepticus detection for each subject by evaluating the frequency with which they were correct in determining a seizure was present when the image contained status epilepticus. Sensitivity, specificity, accuracy were presented as percentage with its 95% confidence intervals for the first replication of 100 CDSA images. PPV, NPV, and AUC were also calculated for the first replication of 100 CDSA images. Measures of diagnostic accuracy were presented for all participants and by participant group (i.e., attending physicians, fellows, nurses).

To examine reproducibility of results, Cohen's Kappa Coefficients for reliability were calculated for estimating inter-rater (for all participants) and intra-rater reliability of CDSA (by each group of participants).

Statistical analyses were performed using SAS 9.3. AUC calculations were performed using MedCalc and SPSS.

Results

Twenty critical care physicians (12 attendings and 8 fellows) and 19 critical care nurses completed the tutorial and CDSA image classification. The median [minimum, maximum] years of critical care experience were 11 [3, 30] for critical care attendings, 2 [1, 3] for critical care fellows, and 11 [3, 30] for critical care nurses. No participant had received previous CDSA training and only one critical care attending had previous EEG training.

All Critical Care Providers

When grouped together, the 39 critical care providers (attending physicians, fellows and nurses) had a CDSA seizure detection sensitivity of 70% [95% CI: 67%, 73%], specificity of 68% [95% CI: 67%, 70%], PPV of 46% and NPV of 86%. (Table 1). The Cohen's Kappa was 63% [95% CI: 61%, 66%] for all images and 69% [64%, 72%] for images with seizures. There was no difference between critical care attendings AUC 0.68 and critical care nurses AUC 0.66, but critical care fellows were better than both groups AUC 0.73 (p<0.001). Sensitivity, specificity, accuracy, PPV and NPV are presented by critical care provider role on Table 2.

Table 1.

Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) and Receiver Operative Curve (ROC) for any seizure detection for critical care nurses, fellows and attendings based on the first 100 questions.

First 100 Questions n=3900	No Seizure on EEG	Seizure on EEG	Total
No seizure on CDSA	1923	327	2250
Seizure on CDSA	885	765	1650
Total	2808	1092

	First 100 Questions
Measure	Value	Exact 95% CI
Sensitivity	0.70	(0.67, 0.73)
Specificity	0.68	(0.67, 0.70)
Accuracy	0.69	(0.67, 0.70)
ROC curve (AUC)	0.69	(0.68, 0.71)
PPV	0.46
NPV	0.86

CI: Confidence Interval, AUC: Area Under the Curve

Table 2.

Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) and Receiver Operative Curve (ROC) for any seizure detection by critical care provider role based on the first 100 questions. Contingency tables are presented by specialty for the first 100 questions.

Measure	Participants	Value	Exact 95% CI
Sensitivity	Attending	0.72	(0.70, 0.80)
	Fellow	0.78	(0.72, 0.82)
	Nurse	0.64	(0.59, 0.68)
Specificity	Attending	0.69	(0.66, 0.72)
	Fellow	0.68	(0.64, 0.72)
	Nurse	0.68	(0.66, 0.71)
Accuracy	Attending	0.71	(0.68, 0.74)
	Fellow	0.71	(0.68, 0.74)
	Nurse	0.67	(0.65, 0.69)
ROC Curve (AUC)	Attending	.68	(0.67, 0.69)
	Fellow	.73	(0.67, 0.69)
	Nurse	.69	(0.64, 0.68)
PPV	Attending	0.49
	Fellow	0.49
	Nurse	0.44
NPV	Attending	0.88
	Fellow	0.89
	Nurse	0.83

Attendings First 100 Questions N=1200	No Seizure on EEG	Seizure on EEG	Total
No seizure on CDSA	599	83	682
Seizure on CDSA	265	253	518
Total	864	336

Fellows First 100 Questions N=800	No Seizure on EEG	Seizure on EEG	Total
No seizure on CDSA	393	50	443
Seizure on CDSA	183	174	357
Total	576	224

Nurses First 100 Questions N=1900	No Seizure on EEG	Seizure on EEG	Total
No seizure on CDSA	931	194	1125
Seizure on CDSA	437	338	775
Total	1368	532

CI: Confidence Interval, AUC: Area Under the Curve

Critical Care Attending Physicians

Attending physicians had a CDSA seizure detection sensitivity of 72% [70%, 80%]. (Table 2). Post hoc evaluation of sensitivity for status epilepticus detection was 79% [95% CI: 75%, 83%]. Attending physicians had the same response to images provided in duplicate for a median of 83% of images [74%, 94%], leading to an inter-rater kappa score 70% [95% CI: 66%, 74%] for all images and 70% [62%, 79%] for images with seizures

Critical Care Fellows

Fellows had a CDSA seizure detection sensitivity of 78% [72%, 82%]. (Table 2) Post hoc evaluation of sensitivity for status epilepticus detection was 79% [95% CI: 74%, 84%]. Fellows had the same response to questions provided in duplicate a median of 83% [74%, 94%], kappa score 72% [95% CI: 56%, 67%] for all images and 66% [55%, 78%] for images with seizures.

Critical Care Nurses

Nurses had a CDSA seizure detection sensitivity of 64% [59%, 68%]. (Table 2) Post hoc evaluation of sensitivity for status epilepticus detection was 64% [95% CI: 60%, 68%]. Nurses had the same response to questions provided in duplicate a median of 80% [73%, 87%], kappa score 58% [54%, 62%] for all images and 63% [56%, 70%] for images with seizures.

Discussion

This study demonstrates that is feasible for critical care providers to detect seizures using CDSA EEG in many patients surviving pediatric cardiac arrest. While the sensitivity for seizure detection was only 70%, the NPV was high at 86%. There was a difference in sensitivity between critical care fellows and critical care nurses and attendings, but no difference in specificity or NPV. Importantly, attending physicians were better at identifying images with status epilepticus than isolated seizures.

EEG seizures are common in critically ill patients with acute encephalopathy and there is increasing evidence that high seizure burdens are associated with both worse short and long term outcomes (1–5, 19). Furthermore, studies of patients with status epilepticus indicate that treatment delays are associated with reduced medication efficacy for status epilepticus termination (6, 7). There is increasing evidence that persisting seizures become more self-sustaining and pharmaco-resistant, and recent guidelines addressing status epilepticus management indicate that both clinical and EEG-only seizures should be treated aggressively (20–22). This management objective can only be accomplished if seizures are rapidly identified.

Since most post-cardiac arrest EEG seizures have no clinical correlate, detection requires EEG monitoring (2, 5, 8–15). Many institutions do not have continual access to EEG services and even though EEG is recorded continuously, data review is generally intermittent, most often 1–2 times per day leading to delays between seizure onset and detection (16, 23). Given limited EEG monitoring staffing resources, alternative or supplemental approaches to electroencephalographers' review of conventional EEG interpretation may be valuable. Quantitative EEG techniques may allow more efficient review of EEG data by non-electroencephalographers (24). While the availability of these trends at the bedside cannot obviate the responsibility or need for encephalographer interpretation of raw EEG, it may help augment systems of intermittent EEG review. Adjunctive notification by continuously present ICU personnel of possible seizures may be done with software seizure alerts which may allow for more rapid recognition and notification of encephalographers.

The overall sensitivity for any seizure detection in our study of CDSA EEG interpretation was 70%. This indicates that 70% of images were correctly identified as having seizures by critical care providers. The sensitivity for seizure detection varied by critical care provider training and role. Critical care attendings and fellows had a slightly higher sensitivity than critical care nurses. This may be due to more physician EEG exposure over time in practice or intrinsic differences in practice. While these data would indicate some seizures would not have been recognized, it does not invalidate the importance of these data. On post hoc analysis, critical care attending and fellow participants' sensitivity for status epilepticus detection was 79%. Identifying patients with status epilepticus may be more important than identifying patients with intermittent seizures since studies of critically ill patients undergoing continuous EEG monitoring, higher seizure burdens and status epilepticus, were associated with worse short and long term outcomes (3, 4, 19), whereas briefer seizures were not associated with worse outcomes.

The overall specificity was 68%. This indicates that some images categorized as containing EEG seizures did not contain seizures. False positives may be particularly problematic since they could lead to overtreatment and exposure of non-seizing patients to anti-seizure medications with potential adverse effects. CDSA has the advantage of being easily accessible via multiple commercially available software programs that can convert raw EEG in real time. In large institutions with continuous EEG recording but periodic interpretation, confirmatory electroencephalographer interpretation could be utilized prior to therapeutic interventions based on bedside CDSA EEG interpretations. This approach would allow limited EEG interpretation resources to be directed to specific patients at the right time, rather than having encephalographers screening many hours of EEG most of which would not contain seizures. Healthcare systems where robust continuous EEG monitoring programs are in place could be augmented with more frequent interpretation of EEG by bedside providers plus confirmation by an electroencephalographer's interpretation of the full montage EEG. Health systems where continuous EEG monitoring is not as readily available may be able to use bedside CDSA interpretation, albeit imperfectly, to at least provide some information. This approach has been adopted by many neonatal intensive care units using amplitude integrated EEG; despite recognized inaccuracies in seizure detection it is readily available to bedside providers (25, 26). One study of aEEG interpretation by non-expert adult ICU physicians for seizure identification had a mean sensitivity of 40%. This was lower than our CDSA interpretation sensitivity, however, the aEEG recordings were taken from only bifrontal recordings and are a different trend presentation (27). It is too soon to say whether CDSA EEG will enhance our ability to improve outcomes for our patients with EEG seizures; however, mounting evidence suggests that prolonged seizures lead to worse outcome, and in certain populations earlier detection and treatment may impact outcomes.(2–7, 28) Future studies will need to evaluate whether un-identified and untreated seizures are more harmful than a moderate sensitivity tool for seizure detection.

Participants in our study were provided with a brief standardized powerpoint tutorial that consisted of basic EEG and CDSA training with associated examples and answers. Each subject immediately reviewed and classified the 200 CDSA images. No participant had previous CDSA reading experience. Subject training was brief and focused to reflect how CDSA training might occur in real-time at the bedside. In a 55 bed ICU staffed by over 250 nurses, 15 critical care fellows and 27 critical care attendings it is important to have an accessible training program that can be implemented across a large group. Two previous studies have evaluated CDSA interpretation by neurologists utilizing a more extensive two-hour training session yielding a sensitivity of 83% in one a seizure detection rate of 89% in the other (9, 17, 29). We previously published a study evaluating seizure detection using CDSA after a brief training session by eight encephalographers in two different scenarios with a sensitivity for seizure detection of 65–75% (18). The results of our current study are consistent with the Pensrikul study (18), which consisted of a brief training session. One may speculate that a more intensive training model could enhance sensitivity for seizure detection and specificity to rule out seizure occurrence.

The inter rater reliability for this study was substantial(30). When comparing all providers the kappa was 63% for attendings 70%, fellows 72% and nurses 58%. These data were not substantially different when evaluating the inter-rater reliability for only the images known to contain seizures based on conventional EEG review. Participants were not told they would see images in duplicate. Challenges that may have contributed to these findings were fatigue from reading repeated images, insufficient training, or lack of clarity of the trend presentation. Improvement in training and presenting fewer images may help address these challenges.

There are several limitations to our study. First, EEG tracings were obtained from patients who experienced a cardiac arrest, and these data may not generalize to other acute encephalopathy etiologies. Second, we utilized hemispheric averaging of full montage EEG to minimize the number of panels participants had to interpret, but this approach may have made focal seizures involving only a few channels more challenging to identify. Third, this study was not performed in real-time and therefore clinical context including patient level factors, vital sign changes and previous EEG for these patients were not presented, and this may have made interpretation more challenging. Fourth, we only presented CDSA EEG to our participants and did not evaluate whether other quantitative EEG techniques such as aEEG could have augmented seizure detection or would be superior when used in place of CDSA. Using panels with multiple quantitative EEG analysis types might have led to improved seizure detection. Fifth, our inter-rater reliability was `substantial', but not `almost perfect' as defined by Landis and Koch (30), while intra-rater reliability was a median of 80–84% depending on role. Finally, participants reviewed images showing 2 hours of CDSA EEG as a single static image. In real use at bedside, the image would be slowly acquired, with additional sections in the image appearing over time as EEG is acquired, and this important difference may impact interpretation of the images and timing and accuracy of seizure recognition.

Improving sensitivity and specificity of CDSA interpretation by critical care providers may be achieved by real time bedside use with care providers having access to the patient data and continuous EEG data over prolonged periods of monitoring. Enhanced education, real-time feedback and presentation of multiple quantitative EEG techniques may also improve overall sensitivity and specificity. Thus, our study is just a first step in evaluating CDSA EEG interpretation by critical care providers.

Conclusion

CDSA EEG interpreted by critical care physicians and nurses is feasible after a brief standardized training and has moderate sensitivity for seizure detection in patients resuscitated from cardiac arrest. Evaluation of critical care provider interpretation of CDSA to detect seizures in real time is warranted.