Performance of spectrogram-based seizure identification of adult EEGs by critical care nurses and neurophysiologists

SUMMARY

Introduction

Continuous EEG screening using spectrograms or compressed spectral arrays (CSAs) by neurophysiologists has shorter review times with minimal loss of sensitivity for seizure detection when compared with visual analysis of raw EEG. Limited data is available on the performance characteristics of CSA-based seizure detection by neurocritical care nurses.

Methods

This is a prospective, cross-sectional study that was conducted in two academic neurocritical care units and involved thirty-three neurointensive care unit nurses and four neurophysiologists.

Results

All nurses underwent a brief training session prior to testing. Forty two-hour CSA segments of continuous EEG were reviewed and rated for the presence of seizures. Two experienced clinical neurophysiologists blinded to the CSA data performed conventional visual analysis of the raw EEG and served as the gold standard. The overall accuracy was 55.7% among nurses and 67.5% among neurophysiologists. Nurse seizure detection sensitivity was 73.8%, and the false-positive rate was 1-per-3.2 hours. Sensitivity and false alarm rate for the neurophysiologists was 66.3% and 1-per-6.4 hours, respectively. Inter-rater agreement for seizure screening was fair for nurses (Gwet’s AC1 statistic: 43.4%) and neurophysiologists (AC1: 46.3%).

Conclusions

Training nurses to perform seizure screening utilizing continuous EEG CSA displays is feasible and associated with moderate sensitivity. Nurses and neurophysiologists had comparable sensitivities, but nurses had a higher false-positive rate. Further work is needed to improve sensitivity and reduce false-alarm rates.

1. INTRODUCTION

Numerous studies have shown that seizures are common in critically ill patients, and that the majority of seizures in this population are nonconvulsive (Jordan, 1993; Young et al., 1996; Towne et al., 2000; Claassen et al., 2004). While evaluation with intermittent “routine” (<60 minutes duration) electroencephalograms (EEGs) will capture some nonconvulsive seizures, reliable detection generally requires prolonged continuous EEG (cEEG) monitoring (Claassen et al., 2004; Westover et al., 2015). Full-montage review of cEEG data by an experienced electroencephalographer remains the gold standard for seizure detection. However, the electroencephalographer is typically limited to intermittent cEEG review, particularly as increased cEEG utilization places strain on electroencephalographer-capacity (Scheuer and Wilson, 2004).

Various methods of extracting salient information from cEEG data and compressing cEEG data for more rapid review, broadly referred to as quantitative EEG (qEEG), have been available for many years, and has been proposed as a useful alternative to conventional cEEG review (Talwar and Torres, 1988; Liu et al., 1992; Nuwer, 1996; Nuwer, 1997; Bleck, 2012). Compressed spectral arrays (CSA) are a common way to visualize EEG trends and have been used in most prior studies of the use of qEEG for seizure detection in adults (Moura et al., 2014; Williamson et al., 2014; Dericioglu et al., 2015; Swisher et al., 2015; Topjian et al., 2015). CSA is produced by applying spectral estimation techniques to continuous EEG data, thereby producing a three-dimensional compressed spectrogram with time on the x-axis, frequency on the y-axis, and EEG power on the z-axis (Bickford et al., 1972; Bricolo et al., 1978). Spectral power is conventionally displayed using a color scale, producing a CSA sub-type known as a color density spectral array (CDSA) (Pensirikul et al., 2013).

Prior studies have shown that using CSA as a screening tool to select portions of the cEEG for more detailed review allows trained electroencephalographers to maintain good sensitivity while reducing overall EEG review time (Moura et al., 2014). A handful of studies have also evaluated the sensitivity of CSA alone for detecting seizures by physicians and ICU nurses (Stewart et al., 2010; Pensirikul et al., 2013; Dericioglu et al., 2015; Swisher et al., 2015; Topjian et al., 2015). Of all members of the care team, ICU nurses have the most frequent patient interaction and are typically charged with alerting physicians to changes in vital signs, neurological status, cardiac telemetry and other monitored systems. Therefore, nurses are ideally suited to provide similar screening of CSA data.

Prior studies evaluating the ability of ICU nurses to detect seizures using quantitative EEG tested a small number of subjects at a single institution. Additionally, no formal classification system of specific CSA patterns in critical care EEG monitoring has been described. In the present study we investigated the ability of a large sample of ICU nurses at two academic medical centers to use CSA to identify seizures following a brief training session, and compared their performance with that of experienced electroencephalographers. Moreover, CSA images were categorized based on spectrogram’s visual features, and performance was evaluated for each category separately. We hypothesized that the nurses would be able to detect seizures with good sensitivity, though their sensitivity and specificity would not be as high as trained electroencephalographers.

2. METHODS

Long-term continuous EEG records were obtained from 30 adult subjects during routine clinical care at the Massachusetts General Hospital (MGH) between September 2011 and February 2012 (Table 1). All records were prospectively interpreted by a neurophysiologist during clinical care. Acquisition and de-identification of EEG data, and calculation of spectrograms, was performed under an approved IRB protocol at MGH. Voluntary participation of nurses at MGH was carried out as part of quality improvement efforts, and did not require IRB approval. The study was approved by the University of Michigan (UM) IRB, and all participating nurses provided informed consent. Two independent adult electroencephalographers blinded to the CSA display reviewed the raw EEG data page-by-page. Each two-hour EEG epoch was scored for presence or absence of seizures. Electrographic seizures were defined using previously published criteria as abnormal paroxysmal events that were different from the background, lasted longer than 10 seconds, and had a temporal-spatial evolution in morphology, frequency, and amplitude with a plausible electrographic field (Kilbride et al., 2009). In case of discrepancy, classification and quantification of seizures were achieved by consensus.

Table 1.

Baseline demographics and diagnosis of patients

	Seizure group (N=16)	Control group (N=14)	Total
Age (Mean ± SD)	54.9 ± 19.8	62.7 ± 17.3	58.5 ± 18.8
Female	6	4	10
Primary diagnosis:
Seizure	7	4	11
Encephalopathy	3	2	5
Stroke	0	2	2
Cardiac arrest	2	0	2
Encephalitis	0	1	1
Brain neoplasm	1	2	3
Hypoperfusion syndrome	0	1	1
Subdural hematoma	0	1	1
Sepsis	2	0	2
Subarachnoid hemorrhage	1	1	2

CSA = compressed spectral array

2.1 Participants and CSA interpretation training

All nurses working at a neurocritical care intensive care unit (Neuro-ICU) in two university hospitals (UM and MGH) were invited to participate in the study. The neurologists recruited to this study had completed fellowship training in clinical neurophysiology. The nurses and physicians included in this study were not directly involved in the formulation of the teaching and testing materials. All nurses participated in a small group session led by one of the investigators (EA; CAW; LMVRM; MBW) for 15 minutes. Tutorial content included: EEG-reading introduction, CSA theory, CSA interpretation and correlation with raw EEG data, and epileptic patterns and artifact identification using CSA. The tutorial was followed by a web-based test with a 10-question pre-test performed using Survey Monkey (Survey Monkey, Inc., Palo Alto, CA), and was followed by a discussion of pitfalls on interpretation of CSA images for 25 minutes. The training slides and pre-test questions are included as on-line supplementary materials. The neurophysiologists were given a complete electronic version of the tutorial provided to the nursing team in the form of a handout, and the investigators were available to answer questions remotely or in person on an as-needed basis. All neurophysiologists had used CSA in routine clinical practice for no less than 2 years.

A web-based test with 40 different CSA images obtained from 30 distinct patients (each containing two hours epochs of EEG data) was performed after the tutorial and pre-test answers discussion were completed. For each CSA image, participants were asked to indicate whether any seizures were present and seizure count using a ratio scale. During the test, participants were not permitted to ask questions about the CSA images nor review materials related to the topic. Demographic information for each study participant, including duration of clinical practice and previous experience with EEG analysis was obtained. The complete test images and answers are available as supplementary material.

2.2 CSA images

Two-hour segments of continuous EEG data with a sampling rate of 512Hz were converted to CSA images using color density spectral array and asymmetry index as previously described (Westover et al., 2015). Briefly, spectrograms were computed using multitaper spectral estimation with a window size of four seconds with 0.1 s overlap implemented in the Chronux toolbox (http://www.chronux.org) using MATLAB software (Natick, MA, USA).

Forty CSA displays (see Supplementary Material) were generated for the test with a seizure prevalence of 50%. The CSA displays not containing seizures were selected to provide a broad sampling of common cEEG patterns, including lateralized periodic discharges, rhythmic delta activity, burst-suppression, focal and generalized slowing. Participants were not aware of the prevalence of seizures and other epileptiform patterns included in the test. Concomitant review of raw EEG data was not available for the participants in the study.

2.3 CSA display post hoc visual review

To facilitate analysis of which, if any, CSA patterns are associated with high or low accuracy for seizure detection, the authors reviewed all CSA images and visually classified all seizures into five categories. We chose the following names for these categories to suggest their appearance in the spectrogram: suppressed, broadband-monotonous, solid flame, irregular flame, and artifact. Representative examples of the five CSA categories that we developed to classify events of interest in each spectrogram are shown in Figure 1. “Flames” describe the abrupt appearance of CSA segments with higher bandwidth and power, vaguely reminiscent of candle flames. Flame events that have smooth edges, regular appearance, often with a gradual crescendo or decrescendo pattern, are referred to as “solid flames. Flame events that have a prominent “choppiness” or “irregularity” are termed “irregular flames.” “Broadband/monotonous” spectrogram events are characterized by sustained higher power at low frequencies with either minimal variation or gradual waxing and waning of frequencies within the high-power band. “Suppressed” spectrograms have low power diffusely and tend to be relatively monotonous. “Artifact” describes CSA images that are dominated by various types of artifact. “Massive artifact” describes the appearance of spectrograms with prominent lead, motion or other artifact. These spectrograms can include the abrupt appearance of high-power signals that saturate all frequency bands, typically when significant lead artifact is present. The “stalactite artifact” sub-type features bands of high-power that appear to descend from the top of the spectrogram, and are typically due to muscle or motion artifact.

Figure 1.

Representative images of the 5 categories, including the two artifact sub-categories that each CSA image was assigned to. The vertical axis represents the spectrogram frequency from 0–20 Hz, and the color bar indicates how the log-power (dB) of the frequency components is represented by colors in the spectrogram. The horizontal axis corresponds to 2-hours of time. All test images with their assigned categories are available as supplementary material for review. A. Solid Flame – Note the repeated appearance of regular, rectangular shaped “solid flames” associated with an abrupt increase in power and frequency with gradual frequency decrease (decrescendo). Each of the flames represents a brief nonconvulsive seizure, which were detected by participants with a high rate of accuracy. B. Irregular Flame – This image has several instances of abrupt, irregular increases in frequency and power consistent with the “irregular flame” pattern. There are no seizures present, but this image and others in this category were associated with a high rate of false-positive selections. C. Broadband/Monotonous – The first hour of this slide is monotonous with a broad, high-powered band at lower frequencies. This pattern waxes and wanes slightly in the second hour. Three prolonged nonconvulsive seizures were present in this image. D. Suppressed – This image, taken from a patient in burst-suppression without seizures, is monotonous with low power at all frequencies. E. Artifact (Massive) – This image shows irregular high-power signal that diffusely saturates all frequencies at nearly all time points. No seizures are present. F. Artifact (Stalactite) – Repeated muscle artifact appears to “rain down” from higher frequencies, and is superimposed on a monotonous, relatively suppressed background. No seizures are present.

Two authors (EA, CAW) independently reviewed each image and assigned it to one of these five categories. When multiple or overlapping characteristics were present, the case was assigned to the best fitting category. All disagreements were resolved by joint review consensus of the two authors. The proportions of each pattern category were compared for images with and without seizures, and accuracy for each category was calculated.

2.4 Statistical analysis

The performance of CSA seizure screening by nurses and expert neurophysiologists using two-hour long CSA displays was compared with conventional EEG reading by electroencephalographers as the ground truth. Sensitivity (Se), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) for seizure detection were calculated for the nurse and neurophysiologist groups separately. Given the relative small number of participating neurophysiologists and nurses, formal statistical testing to assess group differences was not performed due to lack of power to detect differences among groups. We defined the false alarm rate as the expected time between incorrectly tagging two-hour epochs as containing seizures when in fact there are no seizures present. We estimated the false alarm rate (FAR) as the duration of the CSA blocks (2 hours) divided by one minus the specificity, FAR = 2/(1−Spec). Accuracy was defined as true positive plus true negative/total.

Inter-rater agreement for seizure detection scoring was obtained using percentage agreement and with the Gwet’s AC1 (Gwet, 2008). Unadjusted odds ratios and 95% confidence intervals were calculated. Statistical significance was determined at the α level of 0.05. All data analysis was conducted with MATLAB, version 17, (Natick, MA, USA).

3. RESULTS

Thirty-three Neuro-ICU nurses and four neurophysiologists were included in this study. None of the nurses had previous training in the interpretation of EEG or CSA. The mean (SD) Neuro-ICU experience of nurses was 6.7 ± 8.2 years. The predictive values for seizure detection for nurses and neurophysiologists are shown in table 2. Average seizure detection sensitivity for nurses was 73.8%, with a specificity of 37.6%. The false alarm rate was one false alarm per 3.2 hours. Average sensitivity and specificity for the neurophysiologist group was 66.3% and 68.8%, respectively, and the false alarm rate was one per 6.4 hours.

Table 2.

Seizure detection performance by nurses and neurophysiologists

	Nurses (N=33)	Neurophysiologists (N=4)
Total Accuracy	55.7%	67.5%
Sensitivity	73.8%	66.3%
Specificity	37.6%	68.8%
False-Positive Rate (per hour of CSA)	3.2/h	6.4/h
Positive Predictive Value	54.7%	69.4%
Negative Predictive Value	58.8%	67.3%

CSA = compressed spectral array

The overall accuracy for the presence or absence of seizures was 55.7% for nurses and 67.5% for neurophysiologists. Average positive and negative predictive values for nurses were 54.7% and 58.8%, respectively. Average positive and negative predictive values for neurophysiologists were 69.4% and 67.3%, respectively. For seizure detection, there was fair inter-rater agreement between nurses (AC1: 43.4%) nurses and neurophysiologists (AC1: 46.3%). Inter-rater agreement for seizure count per CSA display was moderate for nurses (AC1: 71.9%) and fair for neurophysiologists (47.2%).

Of the 20 spectrograms containing seizures, 13 spectrograms had a solid flame pattern, six were broadband/monotonous, and one had an irregular flame pattern. Of the 20 without seizures, 12 had irregular flames, three were suppressed, four were classified as artifact, and one as broadband/monotonous. Table 3 describes the performance of nurses within each spectrogram category.

Table 3.

Nurse and neurologists performance for seizure presence screening stratified by compressed spectral array category

CSA Category	Total CSA Display Number (N=40)	CSA Display with seizures (N=20)	Nurses Accuracy (%)	Neurologists Accuracy (%)
Solid Flame	13	13	83.8	82.7
Irregular Flame	13	1	24.5	59.6
Broadband/monotonous	7	6	51.9	32.14
Artifact	4	0	51.8	75.0
Suppressed	3	0	83.0	100.0

CSA = compressed spectral array

Fourteen spectrograms contained intermittent periodic discharges that did not meet criteria for seizures. Six of these were classified as “solid flames,” three as “irregular flames,” four were broadband/monotonous”, and one as “suppressed.”

4. Discussion

This study suggests that seizure identification by ICU nurses using compressed spectral arrays following a brief training session is feasible. Sensitivity was moderate and the false alarm rates were relatively high, one per 3.2 hours. Experienced neurophysiologists achieved a similar sensitivity, but had less frequent false alarms, one per 6.4 hours, and higher positive predictive values. It should be noted that these values do not represent the absolute sensitivity of seizure detection, but rather the sensitivity and specificity with which the presence of seizures were identified on a CSA display representing a two-hour epoch of EEG data.

Only a handful of earlier studies investigating the use of quantitative EEG for seizure detection are available for comparison. Most recently, Swisher et al. investigated the use of a panel of qEEG displays by 5 neurophysiologists, 7 EEG technologists and 5 ICU nurses at a single center (Swisher et al., 2015). Both neurophysiologists and ICU nurses averaged 87% sensitivity and 61% specificity for seizure detection, while the EEG technologists had both a sensitivity and specificity of 80%, without any statistically significant differences between groups. The current study differs in the much larger number of nurses participating and the use of a standardized training protocol at two academic centers. While nurse sensitivities were somewhat lower in the current study, the specificity was much less (37.6% vs. 61%). Potential reasons for this difference include a lower percentage of seizures in the current study (50% vs. 58%), a more challenging imaging set, and the use of additional qEEG trends in Swisher et al. compared to CSA alone in the present study.

Stewart et al. investigated the sensitivity and false-positive rate with which three neurophysiologists detected seizures using two types of quantitative EEG in pediatric ICU patients: color-density spectral array (CDSA), as was used in the present study, and amplitude-integrated EEG (aEEG), which is widely used for seizure detection in neonatal ICUs. Their sensitivity ranged from 73.3% to 86.7% with CDSA, and 80.6% to 83.9% with a-EEG. False-positive rates were low, with one false-positive per 17 hours of CDSA displayed (Stewart et al., 2010). In a similar study of 39 pediatric ICU patients who had all been resuscitated from cardiac arrest, using CSA 12 pediatric ICU attendings, eight ICU fellows and 19 nurses had a sensitivity of 72%, 78% and 64%, and a specificity of 69%, 68% and 68%, respectively (Topjian et al., 2015). In another study by the same group, average neurophysiologist sensitivity for seizure detection in pediatric ICU patients ranged from 64.8% to 75% with two different review methods, and specificities were 92.3% and 78.2% (Pensirikul et al., 2013). In a similar but smaller study, Dericioglu et al. evaluated the ability of a critical care neurology fellow, one neurology resident and two Neuro-ICU nurses to detect seizures using aEEG and CDSA. They reported a sensitivity ranging from 88% to 99% and a specificity of 89 to 95%, with an overall false-positive rate of 1 per 2 hours of EEG (Dericioglu et al., 2015).

Though differing in the patient populations that were assessed, the sensitivity of seizure detection in Topjian et al. was comparable to the results of ICU nurses in the current study (Topjian et al., 2015). The specificity of ICU nurses in the current study was lower than the pediatric ICU nurses assessed by Topjian et al. There are several potential reasons for this difference. In particular, Topjian et al. limited their patient population to pediatric patients who had suffered cardiac arrest, so the images were likely much less heterogeneous. Dericioglu et al. noted similar false-positive rates of one seizure per two-hours of EEG, but the two nurses in this study had much higher sensitivities (Dericioglu et al., 2015). To ensure that nurses and neurophysiologists were presented with EEGs that reflect the wide variety of patterns seen in ICU practice, the current study used a challenging set of images that included many cases with significant artifact as well as periodic patterns on the ictal-interictal continuum (see supplementary material for the complete test images). Moreover, the short duration of the CSA displays that were used limits the ability of reviewers to assess the evolution and recurrence of specific CSA patterns. Taken together, we anticipate that the performance of nurses and neurophysiologists may be much different if they were presented with prolonged continuous EEG data in conjunction with information about which imaging patterns were associated with seizures and other patterns of interest.

Strengths of the present study include its large sample size and the inclusion of Neuro-ICU nurses from two different academic medical centers. With 33 nurses and four neurophysiologists participating, this is the largest study to date of the ability of nurses to detect seizures using quantitative EEG. There are also important limitations that should be highlighted. Though we attempted to include a variety of challenging CSA images that represent the breadth of seizure types, periodic discharges, and artifacts seen in ICU practice, the present study was not conducted in real-time at the bedside, and thus may not accurately reflect nurse performance in clinical practice. By conducting testing following a single brief training session, this study likely underestimates the eventual performance after nurses gain increased experience with the technology.

It is also important to note that, while bedside EEG review using CSA by nurses might expedite seizure screening and bring these patterns to the attention of the treating physician earlier, there is also the chance for increase in false-alarms and resulting “alarm-fatigue”. False-alarm notification can increase workload burden to the neurophysiologist in charge of EEG review as well as the nurses screening CSA displays. Overall, the nurse false alarm rate was approximately twice that of neurophysiologists. These findings highlight that use of CSA review by non-experts or neurophysiologists without accompanying review of the raw EEG may be inadequate for sensitive seizure detection. Future work may improve the utilization of CSA by non-experts by including a more comprehensive training. Alternatively, some studies utilizing fully automated detection of seizures and periodic discharges have shown promise, and with continued development such technology may obviate the need for human screening (Cloostermans et al., 2011).

The comparable inter-rater agreement in both neurophysiologists and nurses’ groups in addition with the inferior inter-rater agreement for seizure count among neurophysiologists compared to nurses indicates that the training model was not the sole explanation of the relatively low specificity found in both groups. These results are likely secondary, at least in part, to the design of the study itself, as raw EEG review was not available to participants and EEG epochs of particularly high complexity were selected. With the intention to simulate real practice and assist in the identification of which CSA patterns would be the most difficult to differentiate from seizures, several EEG records with patterns in the ictal-interictal continuum were included (Chong and Hirsch, 2005; Hirsch et al., 2013). Correct seizure identification in these cases can be exceedingly difficult. These patterns can have identical or near-identical CSA signatures when compared to seizures. Indeed, the distinction between seizure and highly pathological non-seizure patterns remains a matter of debate, and may be largely artificial (Ng et al., 2015).

In order to improve detection of CSA signatures concerning for seizures, defining patterns prone to screening failure is of preeminent importance. Both solid flame and irregular flame patterns are associated with transient increments in power that stand out from the background, and therefore were often scored as seizures. However, irregular flames were rarely considered to be seizures in the ground truth data, and were thus associated with a high rate of false positives. Nurses overwhelmingly classified all types of “flame” patterns as containing seizures, which led to a large number of false-positives in spectrograms with “irregular flames.” The fact that the CSA signature of seizures is sometimes indistinguishable from other interictal patterns reflects the fact that this differentiation can be difficult even when reviewing the raw EEG.

5. Conclusion

This study demonstrates that after an approximately one-hour training session critical care nurses were able to perform seizure screening using continuous EEG CSA displays. Sensitivity of seizure screening between nurses and neurophysiologists was similar, raising the possibility that nurse review of spectrograms may offer earlier seizure detection by virtue of more frequent evaluation without significant loss in sensitivity. However, the high rate of false-positive seizure detections despite additional training might increase alarm-fatigue in clinical practice. In order to make real-time seizure screening by nurses at the bedside feasible, further studies utilizing a more comprehensive teaching program focused on improving screening accuracy are warranted. Lessons learned about the types of CSA signatures associated with missed seizures and false alarms will support the development of more effective teaching strategy for future studies.