Quantitative EEG-Based Seizure Estimation in Super-Refractory Status Epilepticus

Abstract

Background:

The objective of this study was to evaluate the accuracy of seizure burden in patients with super-refractory status epilepticus (SRSE) by using quantitative electroencephalography (qEEG).

Methods:

EEG recordings from 69 patients with SRSE (2009–2019) were reviewed and annotated for seizures by three groups of reviewers: two board-certified neurophysiologists using only raw EEG (gold standard), two neurocritical care providers with substantial experience in qEEG analysis (qEEG experts), and two inexperienced qEEG readers (qEEG novices) using only a qEEG trend panel.

Results:

Raw EEG experts identified 35 (51%) patients with seizures, accounting for 2950 seizures (3,126 min). qEEG experts had a sensitivity of 93%, a specificity of 61%, a false positive rate of 6.5 per day, and good agreement (κ = 0.64) between both qEEG experts. qEEG novices had a sensitivity of 98.5%, a specificity of 13%, a false positive rate of 15 per day, and fair agreement (κ = 0.4) between both qEEG novices. Seizure burden was not different between the qEEG experts and the gold standard (3,257 vs. 3,126 min), whereas qEEG novices reported higher burden (6066 vs. 3126 min).

Conclusions:

Both qEEG experts and novices had a high sensitivity but a low specificity for seizure detection in patients with SRSE. qEEG could be a useful tool for qEEG experts to estimate seizure burden in patients with SRSE.

Introduction

Nonconvulsive seizures (NCS) and nonconvulsive status epilepticus (NCSE) occur often in critically ill patients [1]. By definition, NCS and NCSE have either no or subtle clinical manifestations, and affected patients are often comatose or sedated and/or paralyzed for other medical issues. As such, diagnosis rests on detection with continuous electroencephalographic (EEG) monitoring. Mounting evidence suggests that NCS are harmful and associated with neurophysiologic disturbances, increased morbidity, and mortality [2–10]. Higher seizure burden has been associated in both children and adults with neurologic decline and worse functional outcomes [11–13].

Guidelines recommend continuous EEG monitoring of select high-risk patients within the intensive care unit (ICU) to detect NCS and NCSE [1, 14, 15]. To ensure appropriate treatments are delivered in a timely fashion, strategies such as quantitative EEG (qEEG) can be used to better integrate EEG data into patient care [16].

Studies have shown that both neurophysiologists and bedside providers can use qEEG to efficiently screen for NCS [17–29]. Most studies have focused on the use of a single qEEG trend, whereas few have examined the accuracy of multiple qEEG trends to identify NCS in patients with diverse pathologies [21, 22, 27, 28]. In addition, most studies have focused on the diagnostic utility of qEEG trends in the general ICU population, where seizure prevalence is lower and EEG background is less likely to contain patterns along the ictal-interictal continuum (IIC).

Super-refractory status epilepticus (SRSE) is defined as persisting or recurring seizures despite the use of anesthetic agents [30–32]. Treatment of SRSE is complex, is based on electrographic findings, and carries with it several risks that must be weighed against the potential benefit of seizure suppression [33]. A specific seizure burden threshold may one day serve as a potential therapeutic target [12]. To integrate this potential therapeutic target into patient management, one would need to ensure that seizure burden could be efficiently and effectively quantified in real time.

The aim of this study was to evaluate quantification of seizure count and burden by using qEEG reviewers with varying levels of experience and raw EEG review serving as the gold standard in patients with SRSE. We hypothesized that qEEG can be used as a tool to estimate seizure burden in patients with SRSE.

Methods

Study Cohort

We retrospectively identified 69 patients with SRSE (> 18 years old) admitted to the neurological ICU at Columbia University Medical Center between January 1, 2009, and January 1, 2019. SRSE was defined as ongoing electrographic seizures despite intravenous anesthetic use [30, 31, 34]. All patients received anesthetic infusions for the management of SRSE (midazolam, propofol, ketamine, and/or pentobarbital). We collected basic demographics, status epilepticus etiology, and outcome measures. Outcome measures included tracheostomy, hospital length of stay, mortality, discharge disposition, modified Rankin scale score, and extended Glasgow Outcome Scale score on discharge. The study was approved by the Institutional Review Board of Columbia University Medical Center.

Study Design: EEG and qEEG Review Arms

The study included three groups of blinded reviewers:

1. Gold standard (raw EEG experts): two board-certified neurophysiologists (AA and JK) reviewed all available raw EEG data without access to the qEEG trends and individually marked the onset and offset of all electrographic seizures

2. qEEG experts: two neurocritical care providers with experience in critical care EEG and qEEG analysis (JPK and SLG) reviewed qEEG trends without access to raw EEG data

3. qEEG novices: an in-training neurology resident (NM) and a neurocritical care nurse (RAR) also reviewed qEEG trends without access to raw EEG data

All reviewers were blinded to clinical data, other reviewers’ annotations, and original EEG annotations.

Continuous Raw EEG Recordings and qEEG Trends

Continuous EEG recordings were conducted by using the 10–20 system of electrode placement and a digital bedside video monitoring system (low-pass filter = 70 Hz, high-pass filter = 0.1 Hz, sampling rate = 200, 256, or 512 Hz) (XLTEK; Excel-Tech Corp., Natus Medical Incorporated, Oakville, ON, Canada) with 21 EEG channels available for analysis. We randomly selected 24 consecutive hours of continuous EEG recording for each patient with SRSE to include recordings with and without seizures. All EEG data files were copied and stored locally for further qEEG analysis. Annotations were removed from the study files. A commercially available qEEG software (Persyst 13, Advanced Review Version; Persyst Development Corporation, San Diego, CA, USA) was used to generate qEEG trends for all collected EEG data sets. QEEG trends included a comprehensive panel of muscle and eye artifact intensity, seizure and spike detection, seizure probability, rhythmic delta detection, rhythmicity spectrogram, amplitude-integrated EEG (aEEG), FFT spectrogram (color density spectral array), relative asymmetry spectrogram, and suppression ratio. An artifact reduction feature was applied to all the qEEG trends that employed factory settings available by default in the P13 program.

Seizure Identification Protocol

Seizures were defined as generalized or focal spikes, sharp waves, or spike-and-wave or sharp-and-slow-wave complexes occurring at ≥ 3 per second or < 3 per second but at least ≥ 1 per second with clear evolution in frequency, morphology, or location lasting at least 10 s in duration [35]. Expert reviewers annotated seizure onset and offset on raw EEG to calculate gold standard seizure count and burden (in minutes). Additionally, one of the EEG experts annotated the presence of commonly seen raw EEG patterns in the ICU: burst-suppression (BS), rhythmic, and periodic patterns [35]. We classified definite seizures as an EEG segment that both raw EEG experts identified as seizure and probable seizures when only one expert identified the segment as seizure. For each recording, we calculated the seizure count as the average of seizure counts calculated by the two EEG experts. Seizure burden was calculated as the ratio of duration of seizures to the total length of recording (24 h).

All qEEG reviewers received an hour-long slide-based and hands-on training module. The training included basic seizures and status epilepticus definitions, introduction to qEEG, and qEEG trends and their interpretation. Multiple qEEG examples of common patterns seen in critical care EEG monitoring were presented: seizures, BS, and rhythmic and periodic patterns [35].

qEEG experts and novices used a uniform approach consisting of the following steps to annotate seizures:

• Step 1: Seizure events were identified by using a combination of seizure probability, rhythmicity spectrogram, color density spectral array, muscle and eye artifact intensity, suppression ratio, and aEEG. Spike detection at 3 Hz was used to emphasize seizure events when reviewing qEEG data. Spike detections below 3 Hz and the rhythmic delta patterns were used to help distinguish IIC patterns from unequivocal seizures.

• Step 2: Raters were instructed to precisely mark seizure onset and offset on qEEG using a single-pixel digital cursor with a 1-h time scale view. Clear changes in rhythmicity or FFT spectrograms (color density spectral array) and aEEG from pre-seizure baseline were used to identify seizure onset and offset.

Seizure count and seizure burden (in minutes) was calculated by using the annotations for seizure onset and offset. Seizure burden quantified by each rater or arm was calculated as the proportion of average seizure burden estimated by the experts on raw EEG. If the burden estimated by the rater or arm was greater than the burden estimated by the experts, the proportion was capped at 1.

Statistical analysis

Using seizures identified on raw EEG as the gold standard, diagnostic metrics of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and false positive rate (FPR) were calculated for the qEEG experts and novices. Continuous and categorical variables were summarized by using means and frequencies (percentages), respectively. Pearson correlation tests were used to examine relations between continuous variables, and χ² tests (or Fisher’s exact tests, when appropriate) were used to examine relations between categorical variables. Unpaired t tests (or Mann–Whitney U tests, when appropriate) and paired t tests were used to examine independent and dependent group differences for continuous variables, respectively. Interrater agreement was assessed by using the intraclass correlation and Cohen’s κ for continuous and categorical variables, respectively. Agreement for intraclass correlations was classified as follows: < 0.50 = poor, 0.50–0.75 = moderate, 0.75–0.90 = good, and > 0.90 = excellent; agreement for κ was classified as follows: < 0.20 = poor, 0.20–0.40 = fair, 0.40–0.60 = moderate, 0.60–0.80 = good, and > 0.80 = excellent [36]. All analyses were conducted by using SPSS v.26 (IBM Corp., Armonk, NY, USA), and p values < 0.05 were considered statistically significant. Figures were generated by using Tableau Software (version 2020.3.3).

Data Availability

Data sets are available from the corresponding author for any qualified investigator.

Results

Study Cohort

A total of 69 patients with SRSE were included in the study. The average age was 55 ± 19 years old, and 46% of patients were women. Epilepsy was the most common etiology (16%) for SRSE, followed by traumatic brain injury (13%), central nervous system tumor (13%), infection (12%), and anoxic brain injury (10%). Average hospital length of stay was 20 (quartile 1–quartile 3: 10–37) days. All patients received midazolam with an average of 2 (quartile 1–quartile 3: 1–2) anesthetics (midazolam, propofol, ketamine, and pentobarbital). A total of 29 (42%) patients had tracheostomy during the ICU stay, and a third of patients died during hospitalization. The median extended Glasgow Outcome Scale score was 3 (quartile 1–quartile 3: 1–3), and the modified Rankin Scale score was 5 (quartile 1–quartile 3: 4–6) on discharge.

Raw EEG characteristics

On the basis of raw EEG, board-certified neurophysiologists (gold standard) diagnosed 35 (51%) patients with seizures (focal in 72%). Among our cohort of 69 patients, 16 (23%) patients had lateralized periodic discharges (LPDs), 9 (13%) patients had generalized periodic discharges (GPDs), 6 (9%) patients had IIC patterns, and 15 (22%) patients had BS patterns. The two raters together identified 2,950 seizures contributing to 3,126 min of seizure burden in 99,360 min of recordings. In patients with seizures, the median number of seizures per patient per 24 h was 29 (quartile 1–quartile 3: 7.5–63). On raw EEG, experts identified 2,950 seizures: 1,176 (40% of all events) as definite seizures, wherein there was agreement, and 1,774 (60% of events) as probable seizures, wherein there was disagreement (Fig. 1). There was an association between higher seizure burden and the presence of EEG findings of IIC patterns (odds ratio [OR] 2.3, p = 0.02).

Fig. 1.

Performance of quantitative electroencephalography (QEEG) experts and QEEG novices across electroencephalographic (EEG) recordings. The figure depicts the sensitivity and false positive rates for QEEG experts and novices for individual recordings compared with raw EEG review (gold standard). The average sensitivity for QEEG experts for each recording is depicted in blue, and that for QEEG novices is depicted in orange. The average false positive rate for QEEG experts is depicted in dark green, and that for QEEG novices is depicted in light green. The y₁ axis depicts sensitivity from 0 to 100%, and the y₂ axis depicts the false positive rate in 0–100 events per day. Recordings 67–69 have false positive rates beyond 100 per day. The recordings are arranged in the descending order of average sensitivities

Performance of qEEG Experts

Sensitivity

qEEG experts identified 55 (80%) patients as having seizures. There was no correlation of sensitivity with pre-dominant seizure type (generalized vs. focal) or other EEG patterns (LPDs, GPDs, IIC, BS) in the recordings (Figs. 1, 2).

Fig. 2.

Diagnostic performance of quantitative electroencephalography (QEEG) experts and QEEG novices at the seizure level. The figure shows the specificity (Sp), sensitivity (Sn), positive predictive value (PPV), negative predictive value (NPV), and false positive rate (FPR) for QEEG experts and novices evaluating the presence of seizures in 24-h electroencephalographic (EEG) recordings in patients with super-refractory status epilepticus (SRSE) compared with raw EEG review (gold standard). The heatmap indicates the diagnostic performance measure for each rater. QEEG experts had high Sn, adequate Sp, and low FPRs, whereas QEEG novices had high Sn, low Sp, and high FPRs

Seizure Count

Seizure count quantified by the qEEG experts was comparable with that estimated on raw EEG by experts (p = 0.41). qEEG experts identified approximately 66% of definite seizures, with a higher likelihood of qEEG experts identifying definite seizures vs. probable seizures (χ² = 848 and 356, p < 0.001). There was good agreement (κ = 0.64) between both qEEG experts when seizures were classified as definite and poor agreement (κ = 0.15) when seizures were classified as probable. There was moderate agreement (interclass correlation = 0.52) between both qEEG experts regarding seizure count (Fig. 3).

Fig. 3.

Interrater correlation among quantitative electroencephalography (QEEG) experts and novices. The heatmap presents the interrater correlation among all raters for QEEG recordings. Blue colors indicate high correlation between the raters in term of seizure count, whereas orange colors indicate low correlation

Seizure Burden

Seizure burden estimated by qEEG experts was comparable with that quantified on raw EEG review by EEG experts (3,126 vs. 3,257 min, p = 0.83). qEEG experts reported higher seizure burden when seizures were generalized (OR 2, p = 0.04) and when the EEG background had BS patterns (OR 2.6, p = 0.009).

Individual Performance

qEEG expert 1 identified 51 (74%) patients as having seizures, with seizure-level performance metrics as follows: sensitivity, 94%; specificity, 47%; PPV, 65%; NPV, 89%; and FPR, 26% (12 seizure per day). qEEG expert 1 identified 77% of definite seizures. qEEG expert 2 identified 41 (59%) patients as having seizures, with seizure-level metrics as follows: sensitivity, 91%; specificity, 74%; PPV, 78%; NPV, 89%; and FPR, 13% (1 seizure per day). qEEG expert 2 identified 43% of definite seizures.

Performance of qEEG Novices

Sensitivity

qEEG novices classified all 69 (100%) patients as having seizures. There was no significant correlation of the sensitivity with seizure type (generalized vs. focal) or other EEG patterns (LPD, GPDs, IIC, BS) based on raw EEG background.

Seizure count

qEEG novices counted significantly more seizures per recording when compared with that estimated by experts (gold standard) on raw EEG (t = 2.1, p = 0.04). qEEG novices identified 77% of definite seizures, with a higher likelihood of qEEG novices identifying definite seizures vs. probable seizures (χ² = 913 and 582, p < 0.001). There was fair agreement (κ = 0.4) between both qEEG novices when seizures were classified as definite and moderate agreement (κ = 0.53) when seizures were classified as probable. There was moderate agreement (interclass correlation = 0.7) between both qEEG novices regarding seizure count (Fig. 2). The FPR was much higher in the qEEG novice group in comparison with the qEEG expert group.

Seizure Burden

Seizure burden estimated by the qEEG novices was significantly higher when compared with the gold standard arm (6,066 vs. 3,126 min, p < 0.0001). There was also a statistically significant difference between the qEEG experts and qEEG novices on comparing the estimated seizure burdens (3,257 vs. 6,066 min, p < 0.0001). No significant relation was found between seizure burden and seizure type (generalized vs. focal) or other EEG patterns (LPD, GPDs, IIC, BS).

Individual Performance

qEEG novice 1 identified 34 (49%) patients as having seizures. Seizure-level performance metrics were as follows: sensitivity, 97%; specificity, 26%; PPV, 58%; NPV, 90%; and FPR, 36% (12 seizures per day). qEEG novice 1 identified 73% of definite seizures. qEEG novice 2 identified 69 (100%) patients as having seizures. Seizure-level performance metrics were as follows: sensitivity, 100%; specificity, 0%; PPV, 51%; and FPR, 49% (18 seizures per day). qEEG novice 2 identified 70% of definite seizures.

Discussion

Our study demonstrates that EEG readers with prior qEEG experience were able to quantify the number of seizures and the seizure burden in patients with SRSE accurately using qEEG after an 1-h training module. Raters without any prior qEEG reading experience, despite completing the same 1-h-long training module, reported higher seizure counts and a higher seizure burden compared with the gold standard, resulting in a high FPR. To our knowledge, this is the first study to evaluate the role of multiple qEEG trends in seizure diagnosis and quantification of seizure burden in patients with SRSE. In addition, our study is unique in comparing the performance of users with and without prior qEEG experience. qEEG as a screening tool to identify seizures may have great value because electroencephalographer review of EEGs is infrequent [15] and often not available in real time. Critical care providers’ qEEG reading performance has been evaluated in multiple studies, with high sensitivity and specificity for presence of seizures [21, 22, 29, 37–40]. In the intensive care setting, the sensitivity and specificity for qEEG novices screening for seizures ranged from 40 to 93% and from 38 to 95%, respectively [21, 24, 26–29, 39–41]. Our study validates the findings of prior reports; raters with prior qEEG experience had high sensitivity, adequate specificity, and low FPRs when identifying presence of seizures, whereas raters without prior qEEG experience had high sensitivity, low specificity, and high FPRs. The findings support the role of the expert bedside providers in screening for seizures in the highly complex population of patients with SRSE, who tend to have severely abnormal EEG background and higher seizure burden. This screening approach in patients with SRSE may translate into more timely review of raw EEG by experts through alerting the electroencephalographers, eventually leading to faster decisions regarding therapy and its titration [24–26, 29]. qEEG novices had high FPRs, limiting the role of nonexperts in screening for seizures in patients with SRSE.

Most studies have used a single qEEG trend, whereas few have examined the accuracy of multiple ≥ 2) qEEG trends in identifying NCS in patients with a variety of pathologies [21, 22, 27, 28, 40, 42]. Color density spectral array and aEEG are among the most used trends. The use of a single quantitative trend produced low sensitivity and specificity in some qEEG studies [24, 39, 41]. We evaluated multiple trends for a couple of reasons: (1) to determine the best performance feasible through the simultaneous use of multiple trends, such as muscle artifact, seizure and spike detection, seizure probability, rhythmicity spectrogram, aEEG, color density spectral array, relative asymmetry spectrogram, and suppression ratio, and (2) to overcome the challenges imposed by the SRSE population that is likely to contain several IIC patterns on their EEGs.

In our report, both qEEG experts and novices had higher likelihoods of classifying seizures when both neurophysiologists (gold standard) identified the segment as seizures (definite). Good interrater agreement was only found between the qEEG experts when classifying definite seizures. These findings are reassuring because neurophysiologists may infrequently classify the same events as ictal and nonictal.

In addition to detecting the presence of seizures, quantifying seizure burden is of interest because a higher seizure burden has been associated with worse outcomes [13, 43]. Most qEEG studies are limited to evaluating the diagnostic performance to the presence or absence of seizures. We evaluated both seizure count and burden (per minute) in addition to the presence of seizures. Our qEEG experts reported comparable seizure count and burden based on qEEG when compared with the gold standard. The higher seizure report by the novices is likely linked to the lack of prior EEG or qEEG experience and, potentially, a bias related to being overly cautious about missing seizure events. Additionally, providing multiple trends might have resulted in an information overload for qEEG novices even after training. This highlights the importance of ensuring adequate training and ongoing refresher sessions to critical care providers when considering a critical-care-provider-led qEEG-based seizure screening.

Our study has several limitations. Firstly, our sensitivity metrics could be an overestimate. There was no limit imposed on the time that qEEG reviewers spent analyzing the trends. At the bedside, critical care providers have competing priorities, and this may limit the time available for qEEG review. In addition, at the bedside, clinical problems that distract qEEG reviewers could negatively impact the sensitivity for seizure identification. Secondly, we only used a 24-h EEG sample from each patient’s prolonged recording because most often critical care providers are concerned about the last 12–24 h of EEG recordings to determine appropriate titration of antiseizure medications in patients with SRSE. Although we selected these epochs randomly to mitigate selection bias, the performance could be different when reviewers evaluate qEEG trends derived from several days of recording. Third, in a complex SRSE data set, many EEG patterns can be on the IIC. To solve this issue, we incorporated definite and probable seizure classification in addition to reporting other commonly seen ICU patterns. We found that qEEG experts reported higher seizure burden when EEG patterns were generalized and had IIC and BS patterns. Lastly, the brief training module without a before and after test to formally evaluate the knowledge gained and competence of the qEEG novices is a limitation of our study design.

Conclusions

In summary, this study supports the use of qEEG to identify seizures and quantify seizure burden in patients with SRSE. More studies are needed in the future to evaluate the characteristics of other commonly seen patterns in the ICU on qEEG in addition to the performance of the individual trends and automated seizure detection algorithms [43]. Programs implementing qEEG-based seizure screening could consider building in hands-on qEEG experience for the critical care providers to allow feedback-based reduction in false positivity.