Automated estimation of frequency and spatial extent of periodic and rhythmic epileptiform activity from continuous electroencephalography data

Abstract

Objective.

Rhythmic and periodic patterns (RPPs) are harmful brain activity observed on electroencephalography (EEG) recordings of critically ill patients. This work describes automatic methods for detection of the frequency and spatial extent of specific RPPs: lateralized and generalized rhythmic delta activity (LRDA, GRDA) and lateralized and generalized periodic discharges (LPD, GPD).

Approach.

The frequency and spatial extent of RPPs is estimated using signal processing and rule-based logic. Three algorithm variants based on fast Fourier transform (FFT) and Hilbert–Huang transforms (HHT) were developed for rhythmic delta activity, and three using derivative and time-based peak detection for periodic discharges. Annotations from three expert neurophysiologists served as the gold standard, and inter-rater reliability (IRR) and mean absolute error (MAE) were used to assess performance.

Main results.

We evaluated the algorithms on segments with 100% agreement on event classification $(n=389)$ and on the full cohort of 1087 segments (including disagreements). For the first subset, top algorithms matched or exceeded expert agreement for RPP frequency/spatial extent. RDA1b-FFT, the best algorithm for rhythmic delta activity, showed an expert-algorithm IRR of good to excellent with an intra-class correlation coefficient (ICC) of 91% and 96% (MAE 0.13 Hz and 0.26 Hz) frequency, and ICCs of 85% and 66% (MAE 0.19 and 0.09) for spatial extent for LRDA and GRDA. For periodic discharges, PD2a, showed and expert-algorithm IRR ICC of 80% and 61% (MAE 0.41 Hz and 0.15 Hz) for frequency, and ICC 77% and 13% (MAE 0.17 and 0.40) for spatial extent of LPD and GPD. For the full cohort, IRR declined, but expert-algorithm IRR remained comparable or superior to experts.

Significance.

The presence of RPPs at high frequencies and spatial extent are associated with a higher probability of poor outcomes. The proposed algorithms for estimating frequency and spatial extent of RPPs match expert performance and are a viable tool for large-scale EEG analysis.

1. Introduction

The last two decades have seen a significant increase in continuous electroencephalography (cEEG) utilization in the critical care setting [1]. cEEG is used for detection of seizures and other seizure-like rhythmic and periodic epileptiform patterns (RPPs) in at risk critically-ill and comatose patients [2, 3]. In addition to seizure detection, cEEG is also used for prognostication in patients with severe brain injuries and for detection of cerebral ischemia [4–6].

Up to half of critically-ill patients undergoing cEEG monitoring are found to have seizure-like RPPs that do not fall under the definition of seizures [7]. The American Clinical Neurophysiology Society (ACNS) proposed terminology to harmonize the interpretation of RPPs for clinical care [8]. Periodic discharges are defined as repetitive waveforms with clear inter-discharge background activity occurring at regular intervals. Rhythmic delta activity is characterized by repetitive slower waves between 0.5–4 Hz. ACNS further characterizes these patterns as lateralized or generalized periodic discharges (LPDs, GPDs) and lateralized or generalized rhythmic delta activity (LRDA, GRDA), and the ictal-interictal continuum (IIC). The IIC is defined as a combination of LPDs, GPDs or LRDA observed in 10 s of EEG recordings. Figure 1 shows examples of lateralized and generalized periodic and rhythmic activity.

Figure 1.

Examples of periodic and rhythmic patterns. (A) Lateralized rhythmic delta activity (LRDA) with a blue arrow highlighting an example of an event. (B) Generalized rhythmic delta activity (GRDA). (C) Lateralized periodic discharges (LPD) with a blue arrow highlighting and example of an event. (D) Generalized periodic discharges (GPD).

RPPs and the IIC have been directly linked to higher likelihood of mortality and worse functional and cognitive outcomes in patients with acute brain injuries such as stroke and trauma [9–11]. In particular, higher frequency and spatial extent of these patterns are linked to worse outcomes [3]. The frequency of RPPs and spatial extent of brain regions involved have been associated with a monotonic increase in metabolic activity as demonstrated by functional imaging and cerebral microdialysis analysis [12]. Several studies have shown that presence of LPDs, particularly at frequencies higher than 2 Hz, is linked to an increased risk of seizures [13, 14] and increased mortality [15]. GPDs have shown an equivalent seizure risk to LPDs when present at frequencies higher than 1.5 Hz [2]. LRDA has a similar frequency-dependent relation with seizures and worse outcomes [16], while GRDAs are the least harmful of the patterns with no association to seizure risks [13]. Hence both the frequency and spread of the pattern (lateralized or generalized) may be helpful in identifying patients at risk of a poorer outcome [17].

While RPPs and the IIC have been linked to worse outcomes, there is limited data to guide anti-seizure treatment approaches [2]. One of the major limitations in pursuing large-scale clinical research studies to examine the impact of anti-seizure treatment is the labor-intensive task of manual review of hours of EEG signals collected in critical care settings. Furthermore, there are still controversies related to best treatment approaches, and their link to IIC patterns and outcomes [2]. Automatic methods of identifying RPPs and IIC patterns and estimating their frequency and spatial extent, can help with clinical patient monitoring as well as conducting research studies on large patient populations to identify links between treatments and outcomes.

Prior work on the automatic annotation of EEG signals has focused primarily on classification of RPPs and IIC patterns [18–21]. Several other works have examined the presence or frequency of RPPs but have been limited by sample sizes or restricted only to one pattern type and have not quantified spatial extent [18, 20, 22–24]. This work aims to advance state-of-the-art algorithms for frequency and spatial extent estimation of RPPs and IIC patterns by (i) proposing algorithms to calculate the frequency and spatial extent of LRDA, GRDA, LPD and GPD EEG segments, (ii) comparing agreement between the algorithm and expert EEG readers with the inter-rater agreement between experts in estimating the frequency and spatial extent of epileptiform patterns, (iii) proposing a validation framework for frequency and spatial extent estimations.

2. Methods

2.1. Dataset and annotation protocol

This study was conducted under protocols approved by the institutional review boards of Massachusetts General Hospital (protocols #2023P000487, #2024P002630) and Beth Israel Deaconess Medical Center (protocols #2022P000481, #2022P000417), which waived the requirement for informed consent for this retrospective analysis. We used a dataset comprised of 10 s EEG recordings previously labeled as LPDs, GPDs, LRDAs, and GRDAs by 30 clinical experts trained in neurophysiology as described in Jing et al [25]. From the dataset, EEG segments were selected for further analysis of frequency and spatial extent based on degree of consensus on pattern classification. We selected segments where majority of annotators agreed on a single pattern type. A total of 296 LPD, 296 GPD, 210 LRDA and 285 GRDA segments were selected for further annotation of frequency and spatial extent. Each segment contained 19 raw EEG channels sampled at 200 Hz. Annotations were conducted on 18 differential channels re-referenced to a longitudinal bipolar montage. Three clinical neurophysiologists (SFZ, PH, LB) performed independent annotations of the pattern type, classification, frequency of the event and spatial extent. The participants were instructed to annotated the segments using ACNS terminology [8] as they would implement in their regular clinical practice.

The labeling of the selected 10 s EEG segments was performed using a custom interface that allowed annotators to enter the frequency of the event and spatial extent (see figure A1 from the supplemental material). The reported frequency of event was the mean value observed within the 10 s segment. Spatial extent was reported as the number of channels that have events (0—no channels showed an event, 1—all 18 channels showed the event). If an annotator deemed there was no periodic or rhythmic pattern classification, they were instructed to add a frequency and spatial extent of 0.

2.2. Frequency and spatial extent detection

An overview of the event detection algorithm is provided in figure 2. The initial 10 s segment of the EEG recording was re-referenced to a longitudinal bipolar montage. A total of 18 differential channels are available. Preprocessing was performed by applying a 60 Hz notch filter and a 0.5–40 Hz bandpass filter [26]. Different algorithms were used for RDA and PD events as the information in these phenomena is encoded differently in the recorded EEG signals. RDAs are predominantly characterized by the frequency content of the repetitive waveforms, while PDs are characterized by morphology of the discharges in addition to their repetitive nature. Thus, we base methods for detecting frequency and spatial extent for RDAs on frequency domain transformations, whereas for PDs we base our algorithms on time domain signal processing. For spatial extent, each channel was analyzed individually. Several methods for RDA and PD detection are proposed and evaluated:

• RDA 1a–b—Fourier transform based

• RDA 2—Hilbert Huang transform based

• PD 1—Prior detector—McGraw et al [22]

• PD 2a–b—Derivative peak detection based

These algorithms are detailed in the following sections. The presence or absence of the event along with the corresponding frequency were obtained for each of the differential channels. The output of the algorithm was the median frequency of event across all channels, the spatial extent as a ratio between the number of channels with an event and the total number of channels, and the spatial areas defined based on the channels that show an event.

Figure 2.

Overview of the methods applied for detecting frequency and spatial extent of epileptiform activity. Detection of PDs and RDAs are approached differently through several algorithms. PD—periodic discharges, RDA—rhythmic delta activity, FFT—fast Fourier transform, HHT—Hilbert Huang transform.

2.2.1. RDA detection

Three frequency-based methods were explored for detection of rhythmic delta activity. Schematics of the developed algorithms are illustrated in figure 3. The first two, RDA 1a and RDA 1b, are variations based on the Fourier Transform of the EEG, while the third RDA 2 is based on the Hilbert–Huang transform (HHT).

Figure 3.

Schematics of the three algorithm variations for the detection of rhythmic delta activity frequency and spatial extent. (A), (B). Detection methods based on the Fourier transform; (C). Detection method based on the Hilbert Huang transform. RDA—rhythmic delta activity, FFT—fast Fourier transform, HHT—Hilbert–Huang transform, FOOOF—fitting oscillations and one over f, PKs—adjusted power of peaks, F—center frequency, BW—peak bandwidth, IMF—intrinsic mode function.

2.2.1.1. RDA1—Fourier transform based

The power spectral density (PSD) of the 10 s EEG segment was calculated based on the Fourier transform:

$$X[k]=\sum _{n=0}^{N-1}x[n]{\text{e}}^{-j2\pi \frac{kn}{N}},k=0,1,\dots ,N-1;$$ (1)

$$P=\frac{2}{F_{\text{s}}N}|X[k]{|}^2,1\le k\le \frac{N}{2}$$ (2)

where $P$ is the PSD, $F_{\text{s}}$ is the sampling frequency, $N$ the signal length, $X[k]$ the Fourier transform of the discrete $x[n],x[n]$ the input EEG signal, $n$ the number of samples, the frequency components.

The PSD of the EEG is characterized by both periodic and aperiodic components [27]. In case of RDAs, the aim is to detect periodic components represented as narrow band peaks in the 0.5–4 Hz range. The power spectrum of EEG data at low frequencies is characterized by a high power aperiodic (1/f like) component that overlaps with the target delta peaks and can compromise their detection. To address this challenge, a model fitting the aperiodic component as a Lorentzian function and the periodic components with Gaussians (fooof) as described by Donoghue et al [28] is applied:

$$P=L+\sum _{m=0}^M{G}_m$$ (3)

$$L=b-\text{log}\left(k+F^{\chi }\right)$$ (4)

$$G=a\times \text{exp}\left(\frac{-{\left(F-F_{\text{c}}\right)}^2}{2w^2}\right)$$ (5)

where $L$ is the aperiodic component, $G$ represents the Gaussian function and $M$ the total number of Gaussians, $b$ the broadband offset, $\chi$ the exponent, $k$ the parameter controlling the bend of the aperiodic component, $F$ the vector of frequencies, $a$ is the peak power, $F_{\text{c}}$ is the center frequency and $w$ is the standard deviation of the Gaussian.

The output of the model provides the central frequency $\left(F_{\text{c}}\right)$, the aperiodic-adjusted power (PK) and the bandwidth (BW) of detected peaks.

RDA1a-fast Fourier transform (FFT).

The first RDA detection algorithm is described in figure 2(a). It takes the output of the fooof model and selects narrow band peaks by considering the condition BW ≤ 0.5. If no peaks are detected or selected by applying the narrow-band condition, no RDA event is detected. If one or more peaks meet the condition, the frequency of event is considered as the mean central frequency of the detected peaks.

RDA1b-FFT.

The second RDA detection algorithm is a variation of the first and is described in figure 2(b). Prior to applying the Fourier Transform, a channel selection is applied by identifying channels with invalid EEG data. Channels with corrupt EEG data are common in clinical recordings and can be caused by a variety of reasons such as motion artifacts, increased skin-electrode contact impedance or wire disconnections. The following condition is applied to identify channels with valid data:

$$x[n]=\left\{\begin{array}{c}x[n],{\sigma }^2\gg \left({\text{max}}_{n}x[n]-{\text{min}}_{n}x[n]\right)\\ 0,{\sigma }^2\ll \left({\text{max}}_{n}x[n]-{\text{min}}_{n}x[n]\right)\end{array}\right.$$ (6)

$${\sigma }^2=\frac{1}{N}\sum _{n=0}^N(x[n]-\mu {)}^2$$ (7)

where ${\sigma }^2$ is the variance of the signal, ${\text{max}}_{n}x[n]-{\text{min}}_{n}x[n]$ is the range of the signal, $N$ is the signal length and $\mu$ is the mean.

If the variance of the signal was not significant compared to the signal range, we conclude that the channel does not contain a valid EEG signal. If the channel did not contain valid EEG data, we conclude that no RDA was present. Channels with valid EEG data were further processed using fooof modeling as described previously. When selecting narrow band peaks representatives of RDA, selection was enhanced by considering the prominence of the peak with respect to the mean spectrum in the band of interest as an additional control.

For each detected peak, $P_k$, the band of interest was defined as:

$$B_{ik}=\left\{F_i\in F||F_i-F_{\text{c}}|<\frac{\text{BW}}{2}\right\},$$ (8)

while the baseline frequency band is considered:

$$B_{ok}=\left\{F_o\in F\mid 0.5\le F_o\le 4\text{and}{F}_o\notin B_{ik}\right\}.$$ (9)

A peak is selected when:

$${\mu }_{ik}>{\mu }_{ok}$$ (10)

where ${\mu }_{ik}$ and ${\mu }_{ok}$ is the mean PSD for the peak band of interest and baseline. If one or more peaks meet the conditions applied, the frequency of event was considered as the mean central frequency of the selected peaks.

2.2.1.2. RDA2—HHT based

The HHT is an effective tool for analyzing the time-frequency content of non-stationary and nonlinear data [29]. It has been successfully applied in the analysis of EEG data for a variety of tasks including movement identification [30], evaluating connectivity metrics [31] and mental imagery tasks [32]. The HHT is composed of two parts: empirical mode decomposition (EMD) and Hilbert spectral analysis (HSA). By applying EMD to the original signal $x[n]$, a set of sub-signals $c_i[n]$ called intrinsic mode functions (IMF) are obtained. The IMFs represent an adaptive orthogonal basis for the signal which can be described as:

$$x[n]=\sum _{i=0}^p{c}_i[n]+r[n]$$ (11)

where $r[n]$ is a residual component representing a trend or a constant. All IMFs satisfy two conditions: (i) the number of extrema and zero-crossings can differ by maximum one, (ii) the mean value of the upper and lower envelopes must be zero. The Hilbert transform (HT) is applied to all IMFs to obtain their instantaneous frequencies [33]. The HT of the IMFs is calculated as follows:

$${\widehat{C}}_i[n]=\sum _{\begin{array}{c}m=0\\ m\ne n\end{array}}^{N-1}\frac{c_i[m]}{n-m}.$$ (12)

With the HT of the IMFs, the instantaneous amplitude and frequency can be obtained from the analytical signal by combining ${\widehat{C}}_i[n]$ and the original signal:

$${\tilde{C}}_i[n]=c_i[n]+j{\widehat{C}}_i[n].$$ (13)

The module of ${\tilde{C}}_i$ is the instantaneous amplitude:

$$a_i[n]=\left|{\tilde{C}}_i[n]\right|.$$ (14)

The instantaneous frequency can be obtained by deriving the phase of the analytical signal:

$$f_i[n]=\frac{\Delta \text{arctan}\left(\frac{{\widehat{C}}_i[n]}{c_i[n]}\right)}{2\pi \Delta n}.$$ (15)

The Hilbert Spectrum is characterized by both the instantaneous amplitude and frequency functions $\left[a_i[n],f_i[n]\right]$.

RDA2-HHT.

The third RDA detection algorithm is described in figure 2(c). Valid EEG data was identified on the channel by applying the conditions identified in equation (6). If there was corrupt EEG data, the channel was marked as having no RDA. If the data was valid, the HHT was applied by calculating the IMFs through EMD and obtaining the instantaneous frequencies through the HT. Not all information contained in the HSA is relevant to the detection of RDA components. Previous methods focused the selection of IMFs based on statistical properties, periodicity degree or information shared with the original signal [34, 35]. A series of conditions were applied to select relevant IMFs based on the statistical properties of their amplitude and instantaneous frequency content, as well as the proportion of the IMF amplitude with respect to the original signal:

• $0.5\text{Hz}\le {\mu }_{f_{\text{i}}}\le 4\text{Hz}$, where ${\mu }_{f_{\text{i}}}$ is the mean of the instantaneous frequency of $c_{\text{i}}$

• ${\mu }_{f_{\text{i}}}\ge 2{\sigma }_{f_{\text{i}}}$, where ${\sigma }_{f_{\text{i}}}$ is the standard deviation of the instantaneous frequency of $c_{\text{i}}$

• $a_{c_{\text{i}}}>2{\sigma }_{c_{\text{i}}}$, where $a_{c_{\text{i}}}$ is the maximum amplitude of $c_{\text{i}}$, and ${\sigma }_{c_{\text{i}}}$ its standard deviation

• $\text{max}{a}_{c_{\text{i}}}>0.25\text{max}|x|$

If all the conditions were valid, the IMF is selected. The frequency of the RDA event was considered the mean of the instantaneous frequencies of all the selected IMFs.

2.2.2. PD detection

The detection of periodic discharges was approached through the identification of events in the time domain. We made use of the algorithm presented by McGraw et al [22] and integrated it into our event detection framework. Additionally, we proposed a new detection method by shifting the event detection to the first derivative of the EEG signal and applying two different peak detectors.

2.2.2.1. PD 1—prior detector

McGraw et al [22] proposed an algorithm for LPD and GPD frequency of event and spatial extent detection. After pre-processing, a high-low percentile thresholding is applied on the time domain EEG signal to select peaks that are potential PDs. Fixed thresholds on the amplitude and absolute values are applied on the identified peaks to filter out false detections. Once peaks are identified for the entire recording, the spatial extent is calculated based on the global or local energy of the detected events and reported as the ratio with respect to the total number of channels. The frequency of the event is reported as the median inter-discharge interval frequency at the channel group level.

2.2.2.2. PD 2—derivative peak detection based

With the goal of reducing false detection rates, we developed an alternate PD detector with the initial event identification based on the first derivative of the EEG signal. An overview of the proposed detection method is available in figure 4.

Figure 4.

Schematic of the derivative-based PD (periodic discharges) detector algorithms. Two alternative algorithms are used for peak detection based on the signal derivative.

Periodic discharges are often characterized by abrupt changes in the amplitude of the EEG signal. The derivative of a signal captures the rate of change in the original signal. Slopes in the original signal correspond to peaks of the first derivative. A higher rate of change in the original signal is reflected as a high amplitude peak in the first derivative. Thus, the detection of periodic discharges can be performed by detecting corresponding peaks in the first derivative of the EEG signal.

The detection of peaks is essential in processing a variety of physiological recordings [36], however peak detection is challenging due to background noise and interference. To eliminate potential sources of interference and isolate the PD events from other EEG phenomena, several pre-processing steps were applied. First, the channel was evaluated for containing valid EEG data as described previously in equation (6). Next, the signal was detrended by subtracting the mean and smoothed by applying a Savitsky–Golay 2nd order polynomial filter on a 10-sample window [37]. Afterwards the first derivative of the EEG signal was computed. Two methods for peak detection were evaluated.

PD2a.

The first makes use of local adaptive thresholds based on the statistics of the first derivative signal. A peak was identified when a local maximum is above a locally defined statistical based threshold within a 1 s sliding window as defined below:

$$\begin{gathered} p_{n_0}=\underset{n_0\in \left[n_0+1,n_0-1\right]}{\text{max}}x\left[n_0\right]\text{and}x\left[n_0\right]>\mu [n]+3\sigma [n], \\ \text{where}n\in [n-0.5\text{s},n+0.5\text{s}] \end{gathered}$$ (16)

PD2b.

The second peak detector explores the use of a $z$-score signal with peaks identified on a locally derived threshold:

$$\begin{gathered} p_{n_0}=\underset{n_0\in \left[n_0+1,n_0-1\right]}{\text{max}}z\left[n_0\right]\text{and}z\left[n_0\right]>{\mu }_z[n]+3{\sigma }_z[n], \\ \text{where}n\in [n-0.5\text{s},n+0.5\text{s}] \end{gathered}$$ (17)

$$z\left[n_0\right]=\frac{x\left[n_0\right]-\mu [n]}{\sigma [n]}$$ (18)

where $z$ is the $z$-score of the signal in the defined 1 s window, ${\mu }_z$ is the mean of the z-score signal, ${\sigma }_z$ is the standard deviation of the $z$-score signal.

After applying either of the peak detectors, the peaks identified were further filtered based on the length of the intervals between the peaks.

$$I_{k-1}=p_k-p_{k-1},k=0,1,2\dots K$$ (19)

where $I$ is the inter-peak interval, $p_k$ is the timing of peak $k$ and $K$ is the total number of peaks detected. A channel is considered to have PD events, if:

• More than one inter-peak interval is identified: $K>2$

• The detected inter-peak intervals are uniformly spaced with ${\sigma }_I<1$, where ${\sigma }_I$ is the standard deviation of the $I$ vector

The inter-peak intervals were divided by the sampling frequency and the average was reported as the frequency of the event.

2.3. Evaluation method

The gold standard for assessing frequency and spatial extent of LRDA, GRDA, LPD and GPD epileptiform activity were the manual annotations of EEG recordings by experienced clinicians. We evaluated the output of the proposed detection methods by comparing to manual annotations of the frequency and spatial extent of the EEG segments by experts.

We examined the inter-rater reliability between experts (ee-IRR) with the agreement between the algorithms and experts (ea-IRR). The annotations for the frequency and spatial extent of events as well as the output of the algorithms were evaluated through the intra-class correlation coefficient (ICC) [38]. As each EEG segment was rated by all three annotators, the following formula was applied to calculate the ICC:

$$\text{ICC}(3,a)=\frac{\text{BMS}-\text{EMS}}{\text{BMS}+(a-1)\text{EMS}}$$ (20)

where $a$ is the number of raters, BMS represents the between rated segment mean squared and EMS the error mean square from a two-way ANOVA. The numerator represents the variance between rated segments, while the denominator the variance due to each annotator.

Additionally, the continuous frequency and spatial extent data was categorized to allow reporting of mean percentage agreement (PA), as defined by the formula:

$$\text{PA}=\frac{1}{a}\sum \frac{N_{a_i=a_j}}{N}*100$$ (21)

where $a$ is the number of raters, $N$ the total number of annotated segments, $N_{a_i=a_j}$ the number of segments where annotators $a_i$ and $a_j$ agree on the annotation category. The categorization was performed as follows:

• Frequency of event intervals: <1 Hz, 1–1.5 Hz, 1.5–2 Hz, 2–2.5 Hz, 2.5–3 Hz, >3 Hz

• Spatial extent intervals: 1–4 channels, 5–10 channels, 11–14 channels, 15–18 channels

Inter-rater agreement for both ICC and PA were quantified as excellent if the values are 75% - 100%, good for 60% – 74%, fair for 40% – 59% and poor for values < 40% [39].

The output of the algorithms was further evaluated by calculating the mean absolute error (MAE) between the annotated values and the calculated ones:

$$\text{MAE}=\frac{1}{N}\sum _{i=1}^N\left|y_i-\widehat{y_i}\right|$$ (22)

where $N$ is the total number of segments, $y_i$ is the annotated values, $\widehat{y_i}$ is the output of the algorithms.

As the primary goal of the algorithms is to determine the frequency and spatial extent of patterns that are already classified in the EEG segment, we have performed our primary IRR analysis on the segments with 100% agreement among the annotators on the classification of the event. For completeness and as a sensitivity analysis, we also performed IRR analysis on the full dataset which included the ambiguous segments (i.e. segments where all three raters did not agree on pattern classification). In addition, the MAE between the mean value reported by the annotators and the outputs of each algorithm was evaluated on the dataset, including segments with rater disagreement on classification.

3. Results

The IRR among the three annotators for the pattern classification was moderate with a mean percent agreement of 48% for LRDA, 60% for GRDA, 63% for LPD and 58% for GPD. This level of agreement is consistent with prior work that demonstrates moderate agreement on pattern classification among expert neurophysiologists [25].

Out of the original segment set, a subset of 50 LRDA, 119 GRDA, 111 LPD and 109 GPD segments showed 100% agreement on pattern classification.

3.1. IRR between annotators and proposed algorithm for segments with 100% agreement on pattern classification

The results for the ICC and mean PA between the annotators and the proposed algorithms with 100% agreement of the annotators on the pattern classification are available in figure 5 for RDA and figure 6 for PD.

Figure 5.

Intra-class correlation coefficient (ICC) and mean pair-wise agreement (PA) between annotators and algorithms for LRDA and GRDA segments. The star indicates the best performing algorithm with an ea-IRR (expert-algorithm inter-rater reliability) that is higher when compared to ee-IRR (expert-expert inter-rater reliability). (A) ICC for frequency of event. (B) PA for frequency of event. (C) ICC for spatial extent. (D) PA for spatial extent.

Figure 6.

Intra-class correlation coefficient (ICC) and mean pair-wise agreement (PA) between annotators and algorithms for LPD and GPD segments. The star mark indicates the best performing algorithm with an ea-IRR (expert-algorithm inter-rater reliability) that is comparable or better than ee-IRR (expert-expert inter-rater reliability). (A) ICC for frequency of event. (B) PA for frequency of event. (C) ICC for spatial extent. (D) PA for spatial extent.

The IRR analysis shows that the output of the RDA1b-FFT algorithm is comparable to that of human annotators. The ea-IRR ICC for RDA1b-FFT, was 91% [CI 85,94] and 96% [CI 94,97] for LRDA and GRDA respectively. The mean pair-wise agreement for LRDA was 60.66% [47,72]. RDA1b-FFT scored lower for GRDA with a difference of 0.3% from RDA1a-FFT. The highest algorithm ea-IRR for spatial extent was also obtained by RDA1b-FFT with an ICC 85% [CI 77,91] and 66% [CI 46,71] for LRDA and GRDA respectively, and a mean pair-wise agreement of 53% [CI 39,64] for LRDA. Spatial extent for GRDA was best for RDA2-HHT, ~2% higher than RDA1b-FFT.

For calculating the frequency and spatial extent of PDs, the PD2a algorithm showed the highest performance within the same range as the performance of human annotators. The ea-IRR ICC for frequency of LPD and GPD segments, for algorithm PD2a was 80% [CI 73,85] and 61% [CI 47,72] and a mean pair-wise agreement of 54.95% [CI 45,63] and 72.78% [CI 74,88] for LPD and GPD respectively. The calculated spatial extent also showed the highest ea-IRR for PD2a with an ICC of 77% [CI 70,84] and 13% [CI 17,37] for LPDs and GPDs respectively, with a percentage agreement of 43.20% [CI 34,52]. A detailed overview of the IRR analysis is available in table B1 from the supplementary materials.

3.2. IRR between annotators and proposed algorithm for full dataset

As sensitivity analysis we analyzed the full dataset, including the ambiguous segments where annotators did not have 100% agreement on pattern classification. While, both the ee-IRR and ea-IRR dropped, performance of the algorithm performance with respect to the golden standard was comparable to when analysis was restricted to segments with 100% agreement. Table B2 from the supplementary materials shows the IRR analysis on the full dataset. RDA1b-FFT showed the highest ea-IRR for the frequency of event detection with 69% ICC [CI 62, 76] and 44% PA [CI 38, 50] for LRDA and 72% ICC [CI 67,78] and 67% PA [CI 61,72] for GRDA, as compared to the ee-IRR of 58% ICC [CI 48, 67] and 43% PA [CI 36, 49] for LRDA and 55% ICC [CI 45, 64] and 65% PA [CI 60, 71]. PD2a showed the best performance for the frequency of event with an ea-IRR 68% ICC [CI 62,74] and 60% PA [CI 55, 66] for LPD and 28% ICC [CI 0.3,33] and 77% PA [CI 71, 81], when compared to the other algorithms’ results and the baseline ee-IRR of 69% ICC [CI 62,75] and 63% PA [CI 57, 69] for LPD and 37% ICC [CI 24, 49] and 79% PA [CI 74, 84] for GPD. The trends for spatial extent performance followed a similar trend on the full dataset as on the subset with 100% agreement on classification.

A comparison of the individual data outputs for all EEG segments and the three annotators is available in Appendix C of the supplemental materials. In supplemental figures C1–C4 we show the relation of raw pair-wise agreement of the annotators and algorithms, as well as scatter plots of the annotated values for frequency and spatial extent. While the plots show a significant variation in the comparison of the algorithms’ output and the various annotators, the reported R2 is consistently higher for RDA1b-FFT with respect to all annotators and in close range to the annotator-annotator R2 when compared to the values for RDA1a-FFT and RDA2-HHT. Similarly, PD2a shows a consistently higher R2 which aligns with the performance reported via ICC and PA.

3.3. Evaluation of MAE

The MAE between the mean value reported by the annotators and the outputs of each algorithm are shown in table 1. The MAE for the entire dataset, including segments with rater disagreement on classification is available in table B3 of the supplemental materials. RDA1b-FFT and RDA2-FFT had the lowest MAE (0.13 Hz) for LRDA frequency. RDA1a-FFT had the lowest MAE (0.24 Hz) for GRDA frequency. RDA2-HHT had the lowest MAE (0.14) for LRDA spatial extent and RDA1b-FFT had lowest error (0.09) for GRDA spatial extent. PD2a had the lowest MAE for LPD (0.41 Hz) and GPD (0.15 Hz) frequency, as well as the lowest error for LPD spatial extent (0.17). PD2b had the lowest MAE for GPD (0.01) spatial extent. Examples of the output from the best performing algorithms RDA1b-FFT and PD2a are shown in figure 7. The left side displays the filtered EEG signals, with detected RDA channels marked in blue and detected PD events marked in red. The right side of the panel details the estimated frequency of event, spatial extent and spatial areas.

Table 1.

MAE (mean absolute error) between the mean human expert annotated value and the algorithms for RDA (rhythmic delta activity) and PD (periodic discharges), along with their 95% CI (confidence interval). The lowest values for each event type are highlighted.

	Algorithm	LRDA	GRDA	Algorithm	LPD	GPD
Frequency of event [Hz]	RDA1a-FFT	0.18 [0.12,0.24]	0.24 [0.19, 0.30]	PD1	1.18 [1.09, 1.36]	0.98 [0.86, 1.11]
	RDA1b-FFT	0.13 [0.09, 0.17]	0.26 [0.21, 0.32]	PD2a	0.41 [0.38, 0.45]	0.15 [0.13, 0.18]
	RDA2-HHT	0.13 [0.09, 0.16]	0.46 [0.39, 0.52]	PD2b	0.57 [0.50, 0.63]	1.25 [1.20, 1.29]
Spatial extent	RDA1a-FFT	0.25 [0.22, 0.30]	0.32 [0.28, 0.36]	PD1	0.53 [0.50,0.56]	0.01 [0.01,0.02]
	RDA1b-FFT	0.19 [0.16, 0.21]	0.09 [0.07, 0.09]	PD2a	0.17 [0.15, 0.19]	0.40 [0.35,0.44]
	RDA2-HHT	0.14 [0.12, 0.17]	0.08 [0.06, 0.09]	PD2b	0.59 [0.58,0.60]	0.01 [0.01, 0.02]

Figure 7.

Output of the frequency of event and spatial extent. The EEG channels evaluated are displayed on the left, while the algorithm outputs are listed on the right. (A) PD2a—red colored EEG traces represent channels with detected periodic discharges (PD). (B) RDA1b-FFT—blue colored EEG traces represent channels detected by the algorithm as containing rhythmic delta activity (RDA).

4. Discussion

In this work we developed algorithms to estimate the frequency and spatial extent of epileptiform activity that perform comparably to human expert annotators. These algorithms can be applied to large neurophysiologic datasets for automated annotation of RPP frequency and spatial extent, after the classification of periodic and rhythmic patterns by existing validated specialized models [24].

The FFT-based methods designed for RDA segments scored similarly in terms of detection of event frequency, with little variation in corresponding ICC and PA. This is expected, as similar underlying frequency extraction methods are applied. Notably, RDA1a-FFT had lower performance in terms of spatial extent as compared to RDA1b-FFT that had an additional valid channel identifier. This underscores the importance of eliminating problematic EEG data and appropriately pre-processing the signals prior to applying detection algorithms. Across the frequency of event and spatial extent for rhythmic delta activity, the best performance was obtained by RDA1b-FFT. The RDA2-HHT had the lowest performance across the three algorithms. While the HHT is a powerful method for representing frequency content, the rule-based selection of IMF and HSA components may introduce variability in the output across different data, potentially leading to reduced performance.

In case of PD detection, the frequency estimation is dependent on correct identification of peaks in single EEG channels. PD2a achieved the highest performance in terms of ICC, PA and MAE. Applying peak detection to the first derivative of the EEG signal, rather than to the raw signal, led to improved event detection rates. PD2a utilized a locally computed threshold for peak identification, resulting in better performance compared to PD2b, which relied on a global threshold applied across the entire signal. Threshold-dependent detectors are highly sensitive to the empirically chosen threshold values. Depending on the characteristics of the data, this can lead to either over-detection of signal fluctuations or omission of relevant events.

The MAE relative to the average human-annotated values ranged between 0.13–0.41 Hz for the frequency of events detected by the best performing algorithms. These values fall within the estimation range reported by human-annotators, who typically use a maximum granularity of 0.25 Hz to report event frequency.

Overall, both the MAE and the ICC had greater variability for spatial extent with lower agreement both for the human annotators and for human-algorithm comparison. The MAE for the best performing algorithm ranged between 0.14–0.40 which translates to 3–7 channels out of 18 with missed detection. Spatial extent has shown an overall lower IRR for human annotators as well. The highest MAE of 0.40 for the spatial extent identified by PD2a was reported for the GPD segments. Although the algorithm performs poorly in detecting spatial extent for GPD segments, the spatial extent detection can be corrected by factoring in prior classification into generalized activity [24]. Handling of imbalanced data with skewed distributions is a well-known limitation of IRR metrics such as the ICC [40, 41]. The distribution of spatial extent annotation for generalized activity is skewed towards 1 as most EEG channels are expected to be involved. This leads to an unexpectedly low ICC, while the PA shows an almost perfect agreement (see figures 5 and 6).

The current state-of-the-art EEG annotating algorithms focus primarily on classification of RPPs and IIC patterns [18–21]. Jing et al [21] developed SPaRCNet—a deep neural network trained on more than 50 000 ten second EEG segments labeled by multiple neurophysiologists to identify seizures, LPDs, GPDs, LRDAs, GRDAs and data that falls into none of these classes. The performance of the model is in the same range as expert-level inter-rater agreement; however, it only indicates the classification of EEG segments into RPP and not their frequency or spatial extent. In this work, we have further expanded the characterization of these patterns in terms of frequency of events and spatial extent. Fürbass et al [18] proposed a rule- based peak detection algorithm for identification of RDAs and PDs. While their output also includes frequency and spatial information their validation focused only on 10 patient cases where only the classification of patterns was compared to the neurophysiology report for accuracy. Here we have expanded the validation work by incorporating more than 300 EEG segments in the evaluation of the algorithms’ outputs. McGraw et al [22] extended the identification of patterns with the automatic quantification of frequency and spatial extent of LPDs and GPDs while looking at changes in the energy of the time-domain signal with a sliding window. An agreement of above 90% was observed when three clinical experts evaluated the output of the quantification algorithm. While the overall agreement is high, our results and comparison show that when quantitatively evaluated against the frequency of the events and their spatial extent, the IRR metrics showed a reduced performance (figures 6(A)–(D)). By applying a peak detector on the first derivative instead of the time domain signal and including a valid channel selector, our proposed method PD2b improved the results based on the quantitative evaluation. Several other works [23, 24] propose methods for the detection of periodic discharges, but do not include LRDA and GRDA activity and do not evaluate frequency and spatial extent. In this work, we have added to the state-of-the-art by also developing algorithms that target the quantification of LRDA and GRDA patterns.

Our work has several limitations. The evaluation of algorithm performance was based on a limited number of LRDA, GRDA, LPD and GPD segments as we selected only those that had complete consensus on classification between the annotators. While this limits the number of segments, it allows greater accuracy in assessing the frequency and spatial extent. However, we demonstrate similar performance even when using the full dataset with classification disagreement as shown in table B2 of the supplemental material. The annotation process involved only three human experts from a single medical facility. Since manual annotations were considered the ground truth, a degree of subjectivity is inherent in the evaluation. While we focused on creating adaptive data-driven methods for event identification, all presented algorithms rely to a certain extent on rule-based selection of events which may require additional validation when applied to other datasets. Integrating frequency and spatial extent estimation with deep learning models may enhance algorithm performance and improve the generalizability of results. Fine-tunning pre-trained neural network models with the LRDA, GRDA, LPD, GPD frequency and spatial extent annotations may result in a model with better performance and broader applicability.

5. Conclusions

In this study, we proposed several methods for estimating the frequency and spatial extent of rhythmic and periodic epileptiform activity. Our findings indicate that RDA1b-FFT is the most effective for detecting frequency and spatial extent of rhythmic delta activity, while PD2a performs best for periodic discharges. Both methods demonstrated strong agreement with clinical neurophysiologists’ annotations, producing comparable estimates of event frequency and spatial extent.

Supplementary Material

Supplementary material for this article is available online

Data availability statement

The data that support the findings of this study will be openly available at the following URL/DOI: https://bdsp.io/about/database/.