Impact of standardized electroencephalography terminology education among early-career neurologists in Korea: a single-group pretest–posttest study

Abstract

Purpose

This study evaluated the effectiveness of implementing standardized electroencephalography (EEG) terminology and education on improving interpretation skills among non-English speakers.

Methods

EEGs were collected retrospectively from 16 critically ill patients and two alert epilepsy patients (30-minute sessions) and read by 10 neurologists. After initial interpretation, a learning module was provided to the neurologists. One week later, the neurologists reinterpreted the same EEGs randomly reordered. The interpretation accuracies of 144 items were scored as correct, similar, incorrect, or not recognized based on consensus answers.

Results

Overall scores improved significantly after learning modules were completed, increasing from 88.8 ± 23.1 to 118.2 ± 39.2 (p = 0.037). Performance improvement was also observed, regardless of years of clinical experience treating patients with epilepsy. However, more experienced raters tended to have higher inter-rater agreement. The baseline correct answer rates for rhythmic delta activity (RDA), periodic discharges (PD), sporadic epileptiform discharge (ED), and electrographic seizures (ES) were 42.5%, 50%, 60.0%, and 20.0%, respectively. After the learning modules were completed, correct answer rates increased by 21.4% for RDA (p = 0.11), 11.1% for PD (p = 0.06), and 28.1% for ES (p = 0.02), but decreased by 11.1% for sporadic ED (p = 0.02). Specifically, the accuracy for generalized PD improved significantly after learning (p = 0.04). Most minor items except location were often missed even after learning.

Conclusion

Education is imperative for standardizing EEG descriptions. As such, developing educational modules that emphasize minor items including prevalence, frequency, duration, and amplitude will broaden the recognition of EEG features, improving the quality of future research and patient care.

Introduction

Electroencephalography (EEG) is an essential tool for recording brain activity, being particularly valuable for the continuous monitoring of critically ill patients with impaired consciousness. Early recognition of EEG abnormalities is crucial for appropriate treatment, especially in acute neurological conditions such as brain injury [1,2]. Identifying subclinical seizures and related EEG findings is relevant for timely and appropriate intervention but remains suboptimal in clinical settings. Since residents and fellows are often responsible for EEG interpretation, education using standardized terminology is a critical component of comprehensive care.

The advent of artificial intelligence-driven approaches necessitates the creation of a well-annotated EEG database by experts. However, the subjective nature of some EEG findings complicates consensus-building. To address this issue, the American Clinical Neurophysiology Society (ACNS) developed and revised standard EEG terminology [3,4]. Given that EEG reading relies heavily on visual analysis by residents and fellows, identifying strengths and weaknesses in various aspects of ACNS terminology is crucial for skills enhancement.

Within the ACNS framework, not only are electrographic seizures (ES) significant, but other rhythmic and periodic patterns (RPP) are also highlighted. Detailed descriptions of amplitude, background activity, frequency, amount, and duration are imperative in education and patient care. However, due to clinical workload, these detailed assessments are often omitted in practice.

The advent of big data is transforming the artificial intelligence landscape, emphasizing the importance of data standardization for collaborative research [5]. In non-English speaking regions, familiarity with standard terminology can be relatively weak, posing challenges for international collaboration. Therefore, in this study, we investigate the impact of standardized EEG terminology education on Korean neurologists in training, with the goal of strengthening clinical education and identifying components that require improvement.

Methods

Study design

This study consisted of three steps: (1) collection of EEG datasets retrospectively from mainly critically ill patients; (2) interpretation of the EEGs in free text by 10 neurologists; and (3) assessment of changes in EEG reading performance following self-learning of standardized EEG terminology.

Data collection

A total of 10 raters, including three senior residents, two fellows, and five fellowship-trained neurologists, were assigned randomized numbers for anonymization in each session. The raters were allowed to freely describe EEG results without any restrictions. Only data regarding age, sex, and consciousness level, without detailed clinical information, were provided to raters. EEGs were de-identified and presented in European Data Format, each recorded as a referential montage with sensitivity of 7 μV/min, band-pass 1–70 Hz without any existing annotations. The raters were able to adjust the settings and montage freely. After the initial reading, all participants completed a self-learning module created by three experienced epileptologists (SBH, HL, and KIP), who were not among the rater participants. The learning period was freely set within 1 week. A second reading of the same EEGs was conducted after the self-learning phase, following the same procedure as the first reading, with the sequence of EEG sets randomized.

To develop the dataset, we collected portable EEG data, recorded with electrodes of 10–20 systems, each with a duration of 30 minutes, from critically ill patients, and an additional two EEG sets from patients with alert mentality to assess sporadic epileptiform discharge (ED) at Seoul National University Hospital from January 2022 to February 2023. Epileptologists (KIP, HL, and SBH) reviewed and selected data with minimal artifacts and relatively clear findings to determine key factors of EEG reading.

Outcome measurement

The items assessed include background activity, continuity, and the presence of highly epileptiform bursts in a burst-suppression (BS) pattern. The posterior dominant rhythm was assessed only in response to eye-opening and closing commands. The evaluation of RPP covered location, prevalence, duration, frequency, amplitude, dynamics, and plus/minor modifiers. For periodic discharge (PD), the sharpness of discharges or waves was also examined. Sporadic ED was evaluated based on location, prevalence, frequency, and amplitude. ES and electrographic status epilepticus (ESE) were evaluated strictly according to their definitions. Results were rated on a scale from 0 to 3 points. A correct answer received 3 points, while a similar answer, in which the rater recognized the finding but did not use standard terminology, received 2 points. An incorrect answer, where the finding was recognized but misinterpreted, received 1 point. Finally, in cases where the rater failed to recognize or document the findings, a score of 0 points was given. Ten neurologists completed baseline reading, learning, and after-reading. A correct answer was defined as one that aligned with the consensus of epileptologists (KIP and SKL) with over 20 years of experience of epilepsy.

All items assessed consisted of eight EEGs showing PD, four EEGs showing rhythmic delta activity (RDA), six EEGs showing ES or ESE, two rhythmic activities other than RDA, and three sporadic EDs.

We developed a self-learning module based on the standard guidelines of the ACNS. This module, created for educational purposes, comprises 114 PowerPoint slides (Microsoft Corp), encompassing both actual EEGs and illustrative examples.

The study protocol adhered to the ethical guidelines of the 1975 Declaration of Helsinki and its subsequent revisions and was approved by the Institutional Review Board of Seoul National University Hospital (No. 2403-010-1516). The requirement for informed consent was waived due to retrospective data collection.

Statistics

Values following a normal distribution are presented as mean ± standard deviation. Pairwise inter-rater and intra-rater agreement for each item was evaluated using Cohen’s kappa coefficient. The percentages of correct answers according to expert consensus before and after the learning intervention were assessed using the Wilcoxon signed-rank test. Statistical analyses were conducted using IBM SPSS version 29.0 (IBM Corp).

Results

In total, 144 meaningful items were selected from 18 EEGs. Total points increased significantly after self-learning, from 88.8 ± 23.1 to 118.2 ± 39.2 (p = 0.037) (Figure 1A).

Figure 1. Rating before and after learning and interrater agreement according to experience.

BS, burst suppression; PDR, posterior dominant rhythm; HEB, highly epileptiform burst; RPP, rhythmic periodic pattern; ES, electrographic seizure; ESE, electrographic status epilepticus; ED, epileptiform discharge.

*p < 0.05.

The majority of items showed trends of improvement. However, the continuity of BS, highly epileptiform bursts, amplitude or morphology of RPP, minor modifiers such as triphasic morphology, and the presence, prevalence, and amplitude of sporadic ED, did not improve after learning (Figure 1B). Statistically, BS (p = 0.01), ES/ESE (p = 0.04), and PD (p = 0.007) showed significant improvement. The RDA score tended to improve (p = 0.063).

The pairwise Cohen’s kappa values for inter-rater agreement for overall main items, including RDA, PD, ES, BS, other rhythmic activities, and sporadic ED, were 0.14 ± 0.24 pre-learning and 0.12 ± 0.25 post-learning, indicating slight agreement with no significant difference (p = 0.71). The intra-rater kappa values were 0.25 ± 0.22, suggesting fair agreement. More experienced raters tended to have higher inter-rater agreement (Figure 1C). Performance improvement was observed in all raters, regardless of the number of years of clinical experience with epilepsy, with one exception (Figure 1A).

The percentages of correct, similar, incorrect, and missed answers are shown in Figure 2A and Table 1. The baseline correct answer rates for RDA, PD, sporadic ED, and ES or ESE were 42.5%, 50%, 60.0%, and 20.0%, respectively. After learning a module, the correct answer rates increased by 21.4% for RDA (p = 0.11), 11.1% for PD (p = 0.06), and 28.1% for ES or ESE (p = 0.02), but decreased by 11.1% for sporadic ED (p = 0.02). The correct % of generalized PD (GPD) significantly improved after learning (p = 0.04) (Figure 2B).

Figure 2. Correct percentages and changes after learning for EEG parameters.

(A) Percentages of correct, similar, incorrect, and missed responses for each parameter, and changes after learning. (B) Changes in the percentages of correct interpretations for detailed parameters. The red and blue bars represent median values.

RDA, rhythmic delta activity; PD, periodic discharge; B, before; A, after; Lc, location; Pv, prevalence; Fq, frequency; Mo, morphology; Dr, duration; Am, amplitude; Dy, dynamics; HEB, highly epileptiform burst; ED, epileptiform discharge; ES, electrographic seizure; ESE, electrographic status epilepticus; GRDA, generalized rhythmic delta activity; LRDA, lateralized rhythmic delta activity; GPD, generalized periodic discharge; LPD, lateralized periodic discharge; BIPD, bilateral independent periodic discharge.

Table 1.

Correct percentages and changes after learning for electroencephalography parameters

Variable	Pre-training		Post-training
	Correct	Not noted	Correct	Not noted
Burst suppression	11 (55.0)	4 (20.0)	16 (88.9)	0 (0)
Continuity	0 (0)	18 (90.0)	1 (5.6)	17(94.4)
Highly epileptiform burst	3 (15.0)	12 (60.0)	3 (16.7)	14 (77.8)
Rhythmic delta activity	17 (42.5)	7 (17.5)	23 (63.9)	5 (13.9)
Location	15 (37.5)	12 (30.0)	24 (66.7)	8 (22.2)
Prevalence	3 (7.5)	14 (35.0)	12 (33.3)	16 (44.4)
Duration	0 (0)	39 (97.5)	1 (2.8)	35 (97.2)
Frequency	1 (2.5)	39 (97.5)	0 (0)	36 (100)
Amplitude	0 (0)	38 (95.0)	0 (0)	36 (100)
Dynamics	1 (10.0)	9 (90.0)	1 (11.1)	8 (88.9)
Periodic discharge	40 (50.0)	12 (15.0)	44 (61.1)	6 (8.3)
Location	43 (53.8)	9 (11.3)	41 (56.9)	11 (15.3)
Prevalence	7 (8.8)	44 (55.0)	21 (29.2)	36 (50.0)
Frequency	12 (15.0)	64 (80.0)	24 (33.3)	47 (65.3)
Morphology (sharpness)	19 (23.8)	55 (68.8)	12 (16.7)	58 (80.1)
Duration	0 (0)	79 (98.8)	0 (0)	71 (98.6)
Amplitude	2 (2.5)	73 (91.3)	0 (0)	72 (100)
Dynamics	12 (30.0)	26 (65.0)	12 (33.3)	21 (58.3)
Plus modifier	2 (5.0)	36 (90.0)	8 (22.2)	20 (55.6)
Minor modifier	6 (12.0)	14 (28.0)	3 (16.7)	15 (83.3)
Sporadic epileptiform discharge	18 (60.0)	9 (30.0)	16 (59.3)	11 (40.7)
Location	11 (36.7)	10 (33.3)	10 (37.0)	13 (48.2)
Prevalence	8 (26.7)	11 (36.7)	6 (22.2)	16 (59.3)
Amplitude	0 (0)	0 (0)	0 (0)	0 (0)
Other rhythmic activities	3 (15.0)	11 (55.0)	1 (5.6)	10 (55.6)
Location	13 (65.0)	6 (30.0)	13 (72.2)	5 (27.8)
Prevalence	0 (0)	11 (55.0)	3 (16.7)	10 (55.6)
Frequency	2 (10.0)	16 (80.0)	3 (16.7)	13 (72.2)
Morphology (sharpness)	2 (10.0)	18 (90.0)	1 (5.6)	17 (94.4)
Duration	0 (0)	20 (100)	0 (0)	17 (94.4)
Amplitude	0 (0)	20 (100)	0 (0)	20 (100)
Dynamics	6 (30.0)	14 (70.0)	7 (38.9)	11 (61.1)
Electrographic seizure or electrographic status epilepticus	12 (20.0)	32 (53.3)	26 (48.1)	21 (38.9)

Values are presented as number (%).

Inter-rater agreement showed no significant difference according to epilepsy experience with a cutoff of 5 years pre- and post-learning. The kappa values for main items consisting of four RDAs, eight PDs, six ESs, two BSs, two other rhythmic activities, and three sporadic EDs in pre-learning were 0.21 ± 0.22 for senior raters and 0.13 ± 0.25 for junior raters (p = 0.45). Post-learning, the kappa values were 0.11 ± 0.16 for senior raters and 0.12 ± 0.26 for junior raters (p = 0.90).

Discussion

In this study, we observed that ACNS guideline-based education significantly improved the recognition of key EEG features by Korean neurologists. The effect of education varied among different EEG features, indicating differential impacts of the training on various aspects of EEG interpretation. Notably, the identification of PD, particularly GPD, and ES showed significant improvement, while others did not.

Among the main features, we observed the highest correct identification rate in BS, followed by RDA, PD, and ES. BS is more likely to be recognized because it is continuously or nearly continuously observed as the background pattern, while patterns such as RDA or ES are transient with diverse durations within the whole EEG session. A previous study involving full EEGs from 103 post-cardiac arrest patients supports our finding of a kappa value of 0.71 for “highly malignant EEG patterns,” which includes BS [6]. The percentage agreement for highly malignant patterns was 75% among the four raters, while the RDA pattern showed 50%, somewhat comparable to our findings of 88.9% and around 60% of the correct rate, respectively.

We found that the correct rate of ES was 10.0% in pre-training and 48.1% in post-training, which is much lower than a prior study reporting a 70% detection rate [7]. This difference seems to arise from our inclusion of a broader range of EEG items than the previous study.

Our inter-rater agreements for the main items represented by the kappa value were lower than those reported in previous studies, likely due to the inclusion of many trainees and the variability of specific parameters [8]. Although the kappa value is an important metric for assessing inter-rater agreement, we did not present the kappa values for all items due to the limited number of test sets and the frequent omission of many minor items by the raters. Since kappa statistics only analyze answered items, it is difficult to provide meaningful results when there are few or no common responses. However, the strength of this study is that the design simulates real practice using nonselective, whole EEG segments.

Comparing ES and PD, the correct percentage was slightly higher for PD than ES in our study, in contrast with a previous study involving eight experts. In that study, ES was more readily identifiable than PD [9] with an identification rate of 81% for ES and 65% for PD. This discrepancy may have occurred because our participants were mostly epilepsy trainees. Another reason could be the requirement to assess various main and minor items, potentially distracting the raters. The previous study only required participants to determine whether the case was PD or seizure. Among the six sets showing ES, two sets had a correct percentage lower than 50%. Since the raters were required to provide free-text responses, when encountering PD or RDA, they needed to additionally recognize the duration, frequency, or evolution patterns to correctly identify ES in the same set. Many raters correctly identified RPP but did not recognize the additional components needed to fit the definition of ES. Since the Google Sheets information did not provide detailed clinical status, it is likely that the raters did not initially consider whether the cases were nonconvulsive seizures, leading to less attention to evolution and frequency, which differs somewhat from clinical practice.

Ten raters participated in the present study and rated 18 EEGs. A similar study including nine readers and 30 EEGs from 23 patients showed a kappa value of 0.5 for raters with 5 to 10 years of experience and 0.29 for neurology residents [10], higher than our value of 0.12 ± 0.25. In that study, EEGs were provided as 10-second epochs, which can be recognized more easily than full EEGs. Short segments showed much higher inter-rater agreement than full-length EEGs [8]. Another difference compared to our study was that the previous study used Young’s criteria for seizures, which are slightly different from the ACNS guidelines for ES regarding frequency and definition of evolution.

A recent large-scale study investigated the inter-rater reliability of various ictal and ictal-like patterns, encompassing LPD, GPD, LRDA, and generalized RDA (GRDA), presenting 50,697 ten-second segments [11]. It evaluated pairwise agreement and percent agreement compared to group consensus, the latter of which is similar to the correct percentage in our study.

Both short-segment and full-length EEGs were read in a single study to test inter- and intra-rater agreement using ACNS standardized terminology. The kappa value was 0.44 for rhythmic/periodic patterns [8]. Kappa values decreased in the minor parameters in the order of localization, morphology, persistence, duration, and frequency. The previous study showed 0.68 to 0.79 intra-rater agreement, which was much higher than in the present study, because it was calculated in a dichotomized fashion as rhythmic/periodic or ES/non-rhythmic. We calculated kappa values within cases when raters commented on those findings. One more reason for the higher kappa value in the previous study is that raters were required to choose only one pattern. Among minor descriptors, the frequency and prevalence of PD improved from 15.0% to 33.3% and from 8.8% to 29.2%.

A large-scale dataset [11] showed that the majority of inter-rater reliability, similar to our definition of correct percentage, was 71%, 65%, 55%, and 60% for LPD, GPD, LRDA, and GRDA, respectively, consistent with our findings of 61.1% and 63.9% for PD and RDA. The correct detection rate of ES in our study was lower than the 64% found in that study. This discrepancy suggests a need to refine the education module we used and strengthen the weak points of recognition.

We assessed minor items belonging to RDA. Mani et al. [12] used 10-second EEG epochs, finding high agreement regarding amplitude and frequency. Among plus modifiers, F (fast activity) and R (rhythmic activity) had higher agreement than S (stereotyped). While our study showed a 22.2% correct rate, the small sample size included limits the significance of our findings. We did not analyze each plus modifier separately due to the small number of cases assessed. The annotation of amplitude was missed by almost all raters. This can be explained by the fact that most EEG signals had moderate amplitude, but it was missed even in the high amplitude range, possibly because the learning module did not emphasize the amplitude component. Some cases were incorrectly interpreted due to the rater not knowing that amplitude was determined in the bipolar montage according to the ACNS guidelines, since the default montage in all cases was the referential montage.

We observed that the accuracy of reading EEGs using standard terminology is not dependent on years of epilepsy experience, consistent with previous findings [13].

Education based on updated guidelines is crucial after identifying which categories are effectively improved by training. We found that the recognition of PD and ES significantly improved with learning. The use of learning modules could therefore enhance the recognition of periodicity and ES definition. The learning effect of EEG was tested on a large scale in a previous study with a prospective design using an online teaching tool and included findings of wakefulness, drowsiness, normal variants, EDs, and artifacts. Among 91 readers, scores improved by 17.4% [14]. We benchmarked this education module and applied it to EEG features commonly found in critically ill patients.

From a clinical point of view, identifying nonconvulsive seizures is critical for initiating treatment. Among the major findings of the ACNS guidelines, improvement in the readings of ES and ESE after the application of an education module is highly encouraging. Before learning, the response to RDA was slightly poorer than that of PD, while the values were similar after learning, highlighting that rhythmicity should be considered and emphasized in education. The raters often omitted and responded to rhythmicity as irregular slow waves.

This study has some important limitations. The learning module and the EEG datasets in this study were not balanced across all EEG parameters. The inclusion of more balanced EEG datasets will be facilitated as additional EEG data are collected. However, achieving perfect balance across all educational items is not always ideal. Placing more emphasis on certain key items can lead to more effective education. It also remains unclear which items hold greater importance in clinical situations. From a data-sharing perspective, both major and minor items may be equally valuable for recognition. In this study, we utilized a relatively small number of EEG cases relative to the number of items tested. However, this is the first attempt to observe EEG ratings according to ACNS guidelines among non-English-speaking neurologists.

In conclusion, education is important for improving the standardization of EEG descriptions, and developing effective education modules will help improve communication and optimal patient care in the future.