Interrater Agreement of EEG Interpretation after Pediatric Cardiac Arrest Utilizing Standardized Critical Care EEG Terminology

Abstract

Purpose

We evaluated interrater agreement of electroencephalography (EEG) interpretation in a cohort of critically ill children resuscitated after cardiac arrest using standardized EEG terminology.

Methods

Four pediatric electroencephalographers scored 10-minute EEG segments from 72 consecutive children obtained 24 hours after return of circulation using the American Clinical Neurophysiology Society’s (ACNS) Standardized Critical Care EEG terminology. The percent of perfect agreement and the kappa coefficient were calculated for each of the standardized EEG variables and a predetermined composite EEG background category.

Results

The Overall Background Category (normal, slow-disorganized, discontinuous, or attenuated-featureless) had almost perfect agreement (kappa 0.89). The ACNS Standardized Critical Care EEG variables had agreement that was (1) almost perfect for the seizures variable (kappa 0.93), (2) substantial for the continuity (kappa 0.79), voltage (kappa 0.70), and sleep transient (kappa 0.65) variables, (3) moderate for the rhythmic or periodic patterns (kappa 0.55) and inter-ictal epileptiform discharge (kappa 0.60) variables, and (4) fair for the predominant frequency (kappa 0.23) and symmetry (kappa 0.31) variables. Condensing variable options led to improved agreement for the continuity and voltage variables.

Conclusions

These data support the use of the standardized terminology and the composite Overall Background Category as a basis for standardized EEG interpretation for subsequent studies assessing EEG background for neuroprognostication after pediatric cardiac arrest.

Introduction

Cardiac arrest occurs in over 16,000 children per year in the United States, and neurobehavioral morbidity is high among survivors.(1–9) However, biomarkers of brain injury severity early after cardiac arrest are lacking. Clinical and resuscitation characteristics are only moderately predictive of long-term outcomes,(10–14) In contrast, electroencephalographic (EEG) monitoring provides a robust method of assessing brain activity, and EEG data are commonly acquired at bedside early after cardiac arrest to identify non-convulsive seizures.(15, 16) Further, single center data establish that specific early post cardiac arrest EEG features are associated with short-term gross neurologic outcomes, supporting the concept that EEG features could serve as early, reliable, and clinically available brain injury severity biomarkers.(15, 17–22) Additionally, when neurologists and intensivists predict neurobehavioral outcomes from pediatric cardiac arrest cases, the addition of EEG data significantly improves prognostication accuracy.(23)

Given the potential value of EEG in assessing brain injury severity after cardiac arrest, in 2015 the International Liaison Committee on Resuscitation (ILCOR) recommended that “attempts should be made to… examine a standardized approach to EEG analysis,” and that “multicenter prospective studies that include longer-term outcomes would be valuable.”(24) Conducting multi-center studies of EEG in children after cardiac arrest will require implementation of standardized EEG interpretation systems. We have previously assessed the interrater agreement of some EEG variables in children after cardiac arrest,(25) but this work was done prior to the introduction of Standardized Critical Care EEG Terminology by the American Clinical Neurophysiology Society (ACNS).(26) The standardized terminology provides detailed definitions for EEG variables including continuity, variability, symmetry, predominant background frequency, anterior-posterior gradient, voltage, reactivity, variability, stage 2 sleep transients, epileptiform discharges, periodic and rhythmic discharges, and seizures. Recent studies in adult cohorts have evaluated the interrater agreement of this terminology,(27–30) but similar studies have not been conducted in children. We evaluated the interrater agreement of EEG interpretation in a cohort of critically ill children resuscitated after cardiac arrest using the standardized terminology.

Methods

We included consecutively recorded EEGs from children resuscitated after cardiac arrest who received care at the Children’s Hospital of Philadelphia. Demographic and clinical data were collected as part of an Institutional Review Board approved prospective observational study of pediatric cardiac arrest. As part of clinical care guided by an institutional pathway, EEG monitoring was performed in all patients resuscitated after cardiac arrest.(31) EEG monitoring was initiated as soon as possible after resuscitation and was performed using Grass video EEG equipment and the international 10–20 montage with modification for neonates as needed.

This study evaluated 10-minute long EEG segments obtained within 24 hours of return of circulation. The EEG segments were reviewed independently by four pediatric electroencephalographers who had undergone pediatric electroencephalography fellowships with a median of 1.5 years of post-fellowship experience (1 year, 1 year, 2 years, 8 years). The electroencephalographers were blind to all clinical information. All clinical annotations in the EEG tracing were removed prior to review. Video was not available, but reviewers could adjust the montage, filters, and voltage settings. EEGs were scored using the ACNS Standardized Critical Care EEG Terminology(26) with an electronic case report form in Research Electronic Data Capture (REDCap), a web-based electronic data application hosted at the Children’s Hospital of Philadelphia Research Institute.(32) Use of the electronic case report form ensured there were no missing data. Since the reviewed EEG segments were brief and usually did not contain reactivity testing, reactivity was not assessed. Since only 10-minute long EEG segments were assessed and not considered long enough to show variability, variability was not assessed. Following initial analyses using the variable choices provided by the Standardized Critical Care EEG Terminology, we combined some variable response options and evaluated whether these modifications improved interrater agreement. In addition to the ACNS standardized terminology derived variables, the electroencephalographers also scored the Overall Background Category which consisted of: (1) normal (including sedated sleep), (2) slow-disorganized, (3) discontinuous (which had to be excessive for gestational age in neonates) or burst-suppression, and (4) attenuated-featureless. Prior publications of EEG monitoring in critically ill children have used this categorical system.(20–22, 33)

Statistical analyses were performed using Stata 12 (College Station, Texas). Descriptive statistics were used to summarize the data as medians (with interquartile ranges) or counts (with percentages) as appropriate. Interrater agreement was assessed as percent of perfect agreement among the four reviewers and Fleiss’ kappa. The kappa coefficient measures the degree of agreement in classification over that which would be expected by chance. Kappa values range from 0 (interrater agreement does not differ from that by chance) to +1 (complete agreement).(34) The nomenclature presenting the strength of agreement of the kappa statistics was recommended by Landis and Koch (1977) as poor (< 0), slight (0–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), and almost perfect (0.81–1.0). To assess whether any one rater was substantially different than the rest we calculated the Fleiss’ kappa for each variable while removing one rater at a time and compared the kappa values from the remaining 3-rater cohorts to that of the full 4-rater cohort. To assess whether agreement differed by subject age, we compared agreement in tracings from older and younger than two years.

Results

The study included 72 who experienced cardiac arrests between 8/2012 and 4/2016. Table 1 provides the demographic and clinical data from this cohort. The median patient age was 2.2 years (IQR 0.3, 9.9; range 0.01, 17.9), 43 (59%) had a pre-arrest Pediatric Cerebral Performance Category score (PCPC) of 1 (normal) or 2 (moderate disability) and only 19% had no preexisting conditions. The median duration of cardiopulmonary resuscitation was 10 minutes (IQR 4.5, 23.5), and 48 (67%) subjects had in-hospital cardiac arrests.

Table 1.

Cohort demographics (N=72).

Clinical Variable	N (%) or Median [IQR]
Age at Arrest (years)	2.2 [0.3, 9.9]
Sex: Male	48 (67%)
Pre-arrest PCPC Score
1 (normal)	32 (44%)
2 (mild disability)	11 (15%)
3 (moderate disability)	6 (8%)
4 (severe disability)	10 (14%)
5 (coma or vegetative state)	2 (3%)
Unknown	11 (15%)
Pre-existing condition: no	14 (19%)
Arrest location
In-Hospital	48 (67%)
Out-of-Hospital	24 (33%)
Bystander CPR (N=24)	14 (58%)
CPR duration (minutes) (N=56)	10 [4.5, 23.5]
Initial Rhythm
Asystole/pea	24 (33%)
Bradycardia	25 (36%)
VF/VT	8 (11%)
Other/unknown	15 (20%)
Epinephrine Doses	3 [1, 5]
Induced hypothermia	9 (13%)
Benzodiazepine infusion	28 (53%)
Mortality	28 (39%)
Discharge PCPC Score
1 (normal)	14 (19%)
2 (mild disability)	10 (14%)
3 (moderate disability)	9 (12%)
4 (severe disability)	10 (14%)
5 (coma or vegetative state)	1 (1%)

CPR, cardiopulmonary resuscitation; PCPC, pediatric cerebral performance category.

Table 2 provides the EEG variables, available choices for each variable, frequency of reported findings by each of the four electroencephalographers, percent perfect agreement, and kappa values. The Overall Background Category (normal, slow-disorganized, discontinuous, or attenuated-featureless) had almost perfect agreement (kappa 0.89). The ACNS Standardized Critical Care EEG variables had agreement that was (1) almost perfect for the seizures variable (kappa 0.93), (2) substantial for the continuity (kappa 0.79), voltage (kappa 0.70), and sleep transient (kappa 0.65) variables, (3) moderate for the rhythmic or periodic patterns (kappa 0.55) and inter-ictal epileptiform discharge (kappa 0.60) variables, and (4) fair for the predominant frequency (kappa 0.23) and symmetry (kappa 0.31) variables.

Table 2.

Perfect agreement, kappa statistics, and EEG variable choices.

EEG Variable	Perfect Agreement for All 4 Raters (%)	Kappa		EEG Variable Choices Number of Responses (%)
		Kappa (95% confidence interval)	Strength of Agreement
Overall Background Category	86%	0.89 (0.84, 0.95)	Almost Perfect	Normal 49 (17%)	Slow-Disorganized 116 (40%)	Discontinuous 59 (20%)	Burst-Suppression 19 (6%)	Attenuated-Featureless 45 (16%)	-	-	-
Continuity	78%	0.79 (0.77, 0.81)	Substantial	Continuous 152 (53%)	Nearly Continuous with Attenuation 19 (7%)	Nearly Continuous with Suppression 3 (1%)	Discontinuous with Attenuation 39 (14%)	Discontinuous with Suppression 17 (6%)	Burst-Suppression 12 (4%)	Burst Attenuation 7 (2%)	Suppression 39 (14%)
Voltage	72%	0.70 (0.67, 0.73)	Substantial	Normal 177 (61%)	Low 64 (22%)	Suppressed 47 (16%)	-	-	-	-	-
Predominant Frequency	17%	0.23 (0.13, 0.27)	Fair	Attenuated 36 (13%)	Delta 103 (36%)	Delta + Theta 89 (31%)	Delta + Theta + Alpha 60 (21%)	-	-	-	-
Symmetry	88%	0.31 (0.19 0.43)	Fair	Symmetric 270 (94%)	Mild Asymmetry 5 (2%)	Marked Asymmetry 13 (5%)	-	-	-	-	-
Stage 2 Sleep Transients	82%	0.65 (0.54, 0.76)	Substantial	Normal 34 (12%)	Abnormal 10 (3%)	Absent 242 (84%)	Awake Only 2 (0.7%)	-	-	-	-
Seizures	99%	0.93 (0.85, 1.00)	Almost Perfect	Absent 273 (95%)	Present 15 (5%)	-	-	-	-	-	-
Periodic or Rhythmic Patterns	86%	0.55 (0.41, 0.60)	Moderate	Absent 261 (91%)	Present 27 (9%)	-	-	-	-	-	-
Sporadic Epileptiform Discharges	79%	0.60 (0.47, 0.62)	Moderate	Absent 238 (83%)	Present 50 (17%)	-	-	-	-	-	-

For variables with four or more response options we created confusion matrices (error matrices) to allow visualization of the errors made by each EEG reviewer when the most frequent response was considered the true response. These analyses allowed us to determine whether responses without agreement were close (one category) or far (more than one category) from the true (most common) responses. For the Overall Background category (Table 3), most responses were on the diagonal of the table, indicating the respondents provided the true answer. Fourteen responses were not accurate, including 11 responses (79%) within one category of the correct response and 3 (21%) responses more than one category different than the correct response. For the Continuity category, 28 responses were inaccurate, including 13 responses (46%) within category of the correct response and 15 responses (54%) more than one category different than the correct response. For the Frequency category, 98 responses were inaccurate, including 71 responses (72%) within category of the correct response and 27 responses (28%) more than one category different than the correct response. Given that many inaccurate responses were “closest-neighbor” classification errors, we combined some response categories. Agreement increased to almost perfect for the continuity and voltage variables. Revising the continuity category to continuous/nearly continuous (174, 60%) or discontinuous/burst-suppression (75, 26%) or suppression (39, 14%) increased perfect agreement to 88% and kappa to 0.87 (0.82, 0.91; almost perfect). Revising the voltage category to normal (177, 61%) or low/suppressed (111, 39%) only slightly increased perfect agreement to 75% and kappa to 0.71 (0.66, 0.76; substantial agreement). However, revising the voltage category to normal/low (241, 84%) or suppressed (47, 16%) increased perfect agreement to 92% and increased kappa to 0.92 (0.75, 0.90; almost perfect agreement). Revising the symmetry category to include only symmetry (270, 94%)) or asymmetry (18, 6%) did not change perfect agreement (88%) and only slightly increased the kappa to 0.37 (0.13, 0.51; fair agreement). Revising the sleep transient categories to include only present (44, 15%) or absent (244, 85%) only slightly increased the perfect agreement to 89% and the kappa to 0.79 (0.71, 0.91; substantial agreement).

Table 3.

Confusion Matrix for Overall Background Category. The most common answer is considered the true answer for comparison to each of the four raters. Most responses were on the diagonal indicating the respondents provided the true answer. Fourteen responses were not accurate including 11 responses (79%) within 1 category of the correct response and 3 (21%) responses more than one category different than the correct response.

		True (Most Common) EEG Score
		Normal	Slow and Disorganized	Discontinuous	Burst-Suppression	Attenuated and Featureless
Respondent Scores	Normal	Rater 1: 12
		Rater 2: 11
		Rater 3: 12*	1
		Rater 4: 12	1
	Slow and Disorganized		29
			28	1		1
		1*	27	1		1
			27
	Discontinuous			14
			1	13
			1	13	1	1
			1	14
	Burst-Suppression				5
					4
					4
					5	1
	Attenuated and Featureless					12
						11
					1	10
						11

^*Among EEG segments for which the true (most common) response was “normal”, rater 3 scored 12 as “normal” and 1 as “slow and disorganized.”

We performed three sensitivity analyses (Table 4). First, the Kappa values did not change substantially when each rater was individually removed from the analysis indicating no single rater diverged from the group. Second, since the normal EEG changes with age, we analyzed agreement categorized by age less than two years and greater than or equal to two years at the time of the cardiac arrest. Kappa scores were similar by age group except for differences in symmetry (younger 0.15 slight; older 0.51 moderate), periodic or rhythmic patterns (younger 0.64 substantial; younger 0.49 moderate) and sporadic epileptiform discharges (younger 0.35 fair, older 0.79 substantial). Third, since the prevalence of EEG categories might vary based on the use of therapeutic hypothermia, we performed analyses in only the 63 subjects who did not receive therapeutic hypothermia. The Kappa values did not change substantially in this subgroup.

Table 4.

Kappa values for the 4-rater cohort, with each rater removed, by age, and for only subjects who did not receive therapeutic hypothermia.

EEG Variable	Kappa (95% confidence interval)	Kappa for 3 of 4 Raters				Kappa by Subject Age		Kappa for Subjects without Therapeutic Hypothermia (N=63)
		Rater 1 Excluded	Rater 2 Excluded	Rater 3 Excluded	Rater 4 Excluded	< 2 Years (N=38)	≥2 Years (N=34)
Overall Background Category	0.89 (0.84, 0.95)	0.86 (0.77, 0.89)	0.89 (0.78, 0.95)	0.91 (0.85, 0.96)	0.88 (0.85, 0.92)	0.91 (0.86, 0.94)	0.83 (0.65, 0.89)	0.89 (0.88, 0.90)
Continuity	0.79 (0.77, 0.81)	0.80 (0.74, 0.85)	0.78 (0.77, 0.83)	0.81 (0.77, 0.82)	0.76 (0.70, 0.81)	0.79 (0.75, 0.86)	0.75 (0.65, 0.90)	0.79 (0.65, 0.90)
Voltage	0.70 (0.67, 0.73)	0.70 (0.66, 0.86)	0.69 (0.63, 0.74)	0.73 (0.60, 0.80)	0.69 (0.60, 0.80)	0.66 (0.53, 0.75)	0.72 (0.65, 0.76)	0.68 (0.60, 0.72)
Predominant Frequency	0.23 (0.13, 0.27)	0.24 (0.21, 0.27)	0.19 (0.10, 0.22)	0.19 (0.15, 0.29)	0.27 (0.20, 0.31)	0.24 (0.21, 0.37)	0.20 (0.16, 0.22)	0.23 (0.18, 0.28)
Symmetry	0.31 (0.19, 0.43)	0.34 (0.32, 0.44)	0.31 (0.08, 0.45)	0.35 (0.17, 0.38)	0.26 (0.04, 0.40)	0.15 (0.13, 0.19)	0.51 (0.00, 1.00)	0.13 (0.05, 0.22)
Stage 2 Sleep Transients	0.65 (0.54, 0.76)	0.63 (0.60, 0.72)	0.76 (0.70, 0.78)	0.58 (0.57, 0.72)	0.65 (0.48, 0.70)	0.68 (0.50, 0.83)	0.62 (0.54, 0.76)	0.69 (0.59, 0.72)
Seizures	0.93 (0.85, 1.00)	0.90 (0.80, 1.00)	0.90 (0.71, 0.94)	1.00 (1,00, 1.00)	0.90 (0.79, 1.00)	0.85 (0.66, 1.00)	0.92 (0.79, 1.00)	0.87 (0.79, 0.89)
Periodic or Rhythmic Patterns	0.55 (0.41, 0.60)	0.49 (0.47, 0.56)	0.62 (0.35, 0.70)	0.53 (0.43, 0.67)	0.58 (0.47, 0.70)	0.64 (0.45, 0.79)	0.49 (0.23, 0.64)	0.50 (0.30, 0.57)
Sporadic Epileptiform Discharges	0.60 (0.47, 0.62)	0.64 (0.52, 0.72)	0.60 (0.43, 0.68)	0.61 (0.55, 0.63)	0.54 (0.45, 0.62)	0.35 (0.17, 0.43)	0.79 (0.53, 0.86)	0.56 (0.45, 0.62)

Discussion

Four pediatric electroencephalographers evaluated 72 EEG segments from consecutive children resuscitated after cardiac arrest. The Overall Background Category classification had almost perfect agreement. Interrater agreement was variable for EEG features as defined in the ACNS Standardized Critical Care EEG terminology, and combining response categories led to improvement in interrater agreement for some variables.

It is meaningful that assessments of Overall Background Category, continuity, voltage, and seizure occurrence all had substantial interrater agreement since these variables are used in proposed pediatric prognostication systems.(15, 17–22) It is useful that the combined category (overall EEG category) had substantial inter-rater agreement since this is consistent with prior literature indicating that there is higher reproducibility for broad interpretive categories than more narrow EEG features in both children and adults.(28, 29, 35–43) Establishing that key EEG features have substantial interrater agreement is a critical step in developing a standardized and reliable approach to EEG analysis which could be used for multicenter prospective pediatric cardiac arrest trials, as has been advocated for by ILCOR.(24)

EEG-based brain injury stratification has been an effective strategy to define hypoxic-ischemic brain injury severity in neonatal neuroprotective trials(44) and contributed to the evaluation of therapeutic hypothermia as a neuroprotective strategy for neonates moderate brain injury.(45) In contrast, the recent large Therapeutic Hypothermia after Pediatric Cardiac Arrest (THAPCA) trials did not identify any outcome differences in subjects managed with and without therapeutic hypothermia.(7, 8) However, EEG features to stratify early brain injury severity after pediatric cardiac arrest had not been established, and therefore the THAPCA trials lacked the granularity to evaluate the impact of therapeutic hypothermia on subjects with varying degrees of brain injury. Recognizing this knowledge gap, in 2015 the International Liaison Committee on Resuscitation recommended that “attempts should be made to… examine a standardized approach to EEG analysis,” and that “multicenter prospective studies that include longer-term outcomes would be valuable.”(24) However, the lack of standardized terminology for EEG description in critically ill children has been a barrier to understanding the clinical significance of specific EEG findings, and to conducting multi-center research. Thus, a committee of experts from the ACNS developed standardized terminology that is intended to improve reproducibility by providing a common standard for EEG interpretation in critically ill patients.(26)

The ACNS Standardized Critical Care EEG Terminology had not been previously validated for use in children after cardiac arrest. We previously assessed the interrater agreement of some EEG variables in critically ill children after cardiac arrest,(25) but this work was done prior to the introduction of the standardized terminology and used different EEG variables and response options. Despite these differences, the prior findings are generally like the current data. Our prior study identified that agreement was substantial for continuity, burst suppression, sleep architecture, and overall rating. Agreement was moderate for seizure occurrence and inter-ictal epileptiform discharge type. Agreement was fair for inter-ictal epileptiform discharge presence, beta activity, predominant frequency, and fastest frequency. Agreement was slight for maximum voltage and focal slowing presence.(25)

The interrater agreement of the ACNS Standardized Critical Care EEG Terminology was recently assessed in 103 comatose adults following cardiac arrest who were enrolled in the Target Temperature Management trial. In that study, four electroencephalographers blind to clinical information reviewed 20-minute duration EEG segments and had agreement that was substantial for continuity (kappa=0.71) and voltage (kappa=0.65), which was the same as the substantial agreement we identified in our pediatric cohort. Agreement for predominant background frequency was moderate (kappa=0.36) which was better than the fair agreement identified in our pediatric cohort. Agreement was moderate for the presence of periodic or rhythmic discharges (kappa=0.56) which was like the agreement in our pediatric cohort. In addition to the ACNS standardized terminology, the variable “highly malignant EEG” was assessed. It had been used in prior studies assessing EEG for neuroprognostication after cardiac arrest in adults (46) and was characterized by a suppressed background with or without periodic discharges or a burst-suppression background. Interrater agreement was substantial (kappa=0.71).(28) This may be similar to the Overall Background Category which had substantial interrater agreement in this study and has been used in prior pediatric investigations.(20–22, 33)

A similar ACNS consensus based process created standardized terminology for neonatal EEG.(47) Neonatal EEG categorization system by three pediatric electroencephalographers reviewing EEG from 60 neonates with hypoxic-ischemic brain injury had interrater agreement that was very good for identification of seizures (kappa=0.93), moderate for classifying records as normal or having any abnormality (kappa=0.49), and good in classifying EEG backgrounds on a five-category scale (normal, excessively discontinuous, burst suppression, status epilepticus, or electrocerebral inactivity) (kappa=0.70). Agreement was lower for other EEG features including voltage (kappa=0.41), variability (kappa=0.35), symmetry (kappa=0.18), presence of abnormal sharp waves (kappa<0.20), and presence of brief rhythmic discharges (kappa<0.20).(43)

EEG monitoring is often used after pediatric cardiac arrest to identify electrographic seizures which occur in 10–50% of children. Most electroencephalographic seizures have no clinical correlate and thus are not identified by clinical observation.(15, 20) Our study of 128 children post cardiac arrest undergoing EEG monitoring identified electrographic seizures in 15% of subjects. Electrographic status epilepticus occurred in 80% of subjects with seizures and was associated with unfavorable short-term neurologic outcomes.(20) Based on these data, surveys indicate EEG monitoring is increasing in critically ill patients,(48, 49) and recent consensus statements strongly recommend continuous EEG monitoring in patients with acute encephalopathy including after cardiac arrest.(16, 50) Given the clinical effort to identify seizures, it is reassuring that interrater agreement for seizure identification is substantial. A study of inter-observer agreement in seizure detection in critically ill adults demonstrated moderate agreement was demonstrated for seizure occurrence, but agreement was lower for more subtle seizure descriptors including their frequency, onset, and offset.(42) Similarly, identification of seizures has been shown to be very good among neonates with hypoxic-ischemic brain injury(43) and substantial among critically ill adults.(30)

This study has several notable strengths. First, EEG studies were obtained from consecutive children resuscitated after cardiac arrest, indicating the frequency of these EEG categories likely represent the true prevalence of these EEG categories in this population. Second, four electroencephalographers independently reviewed the EEG segments while blind to all clinical data, ensuring the interpretations were not biased by patient characteristics. Third, reviewers scored 10-minute duration segments of EEG with the ability to modify any settings, which mimics real-world EEG reading since it allows data manipulation which could not occur with simpler screenshots of the EEG data. However, there are also limitations to this study. First, we did not assess intra-rater agreement. In a prior study assessing the use of the ACNS Standardized Critical Care EEG Terminology in adults after cardiac arrest, intra-rater agreement varied by EEG variable including almost perfect (continuity), substantial (highly malignant pattern), fair (voltage) and slight (predominant frequency).(28) Second, we did not assess reactivity since many of the EEG segments under review did not assess reactivity. Third, we assessed for the presence of periodic/rhythmic patterns, but since these patterns were rare we did not assess the inter-rater agreement of additional characteristics of these patterns as has been assessed in adult studies.(27, 29, 30) Fourth, some EEG variable categories were more common than other categories, and this lack of sample variety could lead to bias. For example, if a pattern was rarely present there could be high agreement for identifying the pattern as absent by chance alone. For symmetry only 7% of subjects had mild or marked asymmetry leading to 88% agreement but kappa 0.31 (fair). For periodic/rhythmic discharges only 9% had discharges present leading to 86% agreement but kappa 0.55 (moderate). Fifth, the electroencephalographers were from one institution and several had only 1–2 years of post-fellowship experience. Further assessment with readers from different institutions and with more experience might impact the findings. Finally, we did not evaluate whether agreement would improve after a group consensus meeting. Prior studies indicate that agreement improves after group discussion and implementation of rules.(25, 37)

Overall, these data support the use of the ACNS Standardized Critical Care EEG Terminology as a basis for standardized EEG interpretation for clinical and research purposes. There is high reliability in pediatric electroencephalographer interpretation of important EEG variables such as overall background category, continuity, voltage, and seizure occurrence. In contrast, some common and seemingly simple EEG features such as symmetry and predominant frequency had only fair agreement. Terminology improvements or more quantitative methods of assessment might be needed if these EEG features are utilized in neuroprognostication models. Combining some response categories (continuity and voltage) led to improved interrater agreement which may be important to consider in future ACNS terminology revisions.

Figure 1.

Example images for the Overall Background Category variable. The images show normal (delta activity with symmetric sleep spindles), slow and disorganized (delta activity), discontinuity (periods of activity separated by low-amplitude periods), burst suppression (bursts of activity separated by low-amplitude inter-burst intervals), and attenuation-featureless (highly attenuated with no higher voltage periods) patterns.