Quantitative EEG predicts outcomes in children after cardiac arrest

Abstract

Objective

To determine whether quantitative EEG (QEEG) features predict neurologic outcomes in children after cardiac arrest.

Methods

We performed a single-center prospective observational study of 87 consecutive children resuscitated and admitted to the pediatric intensive care unit after cardiac arrest. Full-array conventional EEG data were obtained as part of clinical management. We computed 8 QEEG features from 5-minute epochs every hour after return of circulation. We developed predictive models utilizing random forest classifiers trained on patient age and 8 QEEG features to predict outcome. The features included SD of each EEG channel, normalized band power in alpha, beta, theta, delta, and gamma wave frequencies, line length, and regularity function scores. We measured outcomes using Pediatric Cerebral Performance Category (PCPC) scores. We evaluated the models using 5-fold cross-validation and 1,000 bootstrap samples.

Results

The best performing model had a 5-fold cross-validation accuracy of 0.8 (0.88 area under the receiver operating characteristic curve). It had a positive predictive value of 0.79 and a sensitivity of 0.84 in predicting patients with favorable outcomes (PCPC score of 1–3). It had a negative predictive value of 0.8 and a specificity of 0.75 in predicting patients with unfavorable outcomes (PCPC score of 4–6). The model also identified the relative importance of each feature. Analyses using only frontal electrodes did not differ in prediction performance compared to analyses using all electrodes.

Conclusions

QEEG features can standardize EEG interpretation and predict neurologic outcomes in children after cardiac arrest.

More than 10,000 children in the United States experience a cardiac arrest annually.^1–3 Following return of circulation, accurate and timely neuroprognostication is critical in making management decisions.^4–6 Clinical and resuscitation variables may not optimally predict neurobehavioral outcomes since they do not directly assess brain function.^7–10 EEG data can be obtained noninvasively at the bedside and provide a standardized brain activity assessment, which may be useful in delineating the extent of brain injury.^11,12 When neurologists and intensivists predicted neurobehavioral outcomes from cardiac arrest cases, the addition of EEG data significantly improved prognostication accuracy.¹³ Current practice involves EEG review by an electroencephalographer, and several EEG features have been associated with gross neurologic outcome at hospital discharge in children after cardiac arrest.^13–26 However, since visual EEG interpretation relies on qualitative interpretations by electroencephalographers, inter-rater agreement limitations^12,25–27 may influence EEG-based neuroprognostication.^12,25–27

Quantitative EEG (QEEG) analyses use computationally derived features that highlight specific components of EEG with numerical values.²⁸ Thus, QEEG analyses circumvent inter-rater variability limitations and provide a more objective method for stratifying brain injury severity. This may be helpful for clinical trials, acute treatment decisions, and neuroprognostication. In addition, QEEG features can extrapolate long-term trends²⁹ that may not be identified by an electroencephalographer. With further development, QEEG analyses could be used by bedside caregivers, thereby potentially serving as a widely implementable neuromonitoring tool. QEEG features are associated with outcome after cardiac arrest in adults,^28–30 but similar studies have not been conducted in children. Since the characteristics of cardiac arrest, EEG, and outcomes are age-dependent, studies of QEEG are necessary in children. In this study, we present a machine learning algorithm that uses QEEG features to predict neurologic outcomes in children resuscitated after cardiac arrest.

Methods

Standard protocol approvals, registrations, and patient consents

This study was approved by the Institutional Review Board at the Children's Hospital of Philadelphia. We obtained written informed consent for each participant.

Cardiac arrest database

This was a prospective study of consecutive infants and children treated in the pediatric intensive care unit of a single tertiary care referral hospital between September 2012 and February 2016. Clinical practice at our institution was to perform continuous EEG monitoring in all encephalopathic patients resuscitated after cardiac arrest to identify electroencephalographic seizures, consistent with recent guidelines and consensus statements.^31,32 EEG monitoring was initiated urgently and performed with a portable bedside Grass-Telefactor video-EEG system using 21 gold-over-silver scalp surface electrodes positioned according to the international 10–20 system and affixed with collodion adhesive. EEG data were deidentified and full-length tracings were saved for research purposes.

Study data consisted of prospectively defined demographic, cardiac arrest, resuscitation, postcardiac arrest care, EEG, and neurologic outcome variables. Data were collected prospectively using an electronic case report form in Research Electronic Data Capture (REDCap), a web-based electronic data application.³³

We assessed outcomes at discharge from the pediatric intensive care unit by chart review using Pediatric Cerebral Performance Category (PCPC) scores. The PCPC is a validated 6-point scale that categorizes degrees of functional impairment (1 = normal, 2 = mild disability, 3 = moderate disability, 4 = severe disability, 5 = coma and vegetative state, and 6 = death).³⁴ Unfavorable neurologic outcome was defined broadly as PCPC scores of 4–6, and not only death (PCPC score = 6), to reduce the influence of family decisions regarding withdrawal of technological support on outcome categorization. A patient's outcome would be categorized as unfavorable whether a family chose to withdraw or continue technological support of a child with severe neurologic injury.

We enrolled 87 consecutive patients. We excluded 5 patients who had good neurologic outcome soon after the cardiac arrest but subsequently died due to medical complications unrelated to their initial cardiac arrest (figure 1). We also excluded 9 patients who did not have early EEG recordings and 4 patients who did not have late EEG recordings. Thus, 69 patients were included for analysis (figure 1).

Figure 1. Workflow of data collection and analysis.

AUC = area under the receiver operating characteristic curve; NPV = negative predictive value; PPV = positive predictive value; QEEG = quantitative EEG; ROC = receiver operating characteristic.

Statistical analyses comparing demographic and clinical data were performed using Stata 15.0 (College Station, TX). We report summary statistics as medians and interquartile ranges for continuous variables and counts and proportions for categorical variables. We examined the association of each variable with neurologic outcome using χ² or Fisher exact tests for categorical variables and Wilcoxon signed rank test or Kruskal-Wallis tests for continuous variables.

EEG data extraction and preprocessing

Due to the data recording methods, we did not have continuous data for all patients across all time points. Therefore, we selected 5-minute EEG epochs from every hour of EEG available. The selected 5-minute EEG epochs were filtered and tested for artifacts using a MATLAB script. The signal filtering process was done by first using a 6th order Butterworth bandpass filter^35,36 between 0.1 and 50 Hz. The signals were then filtered with a 60 Hz notch filter to ensure removal of the 60 Hz power line artifact.^37,38 We checked the filtered signals for artifacts using the following steps. We calculated EEG channel SDs for all channels across all 5-minute epochs for all patients. Statistical outliers, defined as values that were more than 3 median absolute deviations away from the median,³⁹ were calculated for the computed SDs through all the 5-minute epochs to identify artifacts. If any of the epochs contained artifacts, then the 5-minute window was shifted until there was a window within the given hour without artifacts. If there were no windows within the hour that were artifact-free, then no epoch was selected from that hour. An electroencephalographer visually inspected and validated all EEG clips to ensure correct preprocessing.

QEEG features

We computed 8 QEEG features: SD of each channel in the 5-minute epochs, normalized band power in alpha, beta, theta, delta, and gamma wave frequencies, line length, and regularity function scores.²⁸ Feature selection was clinically motivated and was intended to capture voltage, spectrographic characteristics, and trace discontinuity. All features were computed per channel from 5-minute epochs from every hour of EEG available.

SD was used as a feature to illustrate the variance of the signal. Spectrographic features of the EEG were captured using the normalized band power values of delta (0.5–3 Hz), theta (4–7 Hz), alpha (8–12 Hz), beta (13–30 Hz), and gamma (25–50 Hz) waves.^29,40 The band power in each frequency range was first computed and then normalized with respect to the total power of the 5-minute epoch. The band powers were normalized in this manner because their relative ratios and the variability in ratios to the total EEG power are considered important in clinical practice.⁴¹ Line length, which is the sum of all absolute distances between consecutive points,⁴² was used to depict overall voltage and continuity. Line length is a running window measure of the path-dependent distance traced out by the EEG signal. Regularity function scores capture patterns of discontinuity in an EEG signal. The regularity function²⁸ outputs a value between 0 and 1 where a higher value corresponds to a signal with a more regular amplitude. In addition to the QEEG features, patient age was added as a feature because age plays a critical role in assessing pediatric EEGs.

Early and late EEGs

For all 5-minute epochs, the 8 QEEG features were computed for each channel and then averaged across all channels. All QEEG features were then averaged across early (0–17 hours after return of circulation) and late (18 hours–end after return of circulation) EEG recordings to determine the predictive ability power of early vs late EEG data.

Model selection and training

The feature matrix formed to train the classifiers had 69 samples with 9 features each. The labels were binary outcomes (favorable or unfavorable neurologic outcomes). We trained logistic regression, support vector machine, and random forest classifiers. We used the random forest classifier to classify the EEG features given its robustness and interpretability.⁴³

Model evaluation

We used 5-fold cross-validation to examine the performance of the 3 machine learning classifiers. To evaluate model performance, we computed receiver operating characteristic (ROC) curves, area under the ROC curve (AUROC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). For all models, we also calculated 95% confidence intervals (CIs) for accuracies and AUROC values based on 1,000 bootstrapped samples.

Data availability statement

Anonymized data not published within this article can be made available to qualified investigators upon request to the corresponding author.

Results

Patient demographics

Table 1 provides demographic data. Thirty-seven (54%) patients had favorable outcomes (PCPC 1–3) and 32 (46%) patients had unfavorable outcomes (PCPC 4–6).

Table 1.

Demographic data for the full cohort divided by neurologic outcome

Model performance

Table 2 provides the test characteristics (accuracy, sensitivity, specificity, PPV, NPV, and AUROC values) for the early and late EEG random forest models with 5-fold cross-validation. Figure 2 provides the ROC curves for the early and late EEG random forest models with 5-fold cross-validation. The ROC curve is a plot of the algorithm's sensitivity vs false-positive rate (1 − specificity) for different cutoff values at which the algorithm assigns outcomes. The AUROC is representative of the algorithm's utility for clinical decision-making. If the area is close to 1.0, then its discrimination is excellent. If the area is near 0.5, then its ability to discriminate between binary outcomes is close to chance. Early EEGs showed better results than late EEGs across all performance metrics, but the differences were not statistically significant. The random forest model could predict clinical outcomes based on early QEEG features with an accuracy of 0.80, sensitivity of 0.84, specificity of 0.75, PPV of 0.79, NPV of 0.80, and AUROC of 0.88. The random forest model could predict clinical outcomes based on late QEEG features with an accuracy of 0.70, sensitivity of 0.76, specificity of 0.62, PPV of 0.70, NPV of 0.69, and AUROC of 0.74.

Table 2.

Performance metrics of random forest models with 5-fold cross-validation using early and late EEGs

Figure 2. Receiver operating characteristic curves (ROC) and area under the receiver operating characteristic curves (AUROC).

Mean ROC and AUROC values for random forest model with 5-fold cross-validation predicting neurologic outcome in (A) early and (B) late EEG using all channels.

To better understand the generalizability of the models, 1,000 bootstrapped samples were used to compute the 95% CIs for the accuracies and AUROC values (table 3). The random forest model for early EEG had a mean accuracy of 0.69 (95% CI 0.68–0.70), and the mean AUROC was 0.80 (95% CI 0.79–0.81). For late EEG, the mean accuracy was 0.61 (95% CI 0.61–0.62), and the mean AUROC was 0.70 (95% CI 0.69–0.71).

Table 3.

Accuracies and area under the receiver operating characteristic curve (AUROC) values of random forest models developed using early and late EEGs

Since limited montage EEG can be obtained more easily than full-array EEG, we evaluated random forest models using only frontal electrodes (Fp1, Fp2, Fz) in the same manner as models built using all EEG channels. The random forest model using early QEEG features performed better than the random forest model using late EEG features across all performance metrics, but the differences were not statically significant. The random forest model with 5-fold cross-validation using early QEEG features from frontal electrodes had an accuracy of 0.71, sensitivity of 0.76, specificity of 0.66, PPV of 0.72, NPV of 0.70, and AUROC of 0.80. The random forest model using late QEEG features from the frontal electrodes had an accuracy of 0.67, sensitivity of 0.70, specificity of 0.62, PPV of 0.68, NPV of 0.65, and AUROC of 0.72. Figure 3 depicts ROC curves for early and late EEG random forest models with 5-fold cross-validation using only frontal electrodes.

Figure 3. Receiver operating characteristic curves (ROC) and area under the receiver operating characteristic curves (AUROC).

Mean ROC and AUROC values for random forest model with 5-fold cross-validation predicting neurologic outcome in (A) early and (B) late EEG using only frontal channels (Fp1, Fp2, Fz).

We computed 95% CIs for the accuracies and the AUROC values of the models using only frontal electrodes using 1,000 bootstrapped samples (table 3). The random forest model for early EEG using frontal electrodes had a mean accuracy of 0.67 (95% CI 0.66–0.67) and a mean AUROC of 0.76 (95% CI 0.75–0.77). The late EEG model using frontal electrodes had a mean accuracy of 0.64 (95% CI 0.63–0.65) and a mean AUROC of 0.71 (95% CI 0.70–0.72).

Feature significance

Feature significance was evaluated using Gini importance.⁴⁴ The reported scores range from 0 to 1, in which higher scores indicate more important features. Figure 4 reports the features importance scores. For the early EEG model, the most important features were gamma band power, delta band power, and beta band power. For the late EEG model, the most important features were age, alpha band power, and delta band power.

Figure 4. Feature importance scores of random forest models.

Feature importance scores of random forest models for (A) early EEG data and (B) late EEG data. Scores were evaluated using Gini importance.⁴⁴ Features scores range from 0 to 1 and higher scores indicate more important features. Bp = band power.

Discussion

We used random forest models to identify multiple QEEG features that are predictive of neurologic outcome in children after cardiac arrest. When predicting outcomes, the worst mistake would be to incorrectly predict an unfavorable outcome, leading to withdrawal of technological support in a patient who might have had a favorable outcome. Thus, a model predicting unfavorable outcomes aims to minimize false-positives and thereby achieve high specificity. The random forest model that best predicted unfavorable neurologic outcome used all electrodes and early QEEG features, and it had a specificity of 0.75 and PPV of 0.79. Since these are imperfect, the model should not be used in isolation when guiding clinical decision-making. However, these data suggest that multimodal models are likely to benefit from inclusion of QEEG data.

Predictive performance was compared between early and late EEGs. The selected cutoffs for early and late EEGs maximized the number of patients included in the data analysis because patients who did not have both early and late EEG recordings were excluded. Early EEGs showed better results than late EEGs across all performance metrics. However, the difference in model performance between early and late EEGs was not statistically significant. Nevertheless, these results are consistent with a previous QEEG study in adults that indicated that the predictive value of EEG is highest at 12–24 hours after cardiac arrest.²⁸ This is a promising avenue of investigation because early EEG features can be reliably used for neuroprognostication during the acute stages when interventions may be most efficacious. This early ability to predict outcome may also be useful for stratifying patients soon after cardiac arrest for future neuroprotection studies.

Considering the small sample size (69 samples with 9 features each), logistic regression was fitted on the dataset for its simplicity, efficiency, and robustness to noise.⁴⁵ A support vector machine was then trained on the same data because it is more flexible than logistic regression and allows for kernels that are nonlinear.⁴⁶ A third classifier, the random forest, was used because ensemble methods perform with the highest accuracy for many datasets, and is commonly used for medical applications.⁴³ Of the 3 machine learning classifiers, the random forest classifier had the best performance based on 5-fold cross-validation, and thus it was used to classify the features.

For our main analyses, QEEG features were averaged across all channels because there were no clinically significant differences in performance between channels. However, since hypoxic–ischemic brain injury is diffuse, we hypothesized that a limited set of frontal channels might yield similar test characteristics. The finding that the performance of the models using only the frontal channels was comparable to the performance of the models using all channels (tables 2 and 3) implies that the model can be implemented using EEG data just from the frontal EEG channels (Fp1, Fp2, Fz), which might be more feasibly obtained soon after cardiac arrest than a full array EEG.⁴⁷

While this study showed promising results, there are several limitations. First, this study had a relatively small sample size (n = 69). Therefore, we were unable to partition our data to include a holdout validation set for our model. While we used cross-validation and bootstrapping to overcome the limitations of the small sample size, these results should be validated in a larger study. Second, this study used a MATLAB script for artifact removal that is not as accurate as clinically trained neurologists. While manual EEG review is the current gold standard, independent component analysis–based algorithms could obtain EEG of improved signal quality.⁴⁰ Third, this study was based only on EEG, but multimodal models also incorporating clinical features could provide a more complete understanding of the neurologic state of the patient and yield more accurate neuroprognostication. Fourth, we assessed short-term outcome using a relatively simple outcome measure. Future studies might incorporate longer-term patient-centered neurobehavioral outcomes. Fifth, clinically interpreted EEG data were known to the clinical teams and may have influenced care decisions. To reduce the influence of individual decisions regarding withdrawal of technological support on outcome categorization, we used an outcome measure that grouped death and unfavorable neurologic outcome. Thus, outcome would be categorized as unfavorable whether a family chose to withdraw or continue technological support of a child with severe neurologic injury.

There are 2 additional technical considerations. First, based on feature importance scores evaluated using Gini importance,⁴⁴ line length had lower importance scores for both early and late EEG models. The line length feature was computed from 5-minute windows, which may have been too wide to capture the important details of the EEG that would have been predictive of outcome. Especially for capturing transient events such as bursts, it may be necessary to use smaller windows. One study that developed an automated burst detection method with 84% accuracy recommended using 1-second windows.⁴² Using a smaller moving window within the 5-minute epoch could be explored in future analyses. We did not implement a shorter time window because some EEG recordings extended up to 10 days and doing computations on 1-second windows would have made the model computationally intensive. Second, it is surprising that the normalized gamma (25–50 Hz) band power had the highest feature importance score for the early EEG model since frequencies above 25 Hz are not commonly encountered in scalp EEG.⁴⁸ Frequencies in the gamma band range may be attributed to muscle and movement artifacts.^40,48 The artifacts in each patient's recording should be similar since they were acquired under similar recording conditions. The frequency band powers were normalized with respect to the total power in each window so the gamma band comprising a higher proportion of the total EEG power indicates that there is relatively lower power in frequencies below 25 Hz, where most brain activity is detected. Decreased brain activity is predictive of an unfavorable outcome. Our results showed that patients who had favorable outcomes generally had higher power in the delta (0.5–3 Hz) band.

In future studies, we are interested in exploring several avenues. First, in this study the models classify patients into 2 classes (favorable or unfavorable outcomes) based on the probabilities of a patient belonging to either class. In binary classification, the class with greater than 0.5 probability is the predicted class. However, there are some patients whose probability of belonging to either class can be close to 0.5, such that predictions are made with considerably less certainty. These patients are more often misclassified, thereby reducing the predictive power of the models for such patients. Instead of binary classification, it might be beneficial to include a third class when the probabilities of a patient belonging to either class are less certain. An initial model could focus on multinomial logistic regression with 3 classes of outcomes. Second, we intend to combine clinical variables with QEEG features to build more comprehensive multimodal models. Specific clinical variables are commonly used by intensive care unit clinicians to predict outcomes in children with cardiac arrest.⁶ Third, employing deep learning models can also improve predictive performance. Recently, there have been studies that use deep learning with convolutional neural networks to extract information from raw EEG recordings.^49,50 Deep convolutional neural networks have shown promise in decoding information from EEGs and revealing novel features that could be used in EEG analysis.⁵⁰

Overall, these data provide initial evidence that QEEG features, which are standardized and may make EEG interpretation more objective for clinical decision-making, can contribute to outcome prediction in children after cardiac arrest. Further study is needed to assess the benefit of using QEEG features in the context of multimodal models for clinical trials or neuroprognostication.

Glossary

AUROC

area under the receiver operating characteristic curve

CI

confidence interval

NPV

negative predictive value

PCPC

Pediatric Cerebral Performance Category

PPV

positive predictive value

QEEG

quantitative EEG

ROC

receiver operating characteristic

Appendix. Authors

Study funding

No targeted funding reported.

Disclosure

S. Lee and X. Zhao report no disclosures relevant to the manuscript. K.A. Davis is funded by NINDS K23-NS073801 (to K.A. Davis) and the Thornton Foundation. A. Topjian reports no disclosures relevant to the manuscript. B. Litt is funded by NINDS R01NS099348-01, Mirowski Family Foundation, and Neil and Barbara Smit Fund. N. Abend is funded by NINDS K02NS096058, PCORI, and EFA. Go to Neurology.org/N for full disclosures.