Machine Learning Models to Predict Electroencephalographic Seizures in Critically Ill Children

Abstract

Objective:

Determine whether machine learning techniques would enhance our ability to incorporate key variables into a parsimonious model with optimized prediction performance for electroencephalographic seizure (ES) prediction in critically ill children.

Methods:

We analyzed data from a prospective observational cohort study of 719 consecutive critically ill children with encephalopathy who underwent clinically-indicated continuous EEG monitoring (CEEG). We implemented and compared three state-of-the-art machine learning methods for ES prediction: (1) random forest; (2) Least Absolute Shrinkage and Selection Operator (LASSO); and (3) Deep Learning Important FeaTures (DeepLIFT). We developed a ranking algorithm based on the relative importance of each variable derived from the machine learning methods.

Results:

Based on our ranking algorithm, the top five variables for ES prediction were: (1) epileptiform discharges in the initial 30 minutes, (2) clinical seizures prior to CEEG initiation, (3) sex, (4) age dichotomized at 1 year, and (5) epileptic encephalopathy. Compared to the stepwise selection-based approach in logistic regression, the top variables selected by our ranking algorithm were more informative as models utilizing the top variables achieved better prediction performance evaluated by prediction accuracy, AUROC and F1 score. Adding additional variables did not improve and sometimes worsened model performance.

Conclusion:

The ranking algorithm was helpful in deriving a parsimonious model for ES prediction with optimal performance. However, application of state-of-the-art machine learning models did not substantially improve model performance compared to prior logistic regression models. Thus, to further improve the ES prediction, we may need to collect more samples and variables that provide additional information.

Introduction

Electroencephalographic seizures (ES) occur in 10-40% of children with acute encephalopathy who undergo continuous EEG monitoring (CEEG),^1-32 increasing evidence indicates that high ES exposure is associated with unfavorable neurobehavioral outcomes,^{8,11,15,23,24} and ES are often treatable with anti-seizure medications.^33-35 Thus, CEEG use is increasing for ES identification and management,³⁶ and guidelines recommend that children with acute encephalopathy undergo 24-48 hours of CEEG to identify ES.^37-39 However, given CEEG is resource-intense,^40,41 a clinical prediction tool that enabled evidence-based targeting of CEEG resources to patients most likely to experience ES would be of great clinical value. Unfortunately, developing an ES prediction model with sufficiently high specificity to meaningfully reduce CEEG utilization and sufficiently high sensitivity to avoid failing to identify patients experiencing ES has been difficult. Prior studies have identified risk factors for ES in critically ill children,^{2,5-7,9-12,15,16,18,19,21,22} and generated multi-variable prediction models using standard regression approaches.^31,32,42,43 In a prospective observational study of 719 consecutive critically ill children with acute encephalopathy who underwent CEEG, we used a logistic regression approach to develop an ES prediction model. Variables associated with increased ES risk included age (≥1 year or <1 year), acute encephalopathy category (epilepsy-related, acute structural, acute non-structural), clinical seizures prior to CEEG initiation (present or absent), EEG background category (normal/sleep, slow-disorganized, discontinuous, burst-suppression, or attenuated-featureless), and epileptiform discharges (present or absent). The model yielded an area under the receiver operating characteristic curve (AUROC) of 0.80. If model cutoff was selected to emphasize sensitivity to avoid failing to identify a child experiencing ES, then the model would limit 29% of patients from undergoing CEEG while failing to identify 8% of patients with ES.³¹ A subsequent study indicated that the model was well calibrated and well discriminating in a separate validation cohort.⁴³

Machine learning techniques could improve prediction performance. In this study, we aimed to determine whether several machine learning techniques, ranging from widely used traditional machine learning models to novel neural network approaches, would enhance our ability to incorporate key variables into a parsimonious model with optimized prediction performance. In addition, given the power of the machine learning techniques, we aimed to develop a ranking algorithm for the features to better understand which features are most important for seizure prediction. This ranking information could provide guidance to prioritize data collection for future seizure risk prediction studies.

Methods

Study Subjects

The Institutional Review Board approved the study with a waiver of consent. We registered the study with clincialtrials.gov (NCT03419260) and applied observational study reporting standards.⁴⁴ Details regarding the cohort characteristics and a standard logistic regression model for ES prediction have been published.^31-33,43

We performed a prospective observational cohort study of consecutive critically ill children with encephalopathy treated in the pediatric intensive care unit (PICU) of a quaternary care hospital who underwent clinically-indicated CEEG to screen for ES based on a guideline-adherent^37-39 institutional pathway⁴⁵ between April 2017 and February 2019. We excluded: (1) neonates (<30 days old); (2) patients who received brief post-operative epilepsy surgery care in the PICU; and (3) patients admitted after >2 days of care for refractory status epilepticus at a different institution since the available initial clinical and EEG data were often insufficient. We performed continuous video-EEG monitoring using Natus Neuroworks (Middleton, WI) using the international 10-20 system for electrode placement. The inpatient Electroencephalography Service EEG interpreted the EEG studies, and the Critical Care Medicine and Neurology Consultation Services managed the patients.

We prospectively collected clinical and EEG data using a Research Electronic Data Capture (REDCap) database.⁴⁶ Clinical data included age, sex, prior neurodevelopmental disorders, medications, CEEG indication, hospital and PICU admission and discharge dates, presence of clinically evident seizures prior to CEEG, acute encephalopathy category (epilepsy-related, acute structural, or acute non-structural) based on the primary presenting problems/diagnoses available at the time of admission, and mental status (comatose or not; baseline or not). A pediatric electroencephalographer (FWF) scored the EEG through review of both tracings and clinical reports using standardized ⁴⁷ and critical care EEG terminology⁴⁸ which has good inter-rater reliability⁴⁹ and has been used in prior studies of CEEG in critically ill children.^{11,23,24,31-33,42,43,50-52} EEG data included initiation and discontinuation date and time, background features (normal/sleep, slow-disorganized, discontinuous, burst-suppression, or attenuated-featureless) epileptiform discharges, and ES presence and characteristics. We defined ES as abnormal paroxysmal events that were different from the background, lasted longer than 10 seconds, had a plausible electrographic field, and had a temporal-spatial evolution in morphology, frequency, and amplitude, as consistent with proposed definitions⁵³ and prior studies.^{16,23,31-33,52} We defined epileptiform discharges based on a glossary of EEG terms as transients distinguishable from the background with sharp or spiky morphology, different wave duration than the background, waveform asymmetry, and sometimes followed by an after-going slow-wave.⁴⁷

Machine Learning Methods

We implemented three machine learning methods to predict ES occurrence. First, random forest, is a popular tree-based machine learning algorithm for classification.⁵⁴ It was preferred for variable ranking for this dataset because all the variables in this study were categorical. In addition, random forest is more robust and interpretable than standard logistic regression. In our implementation, we built a random forest model consisting of 30 unpruned decision trees, each of which was built over a random extraction of the observations and a random extraction of the variables from the data. Thus, each tree only used partial variables and partial observations, which guaranteed that the trees were de-correlated and therefore, less prone to over-fitting. We built random forest models with different numbers of tree estimators, among which the model with best performance according to the five-fold cross validation was selected (Supplemental Table 1). After building the random forest, the entropy decreases from each variable were averaged across the decision trees to determine the importance of each variable. Second, Least Absolute Shrinkage and Selection Operator (LASSO),⁵⁵ is a regression analysis method performing simultaneous variable selection and regularization to enhance prediction accuracy and interpretability.⁵⁵ This method uses a penalty factor to control the number of variables kept in the model through an alpha parameter to regulate the weight of the penalty factor. A higher value of the alpha indicates that fewer variables are considered important and remain in the model. Starting from zero (all variables were included in the model), we gradually increased the value of alpha to 0.05 by 0.0001 at a time, and the model dropped variables which were less important. We tracked the set of remaining variables and ranked the importance of variables by the reverse order of each variable being dropped from the model. Third, Deep Learning Important FeaTures (DeepLIFT), is a state-of-the-art method which derives feature importance scores based on neural networks.⁵⁶ DeepLIFT is a novel algorithm for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every variable of the input. It can assign importance score to each input variable for a given output. With the power of DeepLIFT, we were able to prioritize variables critical for ES prediction and rank them directly using the derived importance score. We implemented fully connected neural networks with different structures, i.e., different number of hidden layers (Supplemental Table 2), and we selected the one with best performance in ES prediction. Then, we used DeepLIFT to rank the variables based on the importance scores. Neural network approaches have shown superior performance beyond traditional statistical methods in a variety of fields.⁵⁷

Model Performance Evaluation and Variable Ranking

We evaluated the model performance in predicting ES using five-fold cross validation with three metrics: (1) prediction accuracy including training and validation accuracy, which measures the percentage of observations, both positive and negative, that were correctly classified or predicted by the model in the training set and validation set, respectively; (2) AUROC; and (3) F1 score which combines precision and recall into one metric by calculating the harmonic mean between the two, and therefore considers the false positive and false negative predictions simultaneously. The standard logistic regression model previously reported from this cohort³¹ was the baseline for model comparisons.

We performed variable ranking using the random forest, LASSO, and DeepLIFT models. Subsequently, we calculated an overall composite ranking for each variable that combined the results across the three models. The overall composite ranking was calculated as a weighted sum of the three rankings using their validation accuracies as the weights. Variables with multiple categories were coded as dummy variables for each level against the baseline. The highest rank among the corresponding dummy variables was used as the rank for the multilevel variable. As comparison, we also obtained the order of the variables selected into the logistic regression model by the stepwise selection algorithm.

We considered three ‘forward’ scenarios using logistic regression to test the validity of the overall composite ranking. First, in the rank-based scenario, variables were added sequentially to the model one at a time in the order of the overall ranking starting from the highest ranking (i.e., the most important variable). Second, in the stepwise selection-based scenario, variables were added according to the stepwise model selection approach in a logistic regression. Third, in the random scenario, variables were added in a random order. Five-fold cross validation accuracies, AUROCs, and F1 scores were monitored during the process to evaluate the performance of these three scenarios. We hypothesized that to achieve similar performance, the prediction model would need fewer variables when it included the variables with the highest overall composite rankings as compared to including variables under the other two scenarios. This ranking algorithm would determine a relatively parsimonious model with the most important variables while achieving reasonably good ES prediction.

Results

Cohort

We enrolled 719 consecutive critically ill children. ES occurred in 184 subjects (26%). Table 1 provides the clinical and EEG characteristics of the full cohort and the subgroups with and without ES. We described the cohort previously as part of the standard logistic regression analysis to predict ES.³¹

Table 1.

Summary of clinical and EEG variable associations with electroencephalographic seizures. Data are presented as counts (percentages).

Variable		Total	Electrographic Seizures	No Electrographic Seizures
		719	184 (26%)	535 (74%)
Clinical	Age
	<1 Year	144 (20%)	47 (26%)	97 (18%)
	≥1 Year	575 (80%)	137 (75%)	438 (82%)
	Sex
	Male	417 (58%)	118 (64%)	299 (56%)
	Female	302 (42%)	66 (36%)	236 (44%)
	Race
	White	329 (46%)	81 (44%)	248 (46%)
	Black or African American	208 (29%)	54 (29%)	154 (29%)
	Asian	22 (3%)	7 (4%)	15 (3%)
	Other	3 (0.4%)	1 (0.5%)	2 (0.4%)
	Unknown	157 (22%)	41 (22%)	116 (22%)
	Ethnicity
	Not Hispanic or Latino	604 (84%)	156 (85%)	448 (84%)
	Hispanic or Latino	100 (14%)	26 (14%)	74 (14%)
	Unknown	15 (2%)	2 (1%)	13 (2%)
	Acute Encephalopathy Category
	Acute Non-Structural	159 (22%)	22 (12%)	137 (26%)
	Acute Structural	350 (49%)	79 (43%)	271 (51%)
	Epilepsy	210 (29%)	83 (45%)	127 (24%)
	Epileptic Encephalopathy
	No	608 (85%)	127 (69%)	481 (90%)
	Yes	111 (15%)	57 (31%)	54 (10%)
	Prior Developmental Delay/Intellectual Disability
	No	344 (48%)	69 (37%)	275 (51%)
	Yes	375 (52%)	115 (63%)	260 (49%)
	Prior Diagnosis of Epilepsy
	No	455 (63%)	82 (45%)	373 (70%)
	Yes	264 (37%)	102 (55%)	162 (30%)
	Clinical Seizures Prior to CEEG
	No Seizures	241 (34%)	30 (16%)	211 (39%)
	Seizures or Status Epilepticus	478 (66%)	154 (84%)	324 (61%)
	Mental Status
	No Coma	589 (82%)	159 (86%)	430 (80%)
	Coma	130 (18%)	25 (14%)	105 (20%)
	Mental Status
	Baseline	611 (85%)	31 (17%)	77 (14%)
	Worse than Baseline	108 (15%)	153 (83%)	458 (86%)
	Sedatives
	None	360 (50%)	114 (62%)	246 (46%)
	Intermittent Dosed	36 (5%)	6 (3%)	30 (6%)
	Continuous Infusion	323 (45%)	64 (35%)	259 (48%)
EEG	EEG Background Initial 30 Minutes
	Normal – Asleep	241 (34%)	36 (20%)	205 (38%)
	Slow and Disorganize	370 (51%)	117 (64%)	253 (47%)
	Discontinuous	38 (5%)	16 (9%)	22 (4%)
	Burst-Suppression	17 (2%)	8 (4%)	9 (2%)
	Attenuated and Featureless	53 (7%)	7 (4%)	46 (9%)
	Variability or Reactivity on Day 1
	No	224 (31%)	70 (38%)	154 (29%)
	Yes	495 (69%)	114 (62%)	381 (71%)
	Epileptiform Discharges in the Initial 30 Minutes
	None	563 (78%)	95 (52%)	468 (87%)
	Present	156 (22%)	89 (48%)	67 (13%)

Model Performance

Table 2 provides performance of the models using five-fold cross validation. Training accuracy was higher for the random forest model (0.963) and DeepLIFT model (0.962) compared to the standard logistic regression (0.762) or LASSO regression model (0.791) with alpha=0.01. All models were similar in validation accuracy, although the LASSO regression model achieved slightly higher validation (0.777) than the other models. Based on the F1 score, all models performed similarly with F1 scores ranging from 0.732-0.758.

Table 2.

Summary of the model performance using five-fold cross validation. Data are presented as mean (range).

Model	Training accuracy	Validation accuracy	AUROC	F1 score
Logistic Regression*	0.757 (0.726-0.773)	0.716 (0.643-0.762)	0.769 (0.741-0.815)	0.729 (0.662-0.773)
Random Forest	0.963 (0.958-0.967)	0.740 (0.699-0.764)	0.706 (0.676-0.732)	0.732 (0.688-0.751)
LASSO Regression	0.786 (0.773-0.794)	0.785 (0.748-0.826)	0.781 (0.736-0.822)	0.762 (721-0.802)
DeepLIFT	0.962 (0.940-0.972)	0.751 (0.706-0.812)	0.716 (0.680-0.753)	0.748 (0.721-0.760)

Notes: AUROC, area under receiver operating characteristic curve; DeepLIFT, Deep Learning Important FeaTures; LASSO, Least Absolute Shrinkage and Selection Operator.

^*The logistic regression values are derived from a prior publication.³¹

Variable Ranking

Table 3 provides rankings for all variables obtained from each model, as well as the overall composite ranking. According to the overall composite ranking, the top five variables for ES prediction were: (1) epileptiform discharges in the initial 30 minutes, (2) clinical seizures prior to CEEG, (3) sex, (4) age dichotomized at 1 year, and (5) epileptic encephalopathy. The last column of Table 3 lists the order of the variables selected into the logistic regression model by the stepwise selection algorithm. While most of the high-ranking variables were similar across the approaches, there were several differences. The variable with the top overall composite ranking, epileptiform discharges in the initial 30 minutes, was the fourth variable selected into the model by stepwise selection algorithm, while the EEG background initial 30 minutes was the eighth overall composite ranked variable yet was the third ranked variable by stepwise selection.

Table 3.

Rankings for all variables obtained from Random Forest, LASSO, and DeepLIFT. An overall composite ranking is calculated based on the three methods weighted by their validation accuracies, and it is compared to the order from stepwise selection algorithm in logistic regression.

Variable	Model			Overall Composite Ranking	Stepwise Selection Ranking
	Random Forest	LASSO	DeepLIFT
Epileptiform Discharges in the Initial 30 Minutes	1	1	2	1	4
Clinical Seizures Prior to CEEG	3	2	1	2	1
Sex	2	7	6	3	5
Age	5	6	4	4	2
Epileptic Encephalopathy	8	4	5	5	6
Sedatives	4	9	7	6	7
Prior Diagnosis of Epilepsy	12	3	8	7	14
EEG Background Initial 30 Minutes	9	5	9	8	3
Mental Status (Coma)	10	13	3	9	11
Variability or Reactivity on Day 1	6	12	10	10	13
Prior Developmental Delay / Intellectual Disability	7	11	11	11	10
Acute Encephalopathy Category	13	8	13	12	9
Ethnicity	14	10	12	13	8
Race	11	14	14	14	12
Mental Status (Worse than Baseline)	15	15	15	15	15

We evaluated the prediction performance of models with the top ranked variables. Figure 1 shows the changes of validation accuracies of the logistic regression models as we sequentially added variables one at a time to the model under the rank-based, stepwise selection-based, and random scenarios. When the variables were added to the model in the ranked order, the model started with a fairly high accuracy even with only the top variable included (0.777), peaked with the top three variables included (0.782), and then decreased slightly with addition of more variables. Comparatively, when variables were added in the stepwise selection-based order, the accuracy started relatively low (0.743), dropped as the top three variables were included (0.735), peaked with 13 variables added (0.779), and then decreased with additional variables added. Finally, when variables were added in a random order, the validation accuracy started low (0.742), dropped as other 8 variables were added (0.736), then increased and peaked with 19 variables included (0.783).

Figure 1.

Validation accuracy in scenarios in which variables were added in the order by ranking algorithm, stepwise selection, or randomly.

Figure 2 provides the comparison of AUROC of the logistic regression models for the rank-based, stepwise selection-based, and random scenarios. The AUROC for the rank-based scenario started lower with only one variable included (0.682) and peaked with seven variables (0.793). The AUROC for the stepwise selection-based scenario started from a lower (0.616) and peaked with seven variables (0.795). The AUROC for the random scenario started low (0.469) and peaked (0.782) with 15 variables.

Figure 2.

Validation using area under the receiver operating characteristic curve (AUC) in scenarios in which variables were added were added in the order by ranking algorithm, stepwise selection, or randomly.

Figure 3 provides the F1 score of the logistic regression models for the rank-based, stepwise selection-based, and random scenarios. For the rank-based scenario, the F1 score started from a peak value of 0.771 and fluctuated no lower than 0.751 as variables were included. For the stepwise selection-based scenario, the F1 score increased from 0.633 to 0.759 as variables were included. For the random scenario, the F1 score increased from 0.632 to 0.759 as more variables were included.

Figure 3.

Validation using the F1 score in scenarios in which variables were added in the order by ranking algorithm, stepwise selection, or randomly.

Thus, all three model performance metrics (prediction accuracy, AUROC, and F1 score) suggested that the top ranked variables by our ranking algorithm were more useful for ES prediction than the low ranked variables. Compared to the stepwise selection-based scenario, the top three variables selected by our ranking algorithm were more informative and achieved much higher validation accuracy, AUROC, and F1 score. In addition, models with the top eight variables by the overall composite rankings would achieve the best prediction performance, while adding more variables to the model did not help improve model performance.

Discussion

ES are common and associated with unfavorable neurobehavioral outcomes in children.^{8,11,15,23,24} However, CEEG is resource-intense,^40,41 so evidence-based strategies for targeting CEEG resources to patients at greatest risk for experiencing ES are needed. We previously used standard logistic regression methodology to develop an ES prediction model that incorporated three clinical and two EEG variables. It had an AUROC of 0.80, and at a 0.10 cutoff selected to emphasize sensitivity (to avoid missing patients experiencing ES), implementation would limit 29% of patients from undergoing CEEG while failing to identify 8% of patients with ES.³³ Applying the model in a separate validation cohort yielded similar performance characteristics.⁴³ Unfortunately, our data indicate that neither application of machine learning techniques, including advanced neural network approaches, nor incorporation of many potential predictor variables improved model performance compared to standard logistic regression. Model performance was similar using the standard logistic regression and all three machine learning techniques when assessed using several measures (prediction accuracy, AUROC, and F1 score). Further, the ranking of most variables was generally similar across the different approaches, including the standard logistic regression approach. Finally, when variables were added in the order by our ranking algorithm or by stepwise regression, the top variables yielded optimal performance by all performance metrics. Adding additional variables did not improve and sometimes worsened model performance. Overall, even with advanced machine learning techniques and inclusion of additional potential predictor variables, model performance reached a threshold with a validation accuracy of 0.72-0.78, and other accuracy assessments indicated a similar threshold effect.

These data indicate that standard logistic regression techniques using a small number of variables^33,43 may have achieved the most effective and parsimonious prediction models for determining ES risk. Novel approaches may be necessary to improve the value of CEEG through more targeted use. Several approaches may be beneficial. First, ES prediction in more homogeneous cohorts may be necessary. Second, value might be improved by instituting a more tailored approach to CEEG duration rather than targeting CEEG initiation to specific patient populations. This kind of strategy would entail initiation of CEEG broadly across patients but reduce CEEG duration in patients at lowest risk for ES. Studies in both adults and children indicate the value of this approach.^32,58 Our data indicated that the top-ranked overall composite variable was epileptiform discharges in the initial 30 minutes, which would be detectable with only a brief screening EEG. Thus, even in models intending to select patients for CEEG initiation, widespread use of a screening EEG may be beneficial, thereby making CEEG duration the key variable to hone for more cost-effective, tailored approaches.

According to the overall composite ranking which combined the three machine learning technique rankings, the top five variables for ES prediction were: (1) epileptiform discharges in the initial 30 minutes, (2) clinical seizures prior to CEEG initiation, (3) sex, (4) age dichotomized at 1 year, and (5) epileptic encephalopathy. The clinical and EEG variables associated with higher risk of ES are concordant with prior studies identifying risk factors for seizures to be: epileptiform discharges,^{2,5,6,12,17,31} the presence of clinically-evident seizures prior to CEEG initiation,^{6,7,11-13,17,31} and younger age (often dichotomized at one-year or two-years).^{6,11,13,14,16,31,42} The finding that sex is a key predictor is a new finding and may warrant further investigation. Acute encephalopathy category was ranked quite low (twelfth) despite being identified as a variable impacting ES incidence in prior studies.^7,12,14,31 Applying this variable in clinical prediction models is sometimes difficult due to overlapping conditions and diagnostic uncertainty at the time of presentation in some patients. Thus, it may be advantageous that our data indicate it is not a key variable for ES prediction. Similarly, prior studies have identified EEG background category as a predictor of ES,^2,5,31 but it was only the eighth ranked variable in our study. However, since some CEEG would be required to identify epileptiform discharges, the top ranked variable, removing EEG background from prediction models would not preclude the need for at least a brief screening EEG. It is also possible that EEG background is ranked lower because it may a better predictor of ES over longer periods of time, whereas epileptiform discharges in the acute period are more predictive initially.

There are several limitations to this study. Some of the limitations relate to the overall cohort and study design have been noted previously^31-33,43 and are summarized below. First, the study was conducted at one center. While the overall ES incidence and risk factors were consistent with prior studies, generalizability may be enhanced by replication in multi-center studies. Second, we did not test our models in an independent cohort due to limited sample size and low prevalence of seizures. For this type of study, it would be ideal to have an independent dataset that is not used for model building or validation to test for the final prediction models. In practice, when sample size is large and the prevalence of the outcome is relatively high, one approach is to divide the data into training, validation, and model testing datasets. Third, although EEG data used standardized terminology^47,48 with good inter-rater reliability⁴⁹ and commonly used definitions for seizures^1,16,23,53 and background categories,^{11,23,24,42,50,51} scoring was performed by a single electroencephalographer. We evaluated the same set of potential predictor variables as previously studied using the standard logistic regression. Inclusion of many variables using data mining approaches, potentially including clinical, EEG, laboratory, and imaging variables not initially expected to improve ES prediction, would be expected to yield superior model performance. However, the results from our ranking algorithm suggest that many variables in this data set provide redundant information, and the top five variables ranked by our algorithm capture the most representative pattern for ES prediction. Last, we tested how the sample size of the training dataset would affect the model performance by randomly selecting 216 samples as validation data and from the remaining data randomly selecting 100, 200, 300, 400, and 500 samples to form the training data to fit the model and then evaluate the model performance. As shown in Supplement Figure 1, the model’s accuracy and F1 score increased as the training sample size increased, which suggested that increasing the sample size could potentially improve the performance of these models.

Overall, the ranking algorithm was helpful in deriving a parsimonious model for ES prediction with optimal performance. However, application of state-of-the-art machine learning models did not substantially improve model performance compared to prior logistic regression models. Thus, with the currently available samples and variables in this data, there may be a limit to the model performance that can be achieved by ES prediction models even with advanced techniques and inclusion of numerous potential predictor variables guided by a ranking algorithm. To further improve the ES prediction, we may need to collect more samples and variables that provide additional information.

Highlights.

• The goal of this study was to determine whether machine learning techniques would enhance our ability to incorporate key variables into a parsimonious model with optimized prediction performance for electroencephalographic seizure (ES) prediction in critically ill children. We analyzed data from a prospective observational cohort study of 719 consecutive critically ill children with encephalopathy who underwent clinically-indicated continuous EEG monitoring (CEEG). We implemented and compared three state-of-the-art machine learning methods for ES prediction: (1) random forest; (2) Least Absolute Shrinkage and Selection Operator (LASSO); and (3) Deep Learning Important FeaTures (DeepLIFT). We developed a ranking algorithm based on the relative importance of each variable derived from the machine learning methods. Compared to the stepwise selection-based approach in logistic regression, the top variables selected by our ranking algorithm were more informative, and adding additional variables did not improve and sometimes worsened model performance. Overall, the ranking algorithm was helpful in deriving a parsimonious model for ES prediction with optimal performance. However, application of the machine learning models did not substantially improve model performance compared to prior logistic regression models. Thus, to further improve the ES prediction, we may need to collect more samples and variables that provide additional information.