Machine Learning Applications in Epilepsy

SUMMARY

Machine learning leverages statistical and computer science principles to develop algorithms capable of improving their performance through interpretation of data rather than through explicit instructions. Alongside widespread use in image recognition, language processing, and data mining, machine learning techniques have received increasing attention in medical applications, ranging from automated imaging analysis to disease forecasting. This review examines the parallel progress made in epilepsy, highlighting applications in automated seizure detection from EEG, video, and kinetic data, automated imaging analysis and pre-surgical planning, prediction of medication response, and prediction of medical and surgical outcomes using a wide variety of data sources. A brief overview of commonly used machine learning approaches, as well as challenges in further application of machine learning techniques in epilepsy, is also presented. With increasing computational capabilities, availability of effective machine learning algorithms, and accumulation of larger datasets, clinicians and researchers will increasingly benefit from familiarity with these techniques and the significant progress already made in their application in epilepsy.

INTRODUCTION

Within the field of artificial intelligence, machine learning bridges statistics and computer science to develop algorithms whose performance improves with exposure to meaningful data, rather than with explicit instructions.¹ Alongside ubiquitous applications in speech recognition, image classification, and text translation,² machine learning has increasingly been used in a variety of medical applications, including triaging of ophthalmology referrals based on optical coherence tomography data,³ diagnosis of malignant melanoma from dermoscopic and photographic images,⁴ and identification of influenza from emergency department encounter records,⁵ in all three instances matching or exceeding the performance of clinical experts. Similar advances have also been made in epilepsy, driven by continuous improvements in data collection, storage, and processing. To illustrate the broad utility of machine learning techniques in epilepsy, this work highlights recent applications in automated seizure detection, analysis of imaging and clinical data, epilepsy localization, and prediction of medical and surgical outcomes, following a brief overview of commonly used machine learning algorithms.

MACHINE LEARNING APPROACHES

Machine learning tasks may broadly be classified as supervised or unsupervised learning (Figure 1A). In a supervised learning task, an algorithm is ‘trained’ on a set of previously-labeled input data in order to estimate outputs for unlabeled data.⁶ For example, annotated EEG recordings may be used to train an algorithm to automatically detect epileptiform discharges. In contrast, in unsupervised learning, an algorithm is used to uncover trends, subgroups, or outliers in unlabeled input data.⁶ Compared to the prior example, an unsupervised algorithm may identify candidate epileptiform discharges by detecting outliers from the background EEG recording. With either approach, informative input features are identified in a process termed feature selection (either manually, based on expert-level knowledge, or by the algorithm itself, without requiring domain expertise), then analyzed using a mapping function that generates output predictions from these features.⁷

Figure 1.

Overview of machine learning concepts. A, In supervised learning (top), an algorithm is trained on labeled input data to generate predicted outputs, while in unsupervised learning (bottom), the algorithm uncovers subgroups or outliers in unlabeled data. B, The random forest algorithm generates a forest of decision trees, each utilizing subsets of input features as bifurcation points to differentiate the training data into expected outputs; the output of the ensemble (e.g., the majority vote) is reported for new inputs. C, In k-nearest neighbor classification, an input is plotted as a vector within a feature space alongside labeled data, and is subsequently assigned to the class of its k nearest neighbors (here, k = 4). D, Support vector machines generate a hyperplane in higher-dimensional feature space to maximally separate labeled training data. E, In artificial neural networks, input data (far left) is passed through a non-linear activation function and assigned a weight, projecting through intermediary nodes until reaching the output node (far right) for classification. F, In cross-validation, a subset of the training data is withheld as the validation set (yellow), allowing for fine-tuning of an algorithm parametrized on the training set (light green); after multiple iterations (here showing K-fold cross-validation with K = 5), the algorithm may be tested on an initially withheld testing set (dark green) to assess accuracy and generalizability of the finalized model.

Among the more commonly used mapping functions, the random forest algorithm creates multiple decision trees from training data, in which each tree uses a randomly selected subset of input features as iterative bifurcation points to best separate the input data into the expected outputs (Figure 1B).⁸ The resultant forest of decision trees then generates an ensemble output (e.g., the most frequently predicted classification among the decision trees) for new inputs.⁸ In k-nearest neighbor (k-NN) classification (Figure 1C), a given input is represented by a vector within a feature space, and the distance between the input vector and labeled vectors in the training set are calculated; the input is then assigned to the class of the majority of its k nearest neighbors (e.g., for k = 5, the input is assigned to the same class as its nearest five training data points).⁹ Support vector machine (SVM) classifiers, in contrast, generate a hyperplane in a higher-dimensional feature space to maximally separate clusters of labeled training data, providing a decision boundary for classifying new inputs (Figure 1D).¹⁰ Multilayer artificial neural networks process data through layers of nodes, in each of which weighted inputs are summated and passed through a non-linear activation function to yield intermediary outputs; these may in turn proceed through additional layers of nodes as desired, ultimately reaching output nodes (Figure 1E).¹¹ Deep learning refers to this use of multiple ‘hidden layers,’ and provides the advantage of automated feature selection with sufficiently large datasets, enabling high accuracy without incorporating domain expertise—in other words, training data can effectively serve as programming code for a deep learning model.¹¹ Advancements in deep learning have thus enabled many of the recent accomplishments in machine learning, such as automated image classification,¹² speech recognition,¹³ and text translation.¹⁴

The choice among mapping functions for a given application may be influenced by several factors, including sample size (e.g., requiring larger datasets to achieve high accuracy with deep learning),¹¹ ease of use (whether from fewer user-specified variables, as in k-NN, or from availability of user-friendly software packages), or relative interpretability (increasingly facilitated by a growing number of algorithm-specific and algorithm-agnostic approaches)¹⁵; ultimately, however, the selection is often empiric, iterative, and informed by investigator experience.¹⁶ Mapping functions may also be combined in a given classification task to form an ensemble, based on the principle that a committee of predictors prone to non-identical errors will prove more accurate than any given predictor alone.¹⁷ Ensembles may be generated by applying the same mapping function to subsets of data, training the same function using different sets of parameters, or by combining different mapping functions, overall resulting in improved accuracy at the expense of computation time, data size, and comprehensibility.¹⁸ Along similar lines, automated machine learning (auto-ML) approaches the selection of a mapping function and its parameters as a machine learning task, allowing for selection and optimization of a mapping function or ensemble of functions with little user input.¹⁶

With small datasets or excessively complex models, the mapping function may ‘overfit’ the training data, learning properties inherent to the sample rather than relationships in the overall population; as a result, the generalizability, or out-of-sample accuracy, of the algorithm is diminished.¹⁹ To limit overfitting, the training data may be divided into a training set, from which the algorithm sets its parameters, and a validation set, from which a preliminary estimate of the algorithm’s generalizability is obtained (Figure 1F). The latter can then be used to adjust the hyperparameters of the algorithm, i.e., aspects of the algorithm not directly learned from training data (including selected features, type of mapping function, or prespecified properties of the mapping function, such as the number of neighbors in k-NN classification) that impact the efficacy of the algorithm.¹⁹ This partitioning and readjustment can be repeated a prespecified number of times (K-fold cross-validation) or repeated across the entire dataset by holding out a single data point for validation in each iteration (leave-one-out cross-validation).¹⁹ Subsequently, a third, testing set of withheld data may be used to estimate the generalizability of the final model (Figure 1F), e.g., by determining accuracy, sensitivity, and specificity for binary classifiers.¹⁹ Another commonly-reported measure is the area under the receiver operating characteristic curve (AUC), which examines sensitivity as a function of the false-positive rate at different test thresholds; the area under this curve provides a prevalence-independent measure of the test’s discriminatory ability.²⁰ An alternate approach to overcoming overfitting is transfer learning, wherein a deep learning model previously trained for one task may be re-trained using a far smaller dataset for a different task, for instance by re-training an image classification algorithm to classify Gram stain images.²¹

AUTOMATED SEIZURE DETECTION

Given the utility of machine learning in analyzing large, complex datasets, considerable attention has been paid to automated seizure detection in EEG recordings. A wide variety of techniques have been applied to this task, including SVM,^22,23 k-NN,^24,25 and deep learning classifiers.^26,27 Similar techniques have also been applied to seizure forecasting from EEG data, achieving prediction times of as much as several minutes using limited databases,^28,29 including on low-power chips suitable for wearable or implantable devices.³⁰ Recent progress and future directions in seizure detection and forecasting have been reviewed elsewhere in greater depth.^31–33

Aside from EEG recordings, however, machine learning techniques have also been applied to novel data sources for seizure detection. In neonatal epilepsy, Karayiannis et al examined extremity movements in bedside video recordings, training neural networks to classify recordings as focal clonic seizures, myoclonic seizures, or non-seizure movements; after training on 120 recordings, the authors achieved seizure detection sensitivities of 85.5–94.4% and specificities of 92.5–97.9% (varying with the type of input data, with overall better detection of myoclonic seizures) on a matched testing set.³⁴ A follow-up study with modified video analysis algorithms and neural network architecture improved on this performance, yielding a peak sensitivity of 96.8% and specificity of 97.7%.³⁵ Ogura et al added EEG data to video-based extremity movement measurements, demonstrating a sensitivity of 96.2% and specificity of 94.2% for detecting epileptic spasms using a probabilistic neural network.³⁶ In pediatric and adult epilepsy, early studies relied on infrared markers attached to anatomic landmarks for motion detection, and were therefore limited by occlusion of the reflective markers.³⁷ Accordingly, Cuppens et al tracked spatio-temporal interest points defined by corner detection algorithms in nocturnal video recordings, achieving a peak sensitivity of 77% in detecting myoclonic jerks using an SVM classifier (though noting reduced performance for recordings including subtle myoclonic jerks or intermixed epileptic and non-epileptic movements).³⁸ Along similar lines, Achilles et al extracted depth and position data from an infrared Kinect camera (Microsoft, Redmond, WA) set up in an epilepsy monitoring unit, using a convolutional neural network to identify tonic, tonic-clonic, and focal motor seizures. After training on videos of seizures from five patients, the algorithm achieved an AUC of 78.3% with five test patients, achieving a processing speed (10 frames per second) compatible with real-time analysis.³⁹ Taken together, while there remains room for improvement in seizure detection accuracy from video recordings, machine learning-enabled techniques are rapidly approaching real-time applicability in seizure telemetry and characterization in epilepsy monitoring units, as well as in supplemental analysis of ambulatory video recordings.

In addition to video data, machine learning approaches have also been applied to analysis of movement information. One recent multicenter study of 69 patients combined wrist-worn accelerometer data with measurements of electrodermal activity (a reflection of sweat gland activation by the sympathetic nervous system), achieving a sensitivity of 94.6% and false-positive rate of 0.20 per day (AUC 94%) with a median detection latency of 29.3 seconds using an SVM classifier.⁴⁰ Milošević et al also applied SVM to data from extremity-worn accelerometers combined with surface electromyography (sEMG) recordings from both biceps, achieving a combined sensitivity of 90.9%, clinical latency of 10.5 seconds, and false positive rate of 0.45 alarms per 12-hour recording in seven pediatric patients. Notably, the authors found 40–45% of these false alarms to be of clinical interest, including for falls, EEG lead removal, or concerning behaviors.⁴¹ In another notable study, individual muscle component transforms of upper extremity accelerometer data were examined with an SVM classifier, demonstrating a 93.3% sensitivity and 85.0% specificity for distinguishing non-epileptic convulsions from motor seizures.⁴² Additional machine learning approaches, including random forest⁴³ and k-NN⁴⁴ classifiers, have also been successfully applied to movement data. Overall, the portability of accelerometer- and sEMG-based systems holds promise for more accurate detection and tracking of seizures in the ambulatory setting, potentially also incorporating pulse oximetry,⁴⁵ electrocardiography,⁴⁶ and audio⁴⁷ data in future applications.

MACHINE LEARNING IN THE DIAGNOSIS OF EPILEPSY

Imaging Analysis in Epilepsy

Aside from detection or prediction of individual seizures, machine learning approaches have also been investigated in the broader diagnosis of epilepsy using a wide variety of data sources. In particular, a number of studies have applied machine learning techniques to analysis of imaging data. Zhang et al examined asymmetry of functional connectivity in homologous brain regions on resting-state functional MRI (fMRI) in 100 patients with epilepsy and 80 controls, achieving a peak diagnostic sensitivity of 82.5% and specificity of 85% using an SVM classifier.⁴⁸ Amarreh et al examined fractional anisotropy, mean diffusivity, radial diffusivity, and axial diffusivity in diffusion tensor imaging (DTI) data from 20 pediatric patients and 29 controls, finding that an SVM classifier was able to accurately distinguish patients with active epilepsy from those in remission (i.e., seizure-free for 12 months and not on medications) as well as from controls.⁴⁹ Soriano et al, in contrast, examined resting-state magnetoencephalography (MEG) data from 14 patients with focal frontal epilepsy, 14 patients with idiopathic generalized epilepsy, and 14 controls, achieving 88% sensitivity and 86% specificity in differentiating the three groups using sequential neural network classifiers.⁵⁰ While these studies are limited by modest sample sizes, they illustrate the capabilities of machine learning techniques in uncovering subtle anatomic and functional network changes in epilepsy, thereby enabling novel lines of inquiry in imaging analysis. In one illustrative example, Pardoe et al trained a Gaussian processes regression algorithm to estimate patient age using T1-weighted MRI sequences from a database of 2001 healthy controls; in subsequently applying the algorithm to 42 patients with newly-diagnosed focal epilepsy, 94 patients with medically refractory focal epilepsy, and 74 contemporaneous controls, they noted a 4.5-year disparity between imaging-based estimated brain age and chronological age in medically refractory patients, with larger disparities noted with earlier age of onset.⁵¹

One particular application of machine learning techniques in imaging analysis has been the automated characterization of focal cortical lesions. Hong et al applied SVM classifiers to cortical thickness and curvature maps from 41 patients with focal cortical dysplasias as well as matched controls, achieving 98% accuracy in distinguishing histologic subtypes, 92% and 86% accuracy (for type I and II FCDs, respectively) in lateralization, and 92% and 82% accuracy (as above) in predicting Engel I seizure freedom at an average of four years of follow-up.⁵² Noting the difficulty of adequately capturing lesional variability with a small number of samples, El Azami et al instead demonstrated an unsupervised approach, training a one-class support vector machine to identify heterotopias and blurring of the gray-white matter junction as outliers on T1-weighted images from 77 healthy controls. When tested on 11 patient scans, the classifier achieved a sensitivity of 77% with a mean of 3.2 false positive detections per patient, compared to a sensitivity of 54% and an average of 6.3 false positives using state-of-the-art statistical parametric mapping (Wellcome Centre for Human Neuroimaging, London, UK), driven by improved detection of lesions not visible on expert-annotated MRI studies.⁵³ More recently, a neural network classifier demonstrated a sensitivity of 73.7% and specificity of 90.0% in detecting solitary FCDs on a dataset of 61 patients with type II FCDs as well as 120 controls from three different epilepsy centers, notably comprising healthy controls as well as controls with hippocampal sclerosis.⁵⁴

Non-Imaging Diagnostics

Outside of imaging analysis, machine learning techniques have similarly enabled epilepsy diagnosis from a wide range of clinical data. Goker et al applied several classification algorithms to 105 scanning EMG recordings from nine juvenile myoclonic epilepsy patients and ten controls, achieving 100% diagnostic sensitivity and 83.6% specificity using an artificial neural network.⁵⁵ Kassahun et al employed a genetics-based data mining approach (which incorporated an auto-ML algorithm) to distinguish temporal lobe epilepsies (TLEs) from extratemporal epilepsies using clinical semiologies, finding favorable performance compared to an individual clinician and comparable performance to a team of clinicians examining the same data.⁵⁶ Using an SVM classifier, Won et al. examined visual evoked potential measurements of contrast gain control to differentiate 10 idiopathic generalized epilepsy patients from 19 controls, achieving sensitivities of 80–86% and specificities of 77–85%.⁵⁷ Frank et al examined neuropsychological batteries administered on 228 patients with epilepsy, using an SVM classifier to distinguish patients with TLEs from extratemporal epilepsies (with 72.2–78.0% accuracy) and to lateralize the epileptic focus among the TLE patients (with 52.8–61.3% accuracy).⁵⁸ A number of investigators have also applied machine learning techniques to analysis of unprocessed medical records for epilepsy characterization. Connolly et al examined progress notes for pediatric patients with focal, generalized, or otherwise unspecified epilepsies, analyzing frequencies of word strings using an SVM classifier; after training on 90 notes of each category from one institution, the algorithm demonstrated better-than-chance classification of progress notes from a different institution, with improved performance if the classifier was trained using notes from two institutions before being tested on notes from a third.⁵⁹ Biswal et al, in contrast, trained a Naïve Bayes classifier on 3,277 EEG reports labeled with the presence or absence of seizures or epileptiform discharges, which then achieved an AUC of 99.05% for detecting reports with seizures and 96.15% for those with epileptiform discharges on a testing set of 39,695 reports.⁶⁰ Building on this approach, Goodwin and Harabagiu combined automated indexing of EEG reports with neural network-generated ‘fingerprints’ of the associated recordings to create a searchable database, allowing for identification of patient cohorts within the database with such queries as, “History of seizures and EEG with TIRDA without sharps, spikes, or electrographic seizures;”⁶¹ a panel of expert reviewers found that the addition of EEG ‘fingerprint’ indices provided more relevant results and captured pertinent records not recovered by searches of EEG reports alone.⁶¹ The broad range of data sources examined in these studies highlights the utility of machine learning techniques in leveraging underutilized clinical data for patient- and population-level characterization of epilepsy.

Lateralization of Epilepsy

Another area of focus in machine learning-enhanced epilepsy diagnostics has been the detection and lateralization of temporal lobe epilepsies. In early studies, Bakken et al analyzed hippocampal magnetic resonance spectroscopy data from 15 patients with TLE and 13 controls using an artificial neural network, correctly distinguishing and lateralizing patients with TLEs from controls in 60 of 67 spectra.⁶² In another study, a neural network classifier demonstrated 85% agreement with an expert reviewer in detecting and lateralizing TLEs on interictal ¹⁸F-fluorodeoxyglucose-positron-emission tomography (FDG-PET) images from 197 patients and 64 controls.⁶³ Kerr et al later applied neural networks to interictal FDG-PET images from 73 patients with TLE and 32 patients with non-epileptic seizures, accurately identifying and lateralizing TLE in 76% of cases, compared to accuracies of 78% with expert review of FDG-PET data and 71% with expert review of MRI data.⁶⁴ Interestingly, limited correlation was noted between the results of expert review and the neural network classifier, suggesting that the neural network did not recapitulate manual analysis.⁶⁴ Multifractal analysis of ictal and interictal single photon emission computed tomography (SPECT) data, examined using an SVM classifier, has also been shown to accurately localize TLE in 19 of 20 cases, comparing favorably to conventional image subtraction.⁶⁵ A number of studies have also applied machine learning techniques to morphometric analysis of structural MRI using T1-weighted and fluid-attenuated inversion recovery (FLAIR) sequences, both for detection⁶⁶ and lateralization⁶⁷ of TLE. Similar results have more recently been shown for analysis of diffusion weighted imaging,⁶⁸ diffusion tensor imaging,⁶⁹ and diffusion kurtosis imaging.⁷⁰ Recently, Jin and Chung also applied an SVM classifier to functional connectivity data from resting-state MEG, distinguishing 46 TLE patients from matched controls with 95.1% accuracy and lateralizing TLEs with 76.2% accuracy.⁷¹

APPLICATIONS IN SURGICAL MANAGEMENT OF EPILEPSY

Machine learning approaches have also increasingly been applied in surgical planning and prediction of surgical outcomes in epilepsy. Dian et al examined intracranial EEG recordings from six patients undergoing resection for extratemporal epilepsy to identify regions of interest using an SVM classifier, finding close concordance with the ultimate resection area in patients achieving Engel I outcomes but limited concordance in patients with Engel III or IV outcomes, potentially suggesting the need for a larger resection area in the former or inaccurate identification of the seizure onset zone in the latter.⁷² In a pediatric population, Roland et al highlighted the use of a neural network-based algorithm in identifying canonical resting-state networks on fMRI as part of presurgical planning, allowing for identification of clinically relevant networks even under general anesthesia.⁷³ Alongside applications in FCD detection and TLE lateralization, these findings begin to highlight the broader applicability of machine learning techniques in surgical planning for extratemporal epilepsies or those with ostensibly unrevealing imaging findings.

In the domain of predicting surgical outcomes, an early study by Grigsby et al trained a neural network classifier on encoded clinical, electrographic, neuropsychologic, imaging, and surgical data from 65 patients undergoing anterior temporal lobectomy, yielding a sensitivity of 80.0% and specificity of 83.3% in predicting Engel I outcomes (improving to 100% and 85.7%, respectively, for Engel I or II outcomes) in a testing cohort of 22 patients.⁷⁴ Arle et al also applied neural networks of varying architecture to a similar dataset (though also incorporating an index of medication number and dosage), reporting an accuracy of 96% in predicting Engel I outcomes in 80 surgical patients; curiously, the authors noted an improvement in accuracy to 98% when excluding intraoperative variables (such as electrocorticography or pathology data).⁷⁵ More recently, Armañanzas et al compared k-NN and Naïve Bayes classifiers in predicting Engel I vs. II–III outcomes using similar presurgical data, though with the notable addition of a larger neuropsychological battery that included personality style as assessed using a Rorschach test; after narrowing the classifiers to the three most informative variables (including personality style), the authors noted predictive accuracies of 89.47% with either classifier.⁷⁶ Memarian et al instead compared linear discriminant analysis with Naïve Bayes and SVM classifiers in examining pre-operative clinical, electrophysiologic, and structural MRI data from 20 patients, noting highest accuracy (of 95%) in predicting Engel I outcomes using a Least-Square SVM classifier.⁷⁷ In contrast, a study utilizing an SVM classifier only on T1-weighted MRI sequences from 49 patients demonstrated 100% sensitivity and 88%–92% specificity (in male and female cohorts, respectively) in predicting post-surgical seizure freedom.⁷⁸ More recently, a neural network classifier examining DTI-based structural connectomes from 50 TLE patients yielded a positive predictive value of 88% and negative predictive value of 79% in predicting Engel I outcomes, comparing favorably to a discriminant function classifier using only clinical variables.⁷⁹ Measures of thalamocortical connectivity on resting-state fMRI have similarly been used to predict favorable response (>50% seizure reduction) to vagus nerve stimulation, achieving an AUC of 86% using an SVM classifier in 21 pediatric patients.⁸⁰ Antony et al, in contrast, applied an SVM classifier to measures of functional connectivity from stereo-EEG in 23 patients undergoing anterior temporal lobectomy, achieving a sensitivity of 90% and specificity of 85% in predicting seizure freedom at one year of follow-up.⁸¹ Tomlinson et al also examined functional connectivity using interictal intracranial EEG recordings from 17 pediatric patients, achieving 100% sensitivity and 87.5% specificity in predicting Engel I outcomes with an SVM classifier.⁸² Interestingly, however, a retrospective study of 118 patients found that neither MRI nor routine, video, or intracranial EEG data predicted post-surgical seizure freedom at 2 years using conditional logistic regression,⁸³ with a separate study similarly failing to predict seizure freedom from FDG-PET and ¹¹C-flumazenil-PET data assessed with a random forest classifier.⁸⁴ Nevertheless, when viewed as a whole, these studies demonstrate the ability of machine learning techniques to uncover prognostically valuable trends within the complex, multimodality data obtained during a typical presurgical evaluation, potentially allowing for improved patient selection and counseling.

APPLICATIONS IN MEDICAL MANAGEMENT OF EPILEPSY

As with surgical planning and prediction of surgical outcomes, machine learning techniques have also been applied to medical decision-making and prediction of medical outcomes in epilepsy. Aslan et al, for example, trained a neural network classifier using seven clinical features (including age of onset, history of febrile seizures, and presentation to clinic >1 year after disease onset) from 302 patients, achieving an accuracy of 91.1% in 456 test cases for predicting seizure freedom, seizure reduction, or absence of significant change (i.e., <50% reduction) in seizure frequency.⁸⁵ Cohen et al instead examined clinic visit notes of 200 pediatric patients (half undergoing epilepsy surgery) to predict surgical candidacy using Naïve Bayes and SVM classifiers, finding comparable performance to a panel of four neurologists up to several months before actual referral; notably, however, this comparison was limited by high measures of dispersion around the mean predictions.⁸⁶ In contrast, Kimiskidis et al examined features derived from paired-pulse transcranial magnetic stimulation-EEG recordings using a Naïve Bayes classifier, achieving a mean sensitivity of 86% and specificity of 82% in distinguishing patients with genetic generalized epilepsies from controls, as well as a mean sensitivity of 80% and specificity of 73% in predicting seizure freedom at 12 months’ follow-up.⁸⁷ More recently, An et al compared machine learning algorithms for prediction of drug-resistant epilepsy (defined as requiring >3 medication changes during the study period) utilizing comprehensive U.S. claims data from 2006–2015. The authors found that the best-performing algorithm, a random forest classifier trained using 635 features (comprising demographic variables, comorbidities, treatment regimens, insurance data, and clinical encounters) from 175,735 records, yielded an AUC of 76.4% and was able to identify patients with drug-resistant epilepsy an average of 1.97 years before failing a second medication trial, using data available at the time of the first medication prescription.⁸⁸

A number of studies have also demonstrated the capabilities of machine learning algorithms in predicting individual medication responses. Devinsky et al examined clinical characteristics (e.g., type and number of medications used, age, and comorbidities) from records of 34,990 patients extracted from a medical claims database, and trained a random forest classifier to predict a medication regimen least likely to require changes in the following 12 months (serving as a proxy of regimen efficacy and tolerability). In a testing set of 8,292 patients, the classifier’s predicted regimens, which yielded an AUC of 72%, might have resulted in 281.5 fewer hospitalization days and fewer physician visits per year if prescribed at the time of the prediction, though only matched the prescribed regimen in 13% of cases.⁸⁹ Another study examining six EEG features before and after a medication change in 20 pediatric epilepsy patients achieved 85.71% sensitivity and 76.92% specificity in predicting subsequent treatment response (i.e., >50% reduction in seizure frequency) using an SVM classifier.⁹⁰ In a similar study, clinical data (e.g., age at onset, seizure frequency, family history, and abnormal imaging) were combined with EEG features (sample entropy of the α, β, δ, and θ frequency bands) from 36 newly-diagnosed patients started on levetiracetam monotherapy to predict Engel I outcomes, achieving a sensitivity of 100% and specificity of 80.0% (AUC 96%) using an SVM classifier.⁹¹ Petrovski et al notably investigated five single nucleotide polymorphisms in 115 patients using a k-NN classifier in order to predict seizure freedom at one year of follow-up, achieving a sensitivity of 91% and specificity of 82% in the cohort of newly-diagnosed epilepsy patients as well as a sensitivity and specificity of 81% in a testing cohort of 108 patients on chronic pharmacotherapy.⁹²

Aside from reductions in seizure frequency, machine learning techniques have also shed light on a number of pertinent clinical outcomes. Paldino et al found that a random forest classifier achieved 100% sensitivity and 95.4% specificity in predicting language impairment using DTI-based whole-brain tractography data from 33 pediatric patients with malformations of cortical development.⁹³ A later study demonstrated that a random forest classifier trained on resting-state fMRI images from 45 pediatric epilepsy patients was also able to estimate disease duration with high accuracy (correlating with true disease duration with r = 0.95, p = 0.0004), which in turn was inversely correlated with full-scale intelligence quotient.⁹⁴ Piña-Garza et al investigated healthcare utilization in patients with Lennox-Gastaut syndrome using a multi-state Medicaid claims database, utilizing a random forest classifier to identify probable patients based on clinical variables (e.g., prescriptions for felbamate or clobazam, vagus nerve stimulator placement, corpus callosotomy, helmet use, or claims for intellectual disability); aside from finding higher lifetime healthcare costs, particularly for home-based or long-term care, the authors found lower rates of clobazam or rufinamide use in older patients, raising questions of suboptimal management in this cohort.⁹⁵ Grinspan et al, examining demographic, insurance, comorbidity, and medication data in medical records at two pediatric referral centers, demonstrated that a random forest classifier achieved AUCs of 84.1% and 73.4% at each center in predicting emergency department visit rates for the following year.⁹⁶ The range of clinical outcomes examined in these studies, from seizure rates to medication response to healthcare utilization, highlights the potential utility of machine learning techniques in clinical practice.

LIMITATIONS AND FUTURE DIRECTIONS

Although machine learning techniques have demonstrated high discriminatory ability in a wide range of applications in epilepsy, there is considerably less experience in the literature with external validation studies. In one notable example, Shazadi et al⁹⁷ applied the five-SNP classifier described by Petrovski et al⁹² to two external cohorts recruited in the UK, finding that the algorithm was unable to predict treatment responses as in the original study. The classifier was similarly unable to predict outcomes when re-trained using a subset of the UK cohorts. The authors noted differences in medication use between the groups, finding high rates of carbamazepine and valproate use in the original cohort but a greater proportion of lamotrigine use in the UK cohorts, with the classifier demonstrating modest predictive power (when re-trained using the UK cohorts) only for patients treated with carbamazepine or valproate; notably, however, there was no clear functional relationship between the SNPs and these medications. The two study populations were also noted to vary in sample size, ethnic composition, and clinical setting (with the larger UK cohorts recruited into, and evaluated as part of, clinical trials).

The study by Shazadi et al. highlights several potential barriers to generalizability of machine learning applications, including training dataset size, confounding clinical variables, and variability in data collection and interpretation. As noted previously, smaller datasets increase the risk of overfitting, particularly for excessively complex models or those of insufficient complexity to handle outliers.¹⁹ Moreover, very small training sets may lead to ostensibly high classification accuracies by chance, regardless of the selected mapping function or cross-validation approach.⁹⁸ Recognizing the difficulty of accumulating large clinical datasets, a number of cloud-based repositories have been developed to allow for cross-institution data sharing⁹⁹ and centralized data quality assurance.¹⁰⁰ These approaches present an opportunity to create larger and more representative sample populations, while also facilitating consistency in data collection and classification, in an overall effort to develop more generalizable models. Robust external validation of these models, alongside advancements in interpretability, may lead to improved clinician confidence in these models and facilitate incorporation into clinical practice.

CONCLUSIONS

Mirroring the increasingly ubiquitous use of machine learning in business, industry, as well as several medical subspecialties, machine learning techniques have already enjoyed widespread application in challenging areas within epilepsy. As evidenced by the rapidly maturing literature in seizure detection, machine learning techniques show promise in expanding capabilities for inpatient and ambulatory seizure monitoring, and may provide an avenue for further progress in seizure forecasting. Advances in automated analysis of EEG and imaging data have similarly demonstrated the capabilities of machine learning in uncovering diagnostic and prognostic information currently inaccessible to expert review, creating opportunities for improved selection of pharmacotherapy, prediction of clinical outcomes, and surgical planning. The relatively limited experience in the literature from validation studies does raise questions about the generalizability of these models, however, and underscores the need for larger, more diverse datasets as well as increased investment in external validation studies. Despite these concerns, novel approaches, including use of automated machine learning to reduce barriers to entry as well as development of cross-institution, cloud-based data repositories to improve training datasets, highlight exciting avenues forward for developing increasingly robust and generalizable algorithms suitable for introduction into clinical practice.

KEY POINTS:

• Advances in machine learning, particularly in deep learning, have enabled increasingly widespread applications in medicine

• Machine learning has already demonstrated utility in seizure detection, imaging analysis, and prediction of medical and surgical outcomes

• Validation studies with larger, more diverse cohorts are needed to bridge the gap to widespread acceptance in clinical practice