Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Mar 31.
Published in final edited form as: Epilepsia. 2021 Feb 2;62(Suppl 2):S106–S115. doi: 10.1111/epi.16786

Can Big Data guide prognosis and clinical decisions in epilepsy?

Xiaojin Li 1, Licong Cui 2, Guo-Qiang Zhang 1,2, Samden D Lhatoo 1
PMCID: PMC8011949  NIHMSID: NIHMS1683593  PMID: 33529363

Abstract

Big Data is no longer a novel concept in health care. Its promise of positive impact is not only undiminished, but daily enhanced by seemingly endless possibilities. Epilepsy is a disorder with wide heterogeneity in both clinical and research domains, and thus lends itself to Big Data concepts and techniques. It is therefore inevitable that Big Data will enable multimodal research, integrating various aspects of “-omics” domains, such as phenome, genome, microbiome, metabolome, and proteome. This scope and granularity have the potential to change our understanding of prognosis and mortality in epilepsy. The scale of new discovery is unprecedented due to the possibilities promised by advances in machine learning, in particular deep learning. The subsequent possibilities of personalized patient care through clinical decision support systems that are evidence-based, adaptive, and iterative seem to be within reach. A major objective is not only to inform decision-making, but also to reduce uncertainty in outcomes. Although the adoption of electronic health record (EHR) systems is near universal in the United States, for example, advanced clinical decision support in or ancillary to EHRs remains sporadic. In this review, we discuss the role of Big Data in the development of clinical decision support systems for epilepsy care, prognostication, and discovery.

Keywords: clinical decision support system, data science, machine learning

1 |. INTRODUCTION

Big Data, once a conceptually novel vision capturing the need for speedy handling of large volumes of disparate data, is now for many fields an unavoidable reality that forces change. Big Data can be described by the characteristics known as the “four V’s”: (1) volume, the quantity of generated and stored data; (2) variety, the types and nature of the data; (3) velocity, the speed at which the data are generated; and (4) veracity, the quality and value of the data.1,2 The footprint of the data landscape in 2015 was estimated at 4.4 zettabytes (ZB), and is projected to exceed 75 ZB in 2025.3 Taming this “data tsunami” has been a driving force behind the innovations that have emerged from places like Silicon Valley (social media and the wearable technologies industry being immediate examples), and there are increasing indications that the health care sector is embracing the change and impact of Big Data. Epilepsy, with its formidable phenotypic, genotypic, investigational, treatment, and research enterprise heterogeneities, lends itself to exciting opportunities and challenges in the Big Data era, and promises great returns. The reward from large-scale clinical and research collaborations is obvious, but their potential scale is unprecedented due to the possibilities promised by the advent of artificial intelligence (AI) and advanced machine learning approaches.4,5 For the vast majority of stakeholders in the epilepsy community, much of the promise lies in clinical decision support systems (CDSSs), which, among others, guide prognostication and management decision-making processes.6 A major goal is not only to inform decision-making, but also to reduce uncertainty in outcomes. Although the adoption of electronic health records (EHRs) is near universal in the United States, for example, CDSSs in or ancillary to EHRs are still not widely available. Even structured clinical documentation support tools are not in widespread EHR use.7 However, there is increasing recognition that electronic learning healthcare systems8 will become central to patient care in the future. In this paper, we present the current status and possibilities of the usage of Big Data tools and techniques for clinical decision support in epilepsy.

2 |. CLINICAL DECISION SUPPORT SYSTEMS, INTERPRETABILITY, AND ADAPTIVITY

Big Data technology provides powerful tools to integrate, process, and analyze large volumes of heterogeneous data collected for different purposes, including clinical care, management, and research.9 Changes and transformations brought about by Big Data directly affect design and implementation of CDSSs. CDSSs are commonly defined as computerized systems designed to directly aid clinical decision-making, in which characteristics of an individual patient are matched to computerized clinical knowledge bases,10 and patient-specific assessments or recommendations are then presented to the clinician for decision-making.6,11 The goal of CDSS is to help health professionals make clinical decisions, work with patient medical records, and in so doing, acquire or enhance knowledge of medicine necessary for interpretation of such data.12 Figure 1 shows the framework for capture and use of literature-based and practice-based evidence, showing relationships inherent to CDSSs to support evidence-based medicine.6 CDSSs are becoming essential tools for health care providers as Big Data technologies have enabled powerful instruments for acquisition and analysis of large amounts of comprehensive, heterogeneous data for clinical care, administration, and research.6 Recent years have witnessed significant advances in Big Data tools and techniques for clinical decision support in various domains such as radiology,13 breast cancer,14 and neurosurgery.15 Long-term evidence of utility and user acceptability is beginning to emerge,16 and randomized clinical trials of CDSSs suggest efficacy.17,18 In epilepsy, a major emphasis in development of CDSSs has been in epilepsy diagnostics.19

FIGURE 1.

FIGURE 1

Framework for capture and use of literature-based and practice-based evidence, showing relationships inherent to clinical decisions support systems to support evidence-based medicine.6

2.1 |. Interpretability

In the context of Big Data, CDSSs can be classified as either interpretable or uninterpretable. Interpretable CDSSs make the reasoning behind decision-making explicit and transparent, whereas uninterpretable CDSSs may provide recommendations without explainable, stepwise evidence. Rule-based CDSSs, a typical interpretable CDSS example, can represent expert understanding of clinical decision processes inherent to patient care, with scientifically established relationships. Rules are organized around literature-based, practice-based, or patient-directed evidence. CDSSs retrieve data with data retrieval tools to evaluate such rules and produce actions and outputs.13 However, rule-based systems cannot handle infinite complexity, noisy data, or unincluded features or variables, and are thus limited by the complexity of domain knowledge. This same complexity is a challenge in the epilepsy domain because of the sheer scale of epilepsy knowledge. Uninterpretable CDSSs, on the other hand, are often found in data-driven approaches that rely on deep learning techniques. Decisions in such systems can be more optimized, but gaps exist in appropriate explanations in the context of expert medical knowledge on how such decisions are derived. Development of effective explainable methods requires more interdisciplinary cooperation between professionals from different domains, such as machine learning researchers and medical experts,20 and the effectiveness of the interpretation of explainable CDSSs must be evaluated based on how its interpretation helps human users, which may require human-in-the-loop psychology experiments to measure mental models, task performance, user satisfaction, and appropriate trust.21

2.2 |. Evidence adaptivity

In evidence-adaptive CDSSs, clinical knowledge is derived from and continually reflects the most up-to-date evidence from research literature and practice-based sources.6 This is an exciting concept for epilepsy care, but is yet to be realized in practice. For example, a CDSS for epilepsy treatment is evidence adaptive if its knowledgebase (rule base in particular), and recommendations are continually updated, reflecting research progress and new discovery. However, a CDSS that alerts clinicians to a known antiepileptic drug–drug interaction is not evidence-adaptive if its knowledge base is scientific evidence based, but provides no tools or mechanisms to automatically incorporate new knowledge. Thus, evidence-adaptive CDSSs for epilepsy practice are to be striven for, although we are still far from realizing this ideal. Clinicians face barriers in reliable information handling, traversing multiple systems to acquire such knowledge for specific situations. The infrastructure for managing information, its effective application to clinical care, and use of the results for use in future scenarios are all challenges. Thus, many clinicians consider research literature to have limited applicability to their immediate clinical practices.22 However, examples exist on the contrary in specific domains as progresses in knowledgebase and CDSS are made.23 CDSSs have been developed for pressure injury, for example, with an expert knowledgebase using an interactive development environment, which is adaptive, with scalable capabilities for expansion to include other CDSSs and interoperability to interface with other existing clinical and administrative systems.24

2.3 |. Epilepsy CDSSs

Some examples of epilepsy CDSSs exist, in limited capacity, usually in a research setting. These include electroencephalographic (EEG) efforts from diagnosis of pediatric epilepsy based on scalp EEG signal25 to automated planning systems for stereo-EEG.26 EpiFinder is a CDSS tool that takes key terms from a patient’s history as input and uses standardized terminology and heuristic algorithms to produce a list of epilepsy diagnoses based on pattern recognition of a cluster of semiologies based on International League Against Epilepsy (ILAE)-defined epilepsy criteria.27 The usability and diagnostic accuracy of EpiFinder in differentiating between epilepsy syndromes and alternative diagnoses was evaluated in 53 adult patients from an epilepsy monitoring unit, where EpiFinder correctly identified the presence of epilepsy with a sensitivity of 86.4%, specificity of 85.1%, and accuracy of 86.8%. EpiSAT is a quantitative tool for seizure risk assessment. It can automatically identify the probability of changes in the frequency of patient-reported clinical seizures; the probability indicates that the susceptibility to seizures has worsened or improved.28 EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries and achieved 75.4% agreement in decision patterns with clinicians. Imaging efforts for CDSSs are likely to make inroads too, and although these are still at an early stage, there is significant promise of scale. The ENIGMA (Enhancing Neuroimaging Genetics through Meta Analysis, http://enigma.ini.usc.edu/) project’s ENIGMA-Epilepsy consortium, with 24 research centers, has analyzed structural brain changes in thousands of subjects, compared to matched healthy controls, in one of the largest neuroimaging studies of epilepsy ever carried out.29 Differences in broad regions of subcortical and cortical changes between temporal lobes have suggested differing pathophysiology between left and right hippocampal sclerosis patients.30 The power of this scale of collaboration lies in the inevitable utility of such information to eventually inform both CDSS-driven diagnosis (e.g., genetic generalized epilepsies vs, unclassifiable, focal, or multifocal epilepsies) and management (surgical candidacy, medication choices, etc.). Struck et al. validated 2HELPS2B score in a CDSS, which was designed to stratify seizure risk in hospital inpatients, as a clinical tool to help in seizure detection and seizure risk prediction with 2111 participants.31 Moffet et al. demonstrated that 2HELPS2B is a reliable and simple method to quantify these EEG findings and their associated risk of seizure based on accurate seizure risk forecasting of 1528 acute brain injury patients.32 Recent work in epilepsy management has shown that EHR-based CDSS tools result in a majority of physicians discussing sudden unexpected death in epilepsy (SUDEP) with patients, a desirable aspect to epilepsy care.33

3 |. BIG DATA BUILDING BLOCKS FOR CDSSs

3.1 |. Ontologies and data elements

In the Big Data era, increasingly large amounts of clinical data for epilepsy research have been produced at an unprecedented scale. Such data include clinical text (e.g., discharge summaries, EEG reports) and electrophysiological signals such as EEG and electrocardiography. This requires more efficient techniques to collect, manage, and analyze such large volumes of data for epilepsy clinical research.

Ontologies and terminology systems in machine-readable format are essential for taming the variety and veracity of Big Data. There have been a few efforts for constructing epilepsy-related ontologies. An ontology is a formal representation of knowledge within a specialty domain by a set of concepts (terms) and relationships between those concepts, often represented in Ontology Web Language. Four epilepsy-related ontologies are available in BioPortal, the world’s most comprehensive repository of biomedical ontologies: (1) Epilepsy Ontology, which consists of the epilepsy domain and epileptic seizures based on the diagnosis proposed by the ILAE34; (2) Epilepsy Semiology, which was designed to capture the semiology of epilepsy, including ictal, postictal, interictal, and aura signs35; (3) Epilepsy Syndrome Ontology, which contains epilepsy syndromes, seizure types, and data elements associated with them36; and (4) Epilepsy and Seizure Ontology (EpSO), which consists of more than 1300 concepts and integrates the latest ILAE recommendations and National Institute of Neurological Disorders and Stroke common data elements.37 EpSO has been successfully used to streamline data capture and integration processes and user interfaces, and to enable mapping across distributed databases to support federated queries, as well as to support centralized data curation while new data are continuously generated and integrated from multiple sites.38 For mobile health (mHealth), Goldenholz et al proposed common data elements to serve as a common language for mHealth apps and devices in epilepsy, which fill the gap in epilepsy mHealth data standards to improve the situation of epilepsy patients, clinicians, academics, and members of the industry.39 Development of new ontologies is an ongoing, iterative process as terminologies, data dictionaries, and classification systems evolve. There is no doubt, however, that epilepsy community efforts to invest in such development are key, because such ontologies will be essential for managing Big Data in epilepsy.

3.2 |. Machine learning

Machine learning combines statistics and computer science to create algorithms that “predict the future based on the past.”40 To leverage the practicality of machine learning in analyzing large and complex datasets, researchers and clinicians have paid great attention to the topic of automatic detection/prediction of epilepsy seizures in EEG recordings.4143 Various techniques have been applied to this task, including k-nearest neighbor,44 support vector machine,45 random forest,46 and deep learning classifiers.43,47 Intractability is an unfortunate and substantial issue in about one third of patients with epilepsy. Prospects of long-term seizure remission in those who fail the first two drugs for lack of efficacy reasons are dismal and are likely less than 5%.48 A law of diminishing returns exists in the natural history of intractability with each new drug change. In those such patients with focal epilepsy, epilepsy surgery can radically change the equation, and tools that help pinpoint the putative epileptogenic zone are a major field of research. CDSSs using signal analysis of high-frequency oscillations (HFOs) in intracranial EEG have been proposed for identification of surgical resection targets.49 Multiple biomarker approaches with AI have also been used, combining phase amplitude coupling, HFOs, and interictal epileptiform discharges to identify the ictal onset zones for surgical resection, without having to wait for actual seizures, thereby increasing accuracy, shortening monitoring periods, and tightening surgical timelines.50 As the role of HFOs and the significance of various patterns of ictal EEG onsets become clarified, the promise of machine learning approaches to harness this knowledge for everyday epilepsy surgery practice is easy to see.

In addition to signal analysis, machine learning is also widely used in other domains for clinical decision support in epilepsy. Kerr et al. proposed a method using multivariate piecewise-linear logistic regression to differentiate psychogenic nonepileptic seizures from epileptic seizures with different types of data, such as the frequency of specific patient-reported historical events, demographic information, age of onset, and delay from first seizure until video-EEG monitoring.51 Okazaki et al. proposed EpiFinder to extract keywords from patients’ medical records and dynamically generate a series of epilepsy syndromes, which could help triage and focus patient encounters with the ultimate goal of improving the diagnostic and therapeutic gap.27 Jin et al. developed a neural network classifier with magnetic resonance imaging data to identify solitary focal cortical dysplasias (FCDs) in a dataset of 61 patients with type II FCDs as well as 120 controls from three different epilepsy centers and achieved a sensitivity of 73.7%, specificity of 90.0%, and area under the curve (AUC) for the receiver operating characteristic analysis of .75.52 Goldenholz et al. proposed an approach with recurrent networks and multilayer perceptron to forecast the probability of future seizures; the evaluation results achieved an AUC of .86 on the testing set of 1613 patients.53 Abbasi and Goldenholz provided a detailed review of machine learning applications with different domains in epilepsy, such as automated seizure detection, analysis of imaging and clinical data, epilepsy localization, and prediction of medical and surgical outcomes.40

3.3 |. Natural language processing of epilepsy clinical text

CDSSs have the potential to be much more effective when they are integrated with data from EHRs to trigger alerts for situations that need physician action.12 EHRs’ structured clinical information (e.g., discharge diagnoses and pharmacy orders in coded form) as well as unstructured information such as radiology reports and discharge summaries in free text form can be extremely granular.54,55 Much of it is embedded in unstructured clinical text. Natural language processing (NLP) is therefore a technique that can facilitate extraction or identification of important facts from free text for clinical decision support.55 A few NLP tools have been developed for processing epilepsy-related clinical text. Two NLP systems were developed to extract epilepsy phenotypes from epilepsy monitoring unit phase reports for patient cohort identification: Epilepsy Data Extraction and Annotation (EpiDEA) and Phenotype Exaction in Epilepsy (PEEP).56,57 EpiDEA leveraged an open source NLP system called cTAKES (clinical Text Analysis and Knowledge Extraction System)58 as well as the EpSO-based regular expressions to extract epilepsy phenotypes, with a precision of 93.59%, recall of 84.01%, and F-measure of 88.53%. PEEP, developed on top of the National Library of Medicine MetaMap program,59 achieved a microaveraged precision of 92.4%, recall of 93.1%, and F-measure of 92.7%.57

Wissel et al. validated an NLP application to assign surgical candidacy scores to identify patients who are potentially eligible for presurgical evaluations for resective epilepsy surgery, achieving an AUC of .79, a sensitivity of .8, and a specificity of .77.60 Machine learning-based NLP methods have been used to identify West syndrome from discharge summaries and EEG reports,61 using different feature sets, with an eventual precision of 76.8%. Extraction of Epilepsy Clinical Text (ExECT) is another automated NLP system for extracting epilepsy information from clinic letters, including epilepsy diagnosis, epilepsy type, seizure type, and seizure frequency.37 ExECT was built on top of open source NLP software called GATE (General Architecture for Text Engineering), and achieved a precision of 91%. NLP tools have also been used for automatic detection of SUDEP risk factors from physician notes in EHRs, to generate electronic prompts for clinicians to counsel patients on SUDEP risk.37,62 Three SUDEP risk factors were targeted: generalized tonic-clonic seizures, refractory epilepsy, and epilepsy surgery candidacy. This tool was developed using notes on 2000 patients from a tertiary care epilepsy center and tested on notes on 1000 patients from the same medical center as well as notes on 1000 patients from five other medical centers in New York City. Tool performance applied in test data from the same medical center achieved F-measures ranging from .86 to .90, whereas those applied at other medical centers achieved a range from .53 to .81. Interestingly, the main cause of this decreased performance was the use of unique “boilerplate” standard text phrases in physician notes from other medical centers. Thus, NLP approaches hold promise as an important component of CDSSs, although significant challenges remain.

3.4 |. Data credibility and knowledge repositories

Credibility of data and knowledge repositories is an essential and fundamental component of CDSSs. The repository should contain high-quality data, which have to be timely, accurate, clean, and unbiased, and stored in an appropriate data schema/format.63 The data can be “dirtier” than what is ideal for research. There may also be hidden biases that affect conclusions of studies, and cleaning and reorganization of data may be difficult, time-consuming, and expensive. It is estimated that data scientists spend 80% of their time obtaining, cleaning, and preparing data, and only 20% of their time building models, analyzing, visualizing, and drawing conclusions from that data.64,65 Accessibility is important; it should be joinable (in a form that can be joined to other clinical data when necessary) and shareable (a data-sharing culture within the hospital ecosystem so that data can be joined).66 If clinicians/researchers do not have a coherent, accurate picture of patient flow, diagnostic processes, and complete longitudinal data acquisition processes of patients, it is hard to analyze and improve processes and care. Finally and very importantly, data should be queryable, and appropriate tools should be developed to interact with, make sense of, and “slice and dice” the data. Virtually all analyses performed require filtering, grouping, and aggregating data to reduce large amounts of raw data into smaller sets of higher level and analysis-ready data that help clinicians/researchers to gain insight into topics of interest.63 Ensuring data credibility and enhancing quality of knowledge repositories, while maintaining accessibility, are issues that are central to effective CDSS creation.

Several large-scale data repositories have been established in the past 15 years. The European Epilepsy Brain Bank was established at the University Hospital in Erlangen, Germany in 2006. This study collected 36 histopathological diagnoses during epilepsy surgery in 2623 children and 6900 adults from 36 centers in 12 European countries.67 The Center for SUDEP Research (CSR) is a National Institute of Neurological Disorders and Stroke-funded Center Without Walls initiative for collaborative research on epilepsy. It comprises researchers from 14 institutions across the United States and Europe, bringing together extensive and diverse expertise to understand SUDEP.68,69 The goal of CSR is to better understand cortical, subcortical, and brain stem mechanisms responsible for SUDEP and to use a data-driven, systems biology approach to elucidate the role of cortical influences in SUDEP. CSR provides a comprehensive, cu-rated repository of prospectively collected multimodal data, including electrophysiological signals in European data format. These data are linked to risk factor and outcome data of more than 2500 epilepsy patients with thousands of 24-h recordings.70

3.5 |. Interpretability of data-driven methods for knowledge discovery

Each machine learning approach has its strengths and weaknesses, constrained by the attributes and scope of available datasets.71 For example, to identify a classifier for seizure detection and imparting knowledge discovery, different approaches/classifiers have been applied and evaluated on EEG data. Literature shows that many data-driven methods have been developed using an algorithmic approach, sometimes with a set of heuristics, but without necessarily having an accompanying set of explicitly interpretable logical rules or patterns.72,73 In general, deep learning models tend to have better performance (e.g., high specificity and sensitivity or AUC) but often lack value addition in gaining insights for advancement of knowledge due to weak interpretability. How to make AI techniques more interpretable is an active research topic. To address this gap, explainable AI has become an emerging research field, with the goal of advancing interpretation of models to produce more interpretable results while maintaining performance.73 Several existing studies have made progress on explaining internal logic rules or patterns to understand what the algorithms are actually doing. Ribeiro et al. proposed a novel explanation technique that explains the predictions of a classifier.74 Xie et al. proposed a “field guide” to distill the important topics, related work, methods, and concerns associated with explainable deep learning.75 By providing appropriate explanation of patterns and rules, interpretable models allow a researcher to learn different statistical perspectives of features (e.g., brain signal) by analyzing statistical properties about them.42 Thus, interpretable machine learning models are important for advancing knowledge.

3.6 |. Validity of CDSSs

Combining Big Data and using machine learning technology to build CDSSs is a major research topic.15 A commonly used method for evaluating the performance of machine learning-based models is cross-validation. It involves dividing the original observation dataset into two nonoverlapping subsets: a training set for training the model and an independent testing set for evaluating the results of the model. For the cross-validation, evaluation metrics are required to quantify model performance. The choice of evaluation metrics depends on a given machine learning task, such as clustering, regression, or classification. The purpose of evaluation is to find a better solution to make the right clinical decision to reduce labor and time costs and improve work efficiency and patient experience. However, existing evaluation techniques may not be able to reflect the performance of the model in real-world clinical scenarios. For example, for epilepsy event detection in EEG recordings, most of the existing work applied signal-segment-based cross-validation methods, and the training and testing sets used for evaluation were class balanced. This implies that the numbers of positive and negative samples in both sets are the same or similar. However, class imbalance is one of the most challenging aspects in machine learning, particularly involving EEG signals. Extreme class imbalance may occur, because the duration of EEG recording is often long whereas the epilepsy event of interest may involve a few seconds.42 Some studies used imbalanced testing sets to evaluate the performance of their models and used traditional performance metrics of sensitivity, specificity, and F-measure. However, in practical scenarios, a model with high sensitivity, specificity, and F-measure (>90%) may not necessarily achieve completely satisfactory results when deployed in continuous EEG signal-recording settings.76 To develop and test robust and reliable machine learning approaches, cross-dataset verification is also an important method for evaluating and verifying performance.70,77 To leverage machine learning-based approaches, or even for other types of clinical decision support approaches, it is important to develop appropriate evaluation techniques that are better fitted to clinically relevant, real-world tasks, with performance metrics intuitive and acceptable to clinical domain experts.

4 |. PATIENT-REPORTED OUTCOMES FOR CDSSs

The use of online seizure diaries is now well-established. Examples include “My Seizure Diary,”78 “Seizure Tracker,”79 and “Texting 4 Control.”80 Accessible through desktops, lap-tops, and smartphone-based apps, these online diaries can record granular data, including seizure types, counts and patterns (ultradian, circadian, circaseptan, seasonal, etc.), medications, side effects, diet, mood, sleep, menstruation, vagal nerve stimulation therapy, compliance, and other features. Integration of such patient-reported outcomes (PRO) information with EHR poses challenges involving aspects such as interfacing disparate platforms, EHR vendors, and systems, and acting on the data thus provided. Algorithmic approaches to appropriate and timely PRO-based intervention may be possible, and are likely to improve, thereby streamlining physician responses and enhancing utility.62 However, pooled PRO data open up tremendous CDSS possibilities for chronotherapy, where accurate identification of an individual’s longer term seizure patterns can facilitate tailored drug treatment that targets periods of greater seizure likelihood.81 As the possibilities opened up by large-volume data collection expand, extremely granular treatment variations may become possible.

5 |. PROGNOSTICATION IN EPILEPSY

Studies of prognosis and mortality in epilepsy, whether retrospective or prospective, have previously been of limited scale, usually numbering in the hundreds of patients to a few thousand.48,8290 Although some data sources, such as claims data from the Centers for Medicare and Medicaid Services (United States), the Nationwide Inpatient Sample (United States), and the General Practice Research Database are examples of data sources with very large numbers (hundreds of thousands to millions) of patients, these vary in granularity and may not easily provide detailed answers to complex research questions. In the age of “-omics” research (Figure 2), an informatics-based approach to addressing the incredible complexity of research data currently generated in disparate epilepsy subdomains, harnessed to enable large-scale data collection that ultimately drives evidence-adaptive CDSSs, is the goal. The field has already risen to the challenge of data granularity as a means to answer comprehensive questions with codification efforts such as the ongoing Human Epilepsy Project (HEP 1, 2, and 3; http://www.humanepilepsyproject.org/), with its stated goals of defining epilepsy biomarkers, health outcomes, and health care utilization in epilepsy, and collection of detailed multimodal data (phenotype, DNA, electrophysiology, imaging, proteomics, cognition, etc.). The longer term goal of the epilepsy Big Data effort should focus on the realization of such multimodal data granularity, on the scale of what have traditionally been epidemiological studies. This will truly realize the promise of precision medicine in epilepsy.

FIGURE 2.

FIGURE 2

The emerging landscape of epilepsy Big Data for clinical decision support systems and individualized care. EHR, electronic health record

6 |. CONCLUSIONS

Big Data in epilepsy poses new opportunities as well as new unmet challenges. In epilepsy, Big Data technologies provide new powerful approaches to collect, manage, and analyze large volumes of data for epilepsy clinical research. This makes effective implementation of CDSSs in epilepsy possible, where health care practice and research play a synergistic role. CDSSs with Big Data in-the-loop provide novel options to support clinical diagnosis, therapy, and disease management. With information properly organized and utilized, scientific discovery and innovation will be empowered. It is therefore inevitable that these same attributes will enrich the foundation that guides our understanding of the epilepsy prognosis of each patient as they come forward for epilepsy care. Future generation of CDSSs needs careful attention to improvement of technologies for acquisition, quality, and machine learning readiness of data. This will in turn allow for wider adoption of such CDSSs for evidence-based adaptive decision support.

Key Points.

  • The Big Data approach provides powerful tools for both clinical care and research in epilepsy

  • CDSSs are a health information technology designed to assist clinical decision-making tasks

  • The creation of novel CDSSs using Big Data needs careful attention to implementation of data science for acquisition, storage, and access of data; this will allow high-quality acquisition and effective processing of routinely collected data, to facilitate the design, testing, and deployment of a data-in-the-loop, continuous learning, decision support ecosystem.

ACKNOWLEDGMENTS

This work was partially supported by the National Institute of Neurological Disorders and Stroke (NINDS) through grants U01NS090407, U01NS090408, and R01NS116287. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NINDS.

Footnotes

CONFLICT OF INTEREST

None of the authors has any conflict of interest to disclose.

ETHICAL APPROVAL

We confirm that we have read the Journal’s position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

REFERENCES

  • 1.Ramesh D The Origins of Big Data. KDnuggets. https://www.kdnuggets.com/2017/02/origins-bigdata.html. Accessed January 9, 2021.
  • 2.Onay C, Öztürk E. A review of credit scoring research in the age of Big Data. J Financial Regulation Compliance. 2018. [Google Scholar]
  • 3.Reinsel D, Gantz J, Rydning J. Data age 2025: the digitization of the world from edge to core. IDC White Paper Doc# US44413318. 2018:1–29. [Google Scholar]
  • 4.Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform. 2018;19:1236–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Wang F, Casalino LP, Khullar D. Deep learning in medicine—promise, progress, and challenges. JAMA Intern Med. 2019;179:293–4. [DOI] [PubMed] [Google Scholar]
  • 6.Sim I, Gorman P, Greenes RA, Haynes RB, Kaplan B, Lehmann H, et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc. 2001;8:527–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Narayanan J, Dobrin S, Choi J, Rubin S, Pham A, Patel V, et al. Structured clinical documentation in the electronic medical record to improve quality and to support practice-based research in epilepsy. Epilepsia. 2017;58:68–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Chambers DA, Feero WG, Khoury MJ. Convergence of implementation science, precision medicine, and the learning health care system: a new model for biomedical research. JAMA. 2016;315:1941–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Dagliati A, Marini S, Sacchi L, Cogni G, Teliti M, Tibollo V, et al. Machine learning methods to predict diabetes complications. J Diabetes Sci Technol. 2018;12:295–302. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Yu PP. Knowledge bases, clinical decision support systems, and rapid learning in oncology. J Oncol Pract. 2015;11:e206–11. [DOI] [PubMed] [Google Scholar]
  • 11.Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Shortliffe EH. Computer programs to support clinical decision making. JAMA. 1987;258:61–6. [PubMed] [Google Scholar]
  • 13.Syeda-Mahmood T Role of Big Data and machine learning in diagnostic decision support in radiology. J Am Coll Radiol. 2018;15:569–76. [DOI] [PubMed] [Google Scholar]
  • 14.Tsougos I, Vamvakas A, Kappas C, Fezoulidis I, Vassiou K. Application of radiomics and decision support systems for breast MR differential diagnosis. Comput Math Methods Med. 2018;2018:7417126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Buchlak QD, Esmaili N, Leveque JC, Farrokhi F, Bennett C, Piccardi M, et al. Machine learning applications to clinical decision support in neurosurgery: an artificial intelligence augmented systematic review. Neurosurg Rev. 2019;17:1–9. [DOI] [PubMed] [Google Scholar]
  • 16.Grout RW, Cheng ER, Carroll AE, Bauer NS, Downs SM. A six-year repeated evaluation of computerized clinical decision support system user acceptability. Int J Med Informatics. 2018;112:74–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Downs SM, Bauer NS, Saha C, Ofner S, Carroll AE. Effect of a computer-based decision support intervention on autism spectrum disorder screening in pediatric primary care clinics: a cluster randomized clinical trial. JAMA Netw Open. 2019;2:e1917676. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Wilkinson TA, Dixon BE, Xiao S, Tu W, Lindsay B, Sheley M, et al. Physician clinical decision support system prompts and administration of subsequent doses of HPV vaccine: a randomized clinical trial. Vaccine. 2019;37:4414–8. [DOI] [PubMed] [Google Scholar]
  • 19.Josephson CB, Sandy S, Jette N, Sajobi TT, Marshall D, Wiebe S. A systematic review of clinical decision rules for epilepsy. Epilepsy Behav. 2016;57:69–76. [DOI] [PubMed] [Google Scholar]
  • 20.Payrovnaziri SN, Chen Z, Rengifo-Moreno P, Miller T, Bian J, Chen JH, et al. Explainable artificial intelligence models using real-world electronic health record data: a systematic scoping review. J Am Med Inform Assoc. 2020;27:1173–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Gunning D, Aha DW. DARPA’s explainable artificial intelligence program. AI Magazine. 2019;40:44–58. [Google Scholar]
  • 22.Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. 1996. Clin Orthop Relat Res. 2007;455:3–5. [PubMed] [Google Scholar]
  • 23.Afzali M, Ballegaard C, Viggers S. A three-day course can increase knowledge and interest in disaster medicine for medical students. Ugeskr Laeger. 2014;176:1268–71. [PubMed] [Google Scholar]
  • 24.Khong PC, Lee LN, Dawang AI. Modeling the construct of an expert evidence-adaptive knowledge base for a pressure injury clinical decision support system. Informatics. 2017;4:20. [Google Scholar]
  • 25.Sargolzaei S, Cabrerizo M, Goryawala M, Eddin AS, Adjouadi M. Scalp EEG brain functional connectivity networks in pediatric epilepsy. Comput Biol Med. 2015;56:158–66. [DOI] [PubMed] [Google Scholar]
  • 26.Scorza D, Rizzi M, De Momi E, Cortes C, Bertelsen A, Cardinale F. Knowledge-based automated planning system for stereoelectroencephalography: a center-based scenario. J Biomed Inform. 2020;108:103460. [DOI] [PubMed] [Google Scholar]
  • 27.Okazaki EM, Yao R, Sirven JI, Crepeau AZ, Noe KH, Drazkowski JF, et al. Usage of EpiFinder clinical decision support in the assessment of epilepsy. Epilepsy Behav. 2018;82:140–3. [DOI] [PubMed] [Google Scholar]
  • 28.Chiang S, Goldenholz DM, Moss R, Rao VR, Haneef Z, Theodore WH, et al. Prospective validation study of an epilepsy seizure risk system for outpatient evaluation. Epilepsia. 2020;61:29–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Bearden CE, Thompson PM. Emerging global initiatives in neurogenetics: the enhancing neuroimaging genetics through meta-analysis (ENIGMA) consortium. Neuron. 2017;94:232–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Whelan CD, Altmann A, Botia JA, Jahanshad N, Hibar DP, Absil J, et al. Structural brain abnormalities in the common epilepsies assessed in a worldwide ENIGMA study. Brain. 2018;141:391–408. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Struck AF, Tabaeizadeh M, Schmitt SE, Ruiz AR, Swisher CB, Subramaniam T, et al. Assessment of the validity of the 2HELPS2B score for inpatient seizure risk prediction. JAMA Neurol. 2020;77:500–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Moffet EW, Subramaniam T, Hirsch LJ, Gilmore EJ, Lee JW, Rodriguez-Ruiz AA, et al. Validation of the 2HELPS2B seizure risk score in acute brain injury patients. Neurocritical care. 2020:1–7. [DOI] [PubMed] [Google Scholar]
  • 33.Downs S, Grout R, Buchhalter J, Patel A, Miller A, Clark A, et al. Clinical decision support to promote SUDEP discussions. Neurology. 2019;92:P5.6–051. [Google Scholar]
  • 34.Epilepsy Ontology. https://bioportal.bioontology.org/ontologies/ EPILONT. Accessed January 5, 2021.
  • 35.Epilepsy Semiology. https://bioportal.bioontology.org/ontologies/ EPISEM. Accessed January 5, 2021.
  • 36.Csernak LM. Evaluation of EpiFinder app: an epilepsy diagnostic tool. Arizona: Arizona State University;2016. [Google Scholar]
  • 37.Sahoo SS, Lhatoo SD, Gupta DK, Santos A, Claggett B, Pieske B, et al. Epilepsy and seizure ontology: towards an epilepsy informatics infrastructure for clinical research and patient care. J Am Med Inform Assoc. 2014;21:82–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Cui L, Huang Y, Tao S, Lhatoo SD, Zhang GQ. ODaCCI: ontology-guided data curation for multisite clinical research data integration in the NINDS Center for SUDEP Research. AMIA Annu Symp Proc. 2016;2016:441–50. [PMC free article] [PubMed] [Google Scholar]
  • 39.Goldenholz DM, Moss R, Jost DA, Crone NE, Krauss G, Picard R, et al. Common data elements for epilepsy mobile health systems. Epilepsia. 2018;59:1020–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Abbasi B, Goldenholz DM. Machine learning applications in epilepsy. Epilepsia. 2019;60:2037–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Ibrahim F, El-Gindy SA, El-Dolil SM, El-Fishawy AS, El-Rabaie ES, Dessouky MI, et al. A statistical framework for EEG channel selection and seizure prediction on mobile. Int J Speech Technol. 2019;22:191–203. [Google Scholar]
  • 42.Siddiqui MK, Morales-Menendez R, Huang X, Hussain N. A review of epileptic seizure detection using machine learning classifiers. Brain Inform. 2020;7:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Yu Z, Nie W, Zhou W, Xu F, Yuan S, Leng Y, et al. Epileptic seizure prediction based on local mean decomposition and deep convolutional neural network. J Supercomput. 2020;76:3462–76. [Google Scholar]
  • 44.Fergus P, Hussain A, Hignett D, Al-Jumeily D, Abdel-Aziz K, Hamdan H. A machine learning system for automated whole-brain seizure detection. Appl Comput Informat. 2016;12:70–89. [Google Scholar]
  • 45.Mahmoodian N, Boese A, Friebe M, Haddadnia J. Epileptic seizure detection using cross-bispectrum of electroencephalogram signal. Seizure. 2019;66:4–11. [DOI] [PubMed] [Google Scholar]
  • 46.Bhattacharyya A, Pachori RB. A multivariate approach for patient-specific EEG seizure detection using empirical wavelet transform. IEEE Trans Biomed Eng. 2017;64:2003–15. [DOI] [PubMed] [Google Scholar]
  • 47.Zhou M, Tian C, Cao R, Wang B, Niu Y, Hu T, et al. Epileptic seizure detection based on EEG signals and CNN. Front Neuroinformat. 2018;12:95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Chen Z, Brodie MJ, Liew D, Kwan P. Treatment outcomes in patients with newly diagnosed epilepsy treated with established and new antiepileptic drugs: a 30-year longitudinal cohort study. JAMA Neurol. 2018;75:279–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Sumsky SL, Santaniello S. Decision support system for seizure onset zone localization based on channel ranking and high-frequency EEG Activity. IEEE J Biomed Health Inform. 2019;23:1535–45. [DOI] [PubMed] [Google Scholar]
  • 50.Varatharajah Y, Berry B, Cimbalnik J, Kremen V, Van Gompel J, Stead M, et al. Integrating artificial intelligence with real-time intracranial EEG monitoring to automate interictal identification of seizure onset zones in focal epilepsy. J Neural Eng. 2018;15:046035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Kerr WT, Janio EA, Braesch CT, Le JM, Hori JM, Patel AB, et al. An objective score to identify psychogenic seizures based on age of onset and history. Epilepsy Behav. 2018;80:75–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Jin B, Krishnan B, Adler S, Wagstyl K, Hu W, Jones S, et al. Automated detection of focal cortical dysplasia type II with surface-based magnetic resonance imaging postprocessing and machine learning. Epilepsia. 2018;59:982–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Goldenholz DM, Goldenholz SR, Romero J, Moss R, Sun H, Westover B. Development and validation of forecasting next reported seizure using e-diaries. Ann Neurol. 2020;88:588–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Demner-Fushman D, Antani S, Simpson M, Thoma GR. Annotation and retrieval of clinically relevant images. Int J Med Inform. 2009;78:e59–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Demner-Fushman D, Chapman WW, McDonald CJ. What can natural language processing do for clinical decision support? J Biomed Inform. 2009;42:760–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Cui L, Bozorgi A, Lhatoo SD, Zhang GQ, Sahoo SS. EpiDEA: extracting structured epilepsy and seizure information from patient discharge summaries for cohort identification. AMIA Annu Symp Proc. 2012;2012;1191–200. [PMC free article] [PubMed] [Google Scholar]
  • 57.Cui L, Sahoo SS, Lhatoo SD, Garg G, Rai P, Bozorgi A, et al. Complex epilepsy phenotype extraction from narrative clinical discharge summaries. J Biomed Inform. 2014;51:272–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Savova GK, Masanz JJ, Ogren PV, Zheng J, Sohn S, Kipper-Schuler KC, et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc. 2010;17:507–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Aronson AR. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp. 2001;17–21. [PMC free article] [PubMed] [Google Scholar]
  • 60.Wissel BD, Greiner HM, Glauser TA, Holland-Bouley KD, Mangano FT, Santel D, et al. Prospective validation of a machine learning model that uses provider notes to identify candidates for resective epilepsy surgery. Epilepsia. 2020;61:39–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Sullivan R, Yao R, Jarrar R, Buchhalter J, Gonzalez G. Text classification towards detecting misdiagnosis of an epilepsy syndrome in a pediatric population. AMIA Annu Symp Proc. 2014;2014:1082–7. [PMC free article] [PubMed] [Google Scholar]
  • 62.Barbour K, Hesdorffer DC, Tian N, Yozawitz EG, McGoldrick PE, Wolf S, et al. Automated detection of sudden unexpected death in epilepsy risk factors in electronic medical records using natural language processing. Epilepsia. 2019;60:1209–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Zhang GQ, Cui L, Mueller R, Tao S, Kim M, Rueschman M, et al. The National Sleep Research Resource: towards a sleep data commons. J Am Med Inform Assoc. 2018;25:1351–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Press G Cleaning big data: Most time-consuming, least enjoyable data science task, survey says. Forbes, March. 2016;23:15. [Google Scholar]
  • 65.Patel J Bridging data silos using Big Data integration. Int J Database Manag Syst 2019;11:01–6. [Google Scholar]
  • 66.Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016;3:160018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Blumcke I, Spreafico R, Haaker G, Coras R, Kobow K, Bien CG, et al. Histopathological findings in brain tissue obtained during epilepsy surgery. N Engl J Med. 2017;377:1648–56. [DOI] [PubMed] [Google Scholar]
  • 68.Lhatoo SD, Noebels J, Whittemore V. NINDS Center for SUDEP Research. Sudden unexpected death in epilepsy: identifying risk and preventing mortality. Epilepsia. 2015;56:1700–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Lhatoo SD, Nei M, Raghavan M, Sperling M, Zonjy B, Lacuey N, et al. Nonseizure SUDEP: sudden unexpected death in epilepsy without preceding epileptic seizures. Epilepsia. 2016;57:1161–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Li X, Huang Y, Tao S, Cui L, Lhatoo SD, Zhang GQ. SeizureBank: a repository of analysis-ready seizure signal data. AMIA Annu Symp Proc. 2019;2019:1111. [PMC free article] [PubMed] [Google Scholar]
  • 71.Fernandez-Delgado M Do we need hundreds of classifiers to solve real world classification problems? J Machine Learn Res. 2014;15:3133–81. [Google Scholar]
  • 72.Matheny M, Israni ST, Ahmed M, Whicher D. Artificial intelligence in health care: the hope, the hype, the promise, the peril. Natl Acad Med. 2020;94–7. [PubMed] [Google Scholar]
  • 73.Carvalho DV, Pereira EM, Cardoso JS. Machine learning interpretability: a survey on methods and metrics. Electronics. 2019;8:832. [Google Scholar]
  • 74.Ribeiro MT, Singh S, Guestrin C .“Why should I trust you?” Explaining the predictions of any classifier. In: KDD ‘16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY: Association for Computing Machinery; 2016. p. 1135–44. [Google Scholar]
  • 75.Xie N, Ras G, van Gerven M, Doran D. Explainable deep learning: a field guide for the uninitiated. arXiv. 2004;14545. [Google Scholar]
  • 76.Li X, Tao S, Jamal-Omidi S, Huang Y, Lhatoo SD, Zhang GQ, et al. Detection of postictal generalized electroencephalogram suppression: random forest approach. JMIR Med Inform. 2020;8:e17061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Abou Jaoude M, Sun H, Pellerin KR, Pavlova M, Sarkis RA, Cash SS, et al. Expert-level automated sleep staging of long-term scalp electroencephalography recordings using deep learning. Sleep. 2020;43(11):zsaa112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Epilepsy Foundation. My Seizure Diary. https://diary.epilepsy.com/. Accessed January 5, 2021.
  • 79.Seizure Tracker LLC. Seizure Tracker. https://www.seizuretracker.com/index.php. Accessed January 5, 2021.
  • 80.Shegog R, Bamps YA, Patel A, Kakacek J, Escoffery C, Johnson EK, et al. Managing epilepsy well: emerging e-tools for epilepsy self-management. Epilepsy Behav. 2013;29:133–40. [DOI] [PubMed] [Google Scholar]
  • 81.Sanchez Fernandez I, Loddenkemper T. Chronotherapeutic implications of cyclic seizure patterns. Nat Rev Neurol. 2018;14:696–7. [DOI] [PubMed] [Google Scholar]
  • 82.Annegers JF, Hauser WA, Shirts SB, Kurland LT. Factors prognostic of unprovoked seizures after febrile convulsions. N Engl J Med. 1987;316:493–8. [DOI] [PubMed] [Google Scholar]
  • 83.Berg AT, Shinnar S, Hauser WA, Alemany M, Shapiro ED, Salomon ME, et al. A prospective study of recurrent febrile seizures. N Engl J Med. 1992;327:1122–7. [DOI] [PubMed] [Google Scholar]
  • 84.Hauser WA, Rich SS, Lee JR, Annegers JF, Anderson VE. Risk of recurrent seizures after two unprovoked seizures. N Engl J Med. 1998;338:429–34. [DOI] [PubMed] [Google Scholar]
  • 85.Cockerell OC, Johnson AL, Sander JW, Hart YM, Shorvon SD. Remission of epilepsy: results from the National General Practice Study of Epilepsy. Lancet. 1995;346:140–4. [DOI] [PubMed] [Google Scholar]
  • 86.Cockerell OC, Johnson AL, Sander JW, Shorvon SD. Prognosis of epilepsy: a review and further analysis of the first nine years of the British National General Practice Study of Epilepsy, a prospective population-based study. Epilepsia. 1997;38:31–46. [DOI] [PubMed] [Google Scholar]
  • 87.Lhatoo SD, Johnson AL, Goodridge DM, MacDonald BK, Sander JW, Shorvon SD. Mortality in epilepsy in the first 11 to 14 years after diagnosis: multivariate analysis of a long-term, prospective, population-based cohort. Ann Neurol. 2001;49:336–44. [PubMed] [Google Scholar]
  • 88.Mahler B, Carlsson S, Andersson T, Adelow C, Ahlbom A, Tomson T. Unprovoked seizures after traumatic brain injury: a population-based case-control study. Epilepsia. 2015;56: 1438–44. [DOI] [PubMed] [Google Scholar]
  • 89.Nilsson L, Ahlbom A, Farahmand BY, Tomson T. Mortality in a population-based cohort of epilepsy surgery patients. Epilepsia. 2003;44:575–81. [DOI] [PubMed] [Google Scholar]
  • 90.Sillanpaa M, Shinnar S. Long-term mortality in childhood-onset epilepsy. N Engl J Med. 2010;363:2522–9. [DOI] [PubMed] [Google Scholar]

RESOURCES