Sage Against the Machine: Promise and Challenge of Artificial Intelligence in Epilepsy

Abstract

Multicenter Validation of a Deep Learning Detection Algorithm for Focal Cortical Dysplasia

Gill RS, Lee HM, Caldairou B, et al. Neurology. 2021 Oct 19;97(16):e1571-e1582. doi:10.1212/WNL.0000000000012698. Epub 2021 Sep 14. PMID: 34521691; PMCID: PMC8548962.

Background and Objective:

To test the hypothesis that a multicenter-validated computer deep learning algorithm detects MRI-negative focal cortical dysplasia (FCD).

Methods:

We used clinically acquired 3-dimensional (3D) T1-weighted and 3D fluid-attenuated inversion recovery MRI of 148 patients (median age 23 years [range 2-55 years]; 47% female) with histologically verified FCD at 9 centers to train a deep convolutional neural network (CNN) classifier. Images were initially deemed MRI-negative in 51% of patients, in whom intracranial EEG determined the focus. For risk stratification, the CNN incorporated bayesian uncertainty estimation as a measure of confidence. To evaluate performance, detection maps were compared to expert FCD manual labels. Sensitivity was tested in an independent cohort of 23 cases with FCD (13 ± 10 years). Applying the algorithm to 42 healthy controls and 89 controls with temporal lobe epilepsy disease tested specificity.

Results:

Overall sensitivity was 93% (137 of 148 FCD detected) using a leave-one-site-out cross-validation, with an average of 6 false positives per patient. Sensitivity in MRI-negative FCD was 85%. In 73% of patients, the FCD was among the clusters with the highest confidence; in half, it ranked the highest. Sensitivity in the independent cohort was 83% (19 of 23; average of 5 false positives per patient). Specificity was 89% in healthy and disease controls.

Discussion:

This first multicenter-validated deep learning detection algorithm yields the highest sensitivity to date in MRI-negative FCD. By pairing predictions with risk stratification, this classifier may assist clinicians in adjusting hypotheses relative to other tests, increasing diagnostic confidence. Moreover, generalizability across age and MRI hardware makes this approach ideal for presurgical evaluation of MRI-negative epilepsy. Classification of evidence: This study provides Class III evidence that deep learning on multimodal MRI accurately identifies FCD in patients with epilepsy initially diagnosed as MRI negative.

Commentary

If you are reading this commentary on your portable electronic device, chances are that you are operating a face or voice recognition application that uses artificial intelligence (AI) to access it and that your browser uses embodied AI technology to retrieve it. If, on the other hand, you are a Gutenbergian enthusiast patiently waiting for the Epilepsy Currents printed version to arrive at your doorstep, the editorial office likely deployed some form of AI to identify you as a consumer and the postman utilized an AI based navigation system to deliver it to you. In other words, AI is already widely adopted in every corner of our daily lives, ¹ even if it is not overtly apparent. Any field of medicine, including epilepsy, could not be an exception to this rule.

The current manuscript ² exemplifies the use of AI in the realm of neuroimaging of epilepsy. By coalescing data from 9 tertiary epilepsy centers worldwide, the investigators created a repository of 148 children and adults with histologically-proven, type II focal cortical dysplasia (FCD), half of whom were deemed to be non-lesional on visual analysis of their pre-surgical magnetic resonance imaging (MRI). Using the gray matter volume in their high resolution (3T), three-dimensional (3D), T1-weighted and fluid-attenuated inversion recovery (FLAIR) sequences as the input, they trained an algorithm that could facilitate detection of the FCDs through deep learning. By retrospectively applying this algorithm to the MRI-negative patients, they demonstrated sensitivity of 85% in detecting FCDs. Applying the algorithm on an independent cohort of 42 healthy controls and 89 controls with temporal lobe epilepsy (TLE) due to radiological and/or histologically proven hippocampal sclerosis (HS), they demonstrated a specificity of 89%.

Given that patients with MRI-negative, drug-resistant epilepsy (DRE) constitute a formidable challenge, ³ an automated detection algorithm for FCDs, one of the most commonly encountered culprits, holds significant potential to transform clinical practice. This is not the first, and it is certainly not going to be the last of a series of studies utilizing AI to detect potentially epileptogenic lesions in MRI-negative patients. Compared to prior attempts, this study utilized 3D images rather than surface-based, two-dimensional methods. It also implemented a probabilistic strategy in detecting FCDs rather than the most-commonly deployed dichotomous approach. Its main advantage though lies in the large representation of well-curated, clinical, radiological and histological data of FCD-positive patients across the age spectrum and around the globe. The technical aspects of the created repository (eg segmented lesions co-registration, detection map performance evaluation) were performed meticulously by experts. A systematic approach in controlling for potential sources of bias (eg incorporation bias, spectrum bias, risk of overfitting, generalizability threats) and drawing inferences was deployed. ³

Despite the high quality data and rigorous methodological design leading to an overall sensitivity of 93% in the whole cohort with the leave-one-site-out cross-validation, an average of 6 false positives per patient were observed. This would not be a negligible time commitment for an expert neuroradiologist to scrutinize and, most importantly, not a trivial risk for a patient who may undergo additional electrodes implantation. Interestingly, lesions in the insula or the parahippocampal gyrus demonstrated the highest rates of false positives, perhaps accounting for the cytoarchitectonic similarities of these cortices with FCD histopathologic traits. ² Some of the false positives, particularly in the frontocentral lesions, constituted perilesional anomalies. Given the optimal surgical outcome in many of these cases, it is evident that this algorithm may detect histologically abnormal but clinically insignificant MRI-negative regions, highlighting the importance of additional paraclinical tests and, foremost, clinical acumen in utilizing this information. One of these tests, for example, would be MR spectroscopy for differentiating between FCDs from low-grade tumors that may mimic them but could be automatically detected by the proposed algorithm. ²

Looking at the false negatives, 6 of the 11 unresolved MRI-negative FCDs were situated in the orbitofrontal cortex, likely as a result of insufficient representation of FCDs in these regions in the training set, corroborating the need of high quality, large volume data in any AI paradigm. Interestingly, in 5 of the 11 undetected MRI-negative cases, the lesion could be identified through modulation of the probability threshold by incorporating seizure semiology and electrophysiology, attesting again to the importance of clinical reasoning and integration of any automated detection method with human oversight. Interestingly, the FCD detection sensitivity in the controlled group of TLE patients was similar between MRI-positive HS and MRI-negative HS cases, suggesting good specificity in patients with alternative lesions. ²

Besides FCD detection, the 2 main sub-fields of AI, namely machine learning (ie, data driven statistical modeling dependent on human interventions) and deep learning (ie, application of multiple layers of neural networks with limited human intervention), have been used in many other domains of imaging epilepsy and its comorbidities. ⁴ Distinguishing individuals with epileptic and psychogenic non epileptic seizures from healthy controls, classifying focal and generalized epilepsies, lateralizing and localizing the epileptogenic zone, detecting and predicting seizures and neurobehavioral disturbances, gauging response to antiseizure medications, dietary treatments, neuromodulation and postoperative seizure outcome are a few only examples. ^{4 -6} The majority of these studies used structural, diffusion and functional MRI or nuclear medicine techniques (eg positron emission tomography or single-photon emission computed tomography), but are typically fraught by small sample size, emphasis on adult patients with TLE, arduous engineering processing, limited reproducibility and poor generalizability. Beyond biomedical imaging, AI and computational approaches have been used in epilepsy to analyze clinical, neurophysiologic (eg electromyographic kinetic data, transcranial magnetic stimulation/magnetoencephalographic or scalp/intracranial video-electroencephalographic signals) and genomic/proteomic data for similar purposes. ^{7 -9} The same limitations apply to these studies compared to their neuroimaging counterparts.

In an era when physician shortages and burnout menaces are looming, what prevents AI from being fully integrated in the practice of epilepsy? And what can be done to materialize its hype and hope? Beyond the logistical aspects that require close interdisciplinary collaboration between health care providers, computer scientists and engineers, ⁷ accumulating large and diverse clinical datasets, and ensuring centralized data quality assurance is key in creating accurate and highly reproducible, externally validated algorithms. ⁹ The focus of any algorithm should be on clinically meaningful questions benchmarked against a widely accepted “ground truth” both for model input and output. ¹⁰ In parallel, the principles of human autonomy, transparency, accountability, inclusivity and sustainability need to be taken into account. ¹ Ethical and regulatory framework is paramount to avert the risk of medical negligence, prevent complacency that can threaten further innovation and ensure empathetic implementation. For any application to be widely adopted, it has to be freely available, user-friendly, prompt, readily incorporated into the workflow in real-time, and easily understood by its stakeholders, the patients and health care providers alike.

At the end of the day, the best use of artificial intelligence passes through human intelligence. In the “dilemma” of sage against the machine, it is evident that sage with the machine would be the indomitable answer. And as it always happens with major breakthroughs in human history, it will be the inventor and operator who will define the destiny of the tool as a blessing or a curse.