Multi-scale deep learning of clinically acquired multi-modal MRI improves the localization of seizure onset zone in children with drug-resistant epilepsy

Abstract

The present study investigates the effectiveness of a deep learning neural network for non-invasively localizing the seizure onset zone (SOZ) using multi-modal MRI data that are clinically acquired from children with drug-resistant epilepsy. A cortical parcellation was applied to localize the SOZ in cortical nodes of the epileptogenic hemisphere. At each node, the laminar surface analysis was followed to sample 1) the relative intensity of gray matter and white matter in multi-modal MRI and 2) the neighboring white matter connectivity using diffusion tractography edge strengths. A cross-validation was employed to train and test all layers of a multi-scale residual neural network (msResNet) that can classify SOZ node in an end-to-end fashion. A prediction probability of a given node belonging to the SOZ class was proposed as a non-invasive MRI marker of seizure onset likelihood. In an independent validation cohort, the proposed MRI marker provided a very large effect size of Cohen’s d = 1.21 between SOZ and non-SOZ, and classified SOZ with a balanced accuracy of 0.75 in lesional and 0.67 in non-lesional MRI groups. The subsequent multi-variate logistic regression found the incorporation of the proposed MRI marker into interictal intracranial EEG (iEEG) markers further improves the differentiation between the epileptogenic focus (defined as SOZ resected during surgery) and non-epileptogenic sites (i.e., non-SOZ sites preserved during surgery) up to 15% in non-lesional MRI group, suggesting that the proposed MRI marker could improve the localization of epileptogenic foci for successful pediatric epilepsy surgery.

I. INTRODUCTION

RESECTIVE surgery is an outstanding option for children whose focal seizures do not respond to medical therapy [1]–[3]. Invasive intracranial EEG (iEEG) recording via subdural or stereotactic depth electrodes is a clinical gold standard to localize seizure onset zones (SOZ) [4]–[6] responsible for habitual seizures. The most common practice is to remove the SOZ and neighboring MRI lesions (if present). By definition [1], investigators can conclude that the epileptogenic zone was sufficiently removed and that the SOZ was a part of the epileptogenic zone if a given patient achieved seizure freedom following epilepsy surgery. The present study aimed to determine the effectiveness of advanced deep learning approach using multi-modal MRI analysis to non-invasively localize the SOZ in children with drug-resistant epilepsy. To warrant trustworthy deep learning of structural abnormalites exclusively associated with the SOZ, we studied multi-modal MRI data obtained from children who achieved Internatioal League Against Epilepsy (ILAE) class 1 outcome: seizure freedom following resective surgery [7]. Thus, the present study is novel and innovative, since it investigates preoperative MRI data of such a selected patient population to demonstrate the promise of current deep learning-based SOZ localization with the highest confidence without the uncertainty of undetected epileptogenic zone causing postsurgical seizures (i.e., ILAE classes 2–6 outcomes).

Prior studies [8]–[11] have shown that the success rates of SOZ localization using clinical MRI are highly variable. Cases without lesions on clinical MRI are among the most challenging populations [12]. SOZ could look normal or appear T2-hyperintense and/or T1-hypointense in such non-lesional cases [13]. Subtle changes were rarely reported in fluid-attenuated inversion recovery (FLAIR) intensity, grey matter density, and diffusivity, but these were not closely related to the presence of neuronal loss or abnormal myelin [14]. Diffusion weighted imaging connectome (DWIC) analysis has also been used to extract a potent imaging marker of epileptogenicity based on the elevation of regional connectivity near SOZ. For instance, increased clustering and regional efficiency at the SOZ were found in the intra-limbic connectivity network of medial temporal lobe epilepsy patients [15]. Increased local clustering in regions close to the SOZ, together with a breakdown of long-range connectivity, have also been suggested by whole-brain connectome analysis [16], and these disrupted patterns of axonal connectivity were supported by a resting-state functional MRI study [17]; this study reported that MRI-negative temporal lobe epilepsy was associated with reduced connectivity at the ipsilateral superior and middle temporal gyri compared with healthy controls. However, none of the intensity, diffusivity, or connectivity features alone were reported to localize the SOZ with generalizability due to the heterogeneous nature of epileptogenicity [8], [18].

Recent studies [19], [20] have consistently reported that the absence of an MRI-lesion is significantly associated with lower chances of postoperative seizure freedom. The goal of the present study is to determine how well the SOZ can be localized by the state-of-the-art deep learning of clinically acquired multi-modal MRI features, which can efficiently extract ”data-driven knowledge” of across-patient epileptogenicity from multi-modal MRI abnormalities in cortical, subcortical, and white matter structures as compared to the iEEG data. We propose a variant of a multi-scale residual neural network (msResNet, https://github.com/geekfeiw/Multi-Scale-1D-ResNet) [21] to deeply learn specific patterns of 1) multi-modal MRI abnormalities that are measured from iEEG-defined SOZ sites of two different groups (i.e., lesional and non-lesional on clinical MRI visual assessment) and 2) iEEG-defined non-SOZ sites of non-epileptogenic tissues showing normal MRI appearance, and use the learned patterns to classify SOZ and non-SOZ in given multi-modal MRI data. This classification is achieved by objectively abstracting (and mining) specific patterns of SOZ-related changes in surface-based multi-modal MRI features (relative change of T1-weighted, T2-weighted, FLAIR, DWI diffusivity, fiber density, and DWIC connectivity), that are not readily identified by routine MRI visual inspection.

The present study hypothesizes that comprehensive multi-scale deep learning of multiple multi-modal MRI features contrasted by iEEG-defined SOZ and non-SOZ would efficiently abstract a discriminative set of deep features from multi-modal MRI features that maximize their separation in two iEEG-defined classes: SOZ and non-SOZ. The resulting deep features will be used to generate a new marker for epileptogenicity, called “seizure onset likelihood”, which quantifies the probability of SOZ from given imaging features. We determined whether the accuracy of this non-invasive imaging marker to localize SOZ is comparable with that of invasive interictal iEEG markers [22], including high-frequency oscillations (HFO) at 80 Hz and above [23]–[26] and modulation index (MI), quantifying the strength of coupling between HFO and slow waves at 3–4 Hz [27], [28]. We subsequently determined whether this non-invasive MRI maker has an additive value to the aforementioned interictal iEEG markers, to evaluate its potential value for guiding the placement of intracranial electrodes and ultimately improving the seizure outcomes of resective epilepsy surgery, especially in clinical cases with non-lesional MRI.

The main contributions of our work are summarized as follows:

1. To the best of our knowledge, the present study is the first work that performs SOZ prediction for preoperative evaluation of pediatric epilepsy surgery using deep learning of multi-modal MRI features at the level of brain node. Our model (i.e., msResNet) can identify and leverage the complex patterns of the multi-modal MRI features obtained from epileptogenic hemisphere on preoperative MRI to make an accurate prediction of potential SOZ node.

2. The proposed deep learning network using clinical multi-modal MRI features provides a new non-invasive marker quantifying seizure onset likelihood in drug-resistant focal epilepsy. Although preliminary, the accuracy of the proposed marker is comparable with those of invasive iEEG markers in classifying the SOZ. Incorporating the proposed marker further improves the prediction of epileptogenic foci based on interictal iEEG markers in non-lesional MRI cases.

3. Using saliency visualization, we provide an explainable artificial intelligence (AI) model which can highlight the core MRI features of which changes are critical for SOZ predictions. This model may lead to an improved understanding and assessments of radiological features of epileptogenicity that are currently difficult to extract from routine MRI evaluation.

The rest of the paper is organized as follows: Section II describes study subjects, clinical data, and methodological details of surfce-based laminar analysis, msResNet model, and statistical analysis. Section III describes experimental results. Section IV presents discussion and future direction of our approach. Lastly, Section V presents our conclusion.

II. METHODS

A. Study Subjects

The present study recruited 41 young patients with focal epilepsy (age: 9.9±5.6 years, 22 boys) who underwent two-stage epilepsy surgery involving extraoperative iEEG recording via subdural grid and strip electrodes and achieved ILAE class 1 outcome as assessed at least 1 year after surgery. The inclusion criteria consisted of multi-modal MRI acquisition including DWI tractography before surgery. The exclusion criteria consisted of (i) diagnosis of bilateral multifocal epilepsy purely based on the Phase-1 non-invasive presurgical evaluation, (ii) callosotomy, hemispherotomy, or hemispherectomy, (iii) major brain malformations making the central or lateral sulcus unidentifiable, (iv) malignant brain tumor suspected on preoperative MRI, (v) postoperative follow-up shorter than 1 year, and (vi) history of previous epilepsy surgery. Patient demographics and clinical data are detailed in Supplementary Table 1. The recruited patients were randomly divided to match age and presence of lesion on MRI into 1) a model cohort (18/6 patients with lesional/non-lesional MRI based on a routine visual assessment by a neuroradiologist) that was used to train and test the proposed msResNet for accurate classification of SOZ vs. non-SOZ and 2) a validation cohort (12/5 patients with lesional/non-lesional MRI) that was used to evaluate the reproducibility of the trained msResNet for accurate classification of SOZ in an independent cohort consisting of lesional and non-lesional MRI cases. These cohorts did not differ in demographic or clinical variables of importance (Table I). The study was approved by the University Institutional Review Board (Wayne State University, 111014MP2F, March 9, 2022), and written informed consent was obtained from the patients and guardians of patients. Also, this study was registered as a clinical trial at https://clinicaltrials.gov/ (NCT04986683: Novel DWI Methods to Minimize Postoperative Deficits in Pediatric Epilepsy Surgery).

TABLE I.

Statistical difference between model and validation cohorts.

Demographic and clinical variable	p-value
Age	0.90
Sex	0.23*
Side of operated hemisphere	0.50*
Handedness	0.65*
Presence of lesion on MRI	0.75*
Number of oral anti-epileptic drugs immediately before surgery	0.59
Seizure frequency	0.36*
Seizure onset age	0.84

^*indicates Chi-square test p-value

B. iEEG Acquisition and extraction of iEEG markers

We used platinum subdural electrodes for the SOZ localization [4]. Video-iEEG signals were recorded for 2–7 days with an amplifier band-pass of 0.016–300 Hz and analysis was performed using common average reference [29]. The iEEG-defined SOZ (i.e., ground-truth SOZ) was defined as cortical areas initially generating sustained rhythmic discharges with spatial and/or morphological evolution and resulting in habitual seizure symptoms [4]. The remaining electrode sites were treated as the iEEG-defined non-SOZ. Among 41 patients in the study cohort, in 30 patients (18 in the model cohort and 12 in the validation cohort) we measured the following interictal iEEG biomarkers based on the eligibility criteria described in our previous study [22]. They provided a total of 5,232 electrode sites (SOZ: 533 and non-SOZ: 4,699) to quantify five different iEEG markers for SOZ localization, MI [27], [30] computed using the EEGLAB Toolbox winPACT (https://sccn.ucsd.edu/wiki/WinPACT, Supplementary Fig. 1) and rate of HFO events using RIPPLELAB (https://github.com/BSP-Uniandes/RIPPLELAB, Supplementary Fig. 1) with different detection algorithms, 1) short time energy (STE) [23], 2) short line length (SLL) [24], 3) Hilbert (HIL) method [25], and 4) Montreal Neurological Institute (MNI) method [26]. Each marker was evaluated at two high-frequency bands, 80–300 Hz and 150–300 Hz, yielding a total of 10 iEEG markers: MI_>80Hz, MI_>150Hz, STE_>80Hz, STE_>150Hz, SLL_>80Hz, SLL_>150Hz, HIL_>80Hz, HIL_>150Hz, MNI_>80Hz, and MNI_>150Hz. Details of the above iEEG markers are available in our recent work [22]. Subdural electrodes were spatially localized to corresponding cortical regions on preoperative MRI using neurosurgeon input, the Fieldtrip toolbox (https://www.fieldtriptoolbox.org/) to co-register post-implantation CT to MRI, post-implantation x-ray, and photographs taken during surgical electrode implantation.

C. MRI acquisition and extraction of multi-modal MRI features

All preoperative MRI scans were performed on a GE Signa 3T scanner equipped with an 8-channel head coil. Imaging parameters of each scan are available in Table II. Our MRI protocol followed clinical imaging guidelines for epilepsy [31], including T1-weighted, T2-weighted, and FLAIR that are widely used to measure biophysical changes related to proton relaxation. A DWI scan was also used to generate an apparent diffusion coefficient (ADC) map as a measure of cellularity [32], a fractional anisotropy (FA) map as a measure of white matter integrity [33], an apparent fiber density (AFD) map as a measure of intra-axonal volume [34], and whole-brain tractography for the DWIC analysis, which can measure the degree of white matter connectivity associated with brain reorganization favoring hyper-excitability and/or impaired white matter pruning processes [35], [36].

TABLE II.

MRI imaging parameters.

Modality	Parameters
T1-weighted	TR = 6.1 ms, TE = 2.4 ms, TI = 450 ms, Flip angle = 12°, slice thickness = 1.2 mm, Acquisition matrix = 256×256, FOV = 24 cm
T2-weighted	TR = 4000 ms, TE = 105.6 ms, Slice thickness = 5 mm, Acquisition matrix = 512×512, FOV = 22 cm
FLAIR	TR = 11,002 ms, TE = 169.4 ms, TI = 2,550 ms, Slice thickness = 5 mm, Acquisition matrix = 512×512, FOV = 24 cm
DWI/DWIC	TR = 12,500 ms, TE=75.5 ms, Slice thickness = 3 mm, Acquisition matrix = 128×128, FOV = 24 cm, 55 isotropic gradient directions with b = 1000 s/mm2, single b = 0 s/mm2 image

TR: repetition time, TE: echo time, TI: Inversion time, FOV: field of view.

The present study utilized a Lausanne 2008 cortical parcellation atlas [37] to compare multi-modal MRI data at anatomically equivalent locations. This atlas consists of 500 cortical nodes in an epileptogenic hemisphere (Ω_i=1–500) that are spatially placed from template T1-weighted image space to native T1-weighted image space via non-linear deformation. Briefly, all multi-modal MRI data were co-registered to native T1-weighted image space, which defines all necessary brain surfaces obtained by our previous laminar surface-based analysis [38], [39]: gray matter-pial surface, outer gray matter surface, middle gray matter surface, and 3 mm depth white matter surface (see an example in Fig. 1). At each surface segment of the given i^th node, Ω_i, multi-modal MRI data x_i consisted of both gray matter and white matter surface markers. For gray matter input data, T1-weighted, T2-weighted, FLAIR, ADC, and FA measures were sampled at two gray matter surfaces (located in 75% and 50% of gray matter thickness, approximating cortical layers II through IV where the cortical neurons are most concentrated [39]) and normalized by their mean and histogram peak values to generate relative intensity (RI) [40] measures, controlling for inter-subject inhomogeneity. For white matter input data, RI of FA, ADC, and AFD were sampled from the deep white matter surface, where any present pathologic changes would be most prevalent [38]. At each surface, 100 RI values were sampled at 100 equidistant vertices of the surface segment and taken as the input feature vector, x_i (i.e., the total number of values for Ω_i = 1,300, 100 values × 13 surface segments). DWIC data of x_i consisted of 500 intra-hemispheric edge strengths in the epileptogenic hemisphere. A 2^nd order integration over fiber orientation distribution algorithm was applied to reconstruct whole-brain tracts using anatomically constrained whole-brain tractography [41] with the same tracking parameters (angular threshold = 70°, seeding number/voxel = 10³, step size = 0.5 mm). The resulting tracts were sorted by each pair in Ω_i=1–500 to quantify 500 edge strengths using the MRtrix3 package (http://www.mrtrix.org/).) For DWIC features of x_i, the total count of fiber streamlines connecting Ω_i to other remaining nodes, Ω_j=1–500 was scaled by the total volume of Ω_i and Ω_j. Finally, the scaled counts were sorted in ascending order of inter-nodal streamline lengths (i.e., shorter to longer) and divided by their total counts to quantify the epileptogenic profile of the neighboring connectivity strength in Ω_i (i.e., 500 edge strengths sorted from the nearest node to the farthest node).

Fig. 1.

A. Scheme of surface-based laminar analysis to measure multi-modal MRI data of a given node Ω_i=250 at three brain surfaces of native T1-weighted image: outer gray matter surface (red line), middle gray matter surface (green line), and deep white matter surface (blue line). B. The plots to the right show an example of RI normalization: the T1-weighted image intensity histogram measured across the whole-brain surface vertices of F=(th×0.75) are shown on top, and the RI histogram measured at Ω_i=250 surface vertices of F=(th×0.75) are shown below.

D. Construction of deep learning network

Fig. 2 presents the detailed architecture of the msResNet that performs the node-wise classification for a given node Ω_i by estimating seizure onset likelihood, μ_i, from given input data x_i. To mine the most valuable features in the input data x_i obtained from a node Ω_i, we modified the original msResNet [21] so that it can process multi-modal MRI data of x_i with two identical residual network branches having two different sizes of convolution filtering: 1×3 and 1×7, one branch for morphological features and another branch for connectivity features. This modification was designated to abstract multi-modal features of x_i at the identical residual-based multi-scale formulation that highlights the global difference on a coarse scale and captures the local contrast on a fine scale. This multi-scale architecture of x_i unveils subtle changes that elude visual inspection by incorporating high-dimensional features (i.e., 1×1,800 per x_i) that suppress irrelevant variations and abstract subtle differences in the low-dimensional deep feature domain (i.e, 1×768 per k_i), especially between iEEG-defined SOZ (including lesional and non-lesional SOZ on clinical MRI visual assessment) and iEEG-defined non-SOZ (including non-SOZ electrode sites on clinical MRI visual assessment). From a given input x_i, a deep feature k_i was extracted and analyzed at the output layer that produced the probability vector [42],

$$P_i^{C_l}=softmax\left(k_i\cdot \mathbf{\text{w}}\right),$$

Fig. 2.

Detailed architecture of the proposed msResNet to estimate seizure onset likelihood, μ_i, from a given input data, x_i. Multi-modal MRI data of x_i consisted of 1) gray and white matter surface markers: relative intensity values of T1-weighted, T2-weighted, FLAIR, ADC, and FA each sampled at two laminar gray matter surfaces (located in 75% and 50% of gray matter thickness), and 2) white matter surface markers: relative intensity values of FA, ADC, and AFD at the deep white matter surface.

where $P_i^{C_l}$ is the predicted probability of the input x_i belonging to the class C_l, w (sized by 768×2) is the weight elements of a fully connected layer in Fig. 2, and · is the element-wise multiplication operator.

Adam [43], an adaptive learning rate approach for stochastic gradient descent, was utilized to minimize cross-entropy loss at the learning rate of 0.0001. After training, the fully connected layer produced two output probability values, $P_i^{C_1}$and $P_i^{C_2}$. If the value of $P_i^{C_1}$ is greater than that of $P_i^{C_2}$, the input of x_i was classified into C₁: SOZ node. Otherwise, it was classified into C₂: non-SOZ node. Finally, seizure onset likelihood, μ_i, was defined by $P_i^{C_1}$, the likelihood of x_i belonging to the SOZ class, C₁. A 5-fold cross-validation was employed to train and test all network layers in an end-to-end fashion.

Finally, computational experiments compared the msResNet with current state-of-the-art machine learning models. In the same training and test folds, we compared the classification performances of the following conventional data-driven models that are available in the MATLAB Machine Learning Toolbox (Mathworks, Natick, MA): k-nearest network (KNN) with equal Euclidean distance weight, random forest (RF) with ten bagged trees, and logistic regression (LR) classifier. To avoid an imbalance problem (i.e., n of SOZ instance: 726 nodes ≪ n of non-SOZ instance: 4,764 nodes) in the binary classification, these models were trained and tested using artificially augmented instances of x_i in the model cohort [44], where the instances of x_i in SOZ and non-SOZ nodes were interpolated from their 12 nearest neighbors and then added with the Gaussian random noises to evenly balance total numbers of instances x_i in C₁: SOZ (the augmented n = 10,000 at the augmentation factor of 13.8) and C₂: non-SOZ (the augmented n = 10,000 at the augmentation factor of 2.1). In addition, we also compared the proposed msResNet with three different types of advanced neural networks including multi-layer perceptron (MLP) with 6 hidden layers and cross-entropy loss function, sample-level deep convolutional neural network (CNN) [45] with 10 convolution layers and cross-entropy loss function, and 18-layer ResNet (ResNet18) [46] with cross-entropy loss function.

E. Statistical Analysis

Since our instance number is unbalanced in C₁: SOZ and C₂: non-SOZ, a balanced accuracy (arithmetic mean of sensitivity and specificity, [47]) of the trained msResNet was measured from all x_i of the validation cohort as an overall measure of the correct classification using the proposed msResNet. To determine an additive value of μ_i from the proposed msResNet, we evaluated the MRI marker (μ_i) and three different sets of iEEG markers from the identical patients in the model (18 patients whose MRI and iEEG were available for analysis) and validation cohorts (12 patients whose MRI and iEEG were available for analysis). The evaluated iEEG datasets were 1) low band threshold set including [MI_>80Hz, STE_>80Hz, SLL_>80Hz, HIL_>80Hz, MNI_>80Hz], 2) high band threshold set including [MI_>150Hz, STE_>150Hz, SLL_>150Hz, HIL_>150Hz, MNI_>150Hz], and all band threshold set including [MI_>80Hz, MI_>150Hz, STE_>80Hz, STE_>150Hz, SLL_>80Hz, SLL_>150Hz, HIL_>80Hz, HIL_>150Hz, MNI_>80Hz, MNI_>150Hz]. Each of the three iEEG marker sets was incorporated with the value of μ_i to perform the binary outcome prediction (0: non-epileptogenic sites defined as non-SOZ sites preserved during surgery and 1: epileptogenic sites defined as SOZ sites resected during surgery) using a multi-variate binomial logistic regression model. Briefly, the predictive model was trained in the model cohort with and without μ_i. The area under the curve (AUC) of the receiver-operating characteristics (ROC) analysis was then calculated from the validation cohort to assess the prediction performance of each trained model. Finally, to assess the separability of the msResNet-defined deep feature, k_i between two groups of interest, C₁: SOZ and C₂: non-SOZ, Z-score (i.e., (k_i − m)/σ where m and σ represent group mean and standard deviation of k_i) and p-value of two group t-test were separately evaluated per each element of k_i that were obtained from the input instance, x_i of C₁: SOZ and C₂: non-SOZ.

III. RESULT

The 5-fold cross-validation of the model cohort found that the msResNet successfully converged to global minima of cross-entropy loss in both training and test folds (Fig. 3). All instances of x_i in the training set (4 folds, n of Ω_i =3,271) successfully trained all network layers to minimize the loss between ground truth and predicted class labels, achieving a high accuracy of 0.99 in the training set. The trained layers provided a promising performance to classify SOZ and non-SOZ in the test set correctly (1 fold, n of Ω_i = 818), converging on a high accuracy of 0.96 after 100 epochs that yielded sensitivity/specificity/accuracy = 0.92/0.98/0.97 for lesional MRI patients and 0.91/0.96/0.94 for non-lesional MRI patients. It should be noted that both training and testing tasks were performed only using 4,089 instances of x_i (n of Ω_i = 4,089 in the model cohort), since the iEEG recordings of the model cohort completely confirmed the ground-truth locations of C₁: SOZ and C₂: non-SOZ at those instance nodes.

Fig. 3.

Convergence curve of the proposed msResNet classification for training (blue-colored curve) and test (red-colored curve) sets in the model cohort (n=24 patients).

The sensitivity, specificity, and balanced accuracy of the trained msResNet model to classify iEEG-defined SOZ and non-SOZ in the validation cohort (n of Ω_i = 2,752) are presented in Table III. Compared with other machine learning methods, the msResNet provided the highest balanced accuracy of 0.75/0.67 when classifying new instances of iEEG-defined lesional/non-lesional SOZ at sensitivity = 0.64/0.56 and specificity = 0.85/0.78, suggesting moderate reproducibility for the localization of iEEG-defined lesional/non-lesional SOZ in an independent cohort. However, the proposed msRes-Net outperformed the other three machine learning classifications to correctly classify SOZ in the validation cohort. More importantly, compared with other three neural networks including MLP, CNN, and ResNet18, the proposed multi-scale abstraction of raw imaging features using two different branches of CNN filters (1×3 and 1×7, Fig. 2) improved the balanced accuracy up to 47%/16% (MLP), 37%/8% (CNN), and 44%/12% (ResNet18) in the validation cohort of lesional/non-lesional MRI, respectively.

TABLE III.

CLASSIFICATION RESULTS IN THE VALIDATION COHORT. SEVEN DIFFERENT CLASSIFIERS (MSRESNET, RESNET, CNN, MLP, KNN, RF, LR) WERE TRAINED IN THE MODEL COHORT AND USED TO CLASSIFY THE MULTI-MODAL MRI INPUT INSTANCES OF THE VALIDATION COHORT.

Method	Measure	Lesional MRI	Non-lesional MRI
msResNet	Sensitivity	0.64	0.56
	Specificity	0.85	0.78
	Balanced accuracy	0.75	0.67
ResNet	Sensitivity	0.45	0.69
	Specificity	0.59	0.51
	Balanced accuracy	0.52	0.60
CNN	Sensitivity	0.42	0.30
	Specificity	0.68	0.94
	Balanced accuracy	0.55	0.62
MLP	Sensitivity	0.15	0.23
	Specificity	0.87	0.92
	Balanced accuracy	0.51	0.58
KNN	Sensitivity	0.23	0.15
	Specificity	0.68	0.70
	Balanced accuracy	0.46	0.43
RF	Sensitivity	0.13	0.17
	Specificity	0.81	0.82
	Balanced accuracy	0.47	0.50
LR	Sensitivity	0.51	0.51
	Specificity	0.66	0.41
	Balanced accuracy	0.59	0.46

Fig. 4A shows the probability histograms of the proposed MRI marker, seizure onset likelihood μ_i, measured in the iEEG-defined ground-truth nodes (SOZ and non-SOZ) of the model cohort and validation cohort. In both cohorts, the SOZ shows a peak probability at μ_i = 1 while the non-SOZ shows a peak probability at μ_i = 0, reflecting the very large effect size of Cohen’s d = 1.92 and 1.21 to differentiate SOZ and non-SOZ in both the model cohort and validation cohort, respectively. This effect size could provide the SOZ localization using the magnitude of seizure onset likelihood μ_i > 0.5, which is spatially well-matched with the surface map of the SOZ (with true positive rate: 0.92 and false negative rate: 0.08, Fig. 4B).

Fig. 4.

A. Two probability histograms of seizure onset likelihood μ_i in the model cohort (blue) and validation cohort (pink) were measured at the ictal iEEG-defined (or ground-truth) SOZ (left) and non-SOZ nodes (right). B. Representative case of the validation cohort (patient no.33, 8 year old boy). The ground-truth SOZ nodes Ω_i were colored white in the left 3D-brain visualization. The msResNet-defined SOZ localization in the right 3D-brain visualization could estimate high seizure onset likelihood μ_i at most ground-truth SOZ nodes Ω_i in the left 3D-brain visualization.

Fig. 5 compares performance of logistic regression models using three sets of interictal iEEG markers as individual predictors to classify the epileptogenic sites in the validation cohort (i.e., lesion/non-lesional MRI groups = 8/4 patients who underwent the acquisitions of both iEEG and multi-modal MRI). Multi-variate logistic regression analysis incorporating μ_i and interictal iEEG markers improved the correct prediction of epileptogenic sites (i.e., SOZ sites resected during surgery) in the non-lesional MRI group, AUC = 0.56–0.62 with iEEG alone and AUC = 0.64–0.65 with iEEG and μ_i, increasing the AUC by 5–15%. Meanwhile, the incorporation of μ_i and interictal iEEG markers did not improve the correct prediction of epileptogenic sites in the lesional MRI group, AUC = 0.70–0.77 with iEEG alone and AUC = 0.70–0.77 with iEEG and μ_i.

Fig. 5.

Accuracy of multi-variate logistic regression models incorporating interictal iEEG biomarkers and msResNet-based MRI biomarker (seizure onset likelihood, μ_i). Each model was trained on a subset of the model cohort (n = 18 patients who underwent the preoperative acquisitions of both iEEG and MRI, among 30 patients of the model cohort). The model accuracy was evaluated using the area under curve (AUC) from the validation cohort (n = 12 patients who underwent the same acquisitions of both iEEG and MRI, among 18 patients of the validation cohort).

IV. DISCUSSION

This study demonstrated the feasibility of integrating multi-modal MRI radiomics and deep features to develop a promising non-invasive epileptogenicity marker for children with drug-resistant focal epilepsy. Although advances in MRI technology have improved imaging of the epileptogenic brain, visual assessment of multi-modal MRI features (e.g., cortical thickening, atypical signal intensities, grey-white matter junction blurring, abnormal gyral patterns, etc.) remains the standard practice for both lesional and non-lesional MRI cases. Recent technical improvements in high-field MRI [48], [49] and simultaneous PET-MRI [50], [51] have made the lesions more detectable, but the level of improvement is incremental, an additional 5–8% [52], and these new technologies are expensive with limited availability. Thus, the number of detectable abnormalities appears to reach an asymptotic limit, even at a high cost. This fact motivates a need for a diagnostic paradigm shift, which can make the non-lesional SOZ more readily appreciable in clinical multi-modal MRI data via intelligently recognizing and artificially highlighting epileptogenic abnormalities.

To our best knowledge, the present study is the first attempt to incorporate state-of-the-art multi-scale deep learning technology with clinically acquired multi-modal MRI data in order to objectively mine deep imaging features of epileptogenic abnormalities directly measured at the iEEG-defined SOZ and non-SOZ sites. The findings of our study support the existence of subtle abnormalities in deep features of multi-modal MRI data, which can quantify the degree of regional epileptogenicity, μ_i, by systematically profiling complex deep features of multi-modal MRI data in the multi-scale residual neural network. Indeed, the present study successfully demonstrated that our non-invasive MRI marker μ_i is comparable with invasive interictal iEEG markers to correctly predict ictal iEEG-defined SOZ, and although preliminary, it may provide an additive value to improve overall predictability when it is combined with iEEG markers, especially in non-lesional MRI cases.

The other significant outcome of the present study is that the msResNet can identify the essential deep features of multi-modal MRI data that may outperform other state-of-the art machine learning methods to correctly classify SOZ and non-SOZ in the validation cohort that simulated actual practices in a cohort with postoperative seizure freedom. The saliency map [53] of the trained msResNet, which calculates the partial derivative of the output class probability with respect to the particular input feature segment, found 1) FA of gray matter middle surface, 2) DWIC of intra-hemispheric connectivity edge profile, 3) FA of gray matter outer surface, and 4) ADC of gray matter outer surface, as the most effective modality features to classify iEEG-defined SOZ and non-SOZ sites (Fig. 6A). FA and DWC of DWI showed greater contrast than other anatomical modalities and had consistent changes specific to the iEEG-defined SOZ, suggesting they can add practical value to current presurgical imaging protocols [54]. This finding is in accord with other studies. Our previous work [38], [39] reported that neocortical epilepsy had different patterns of diffusion abnormalities in both the seizure onset and electrographically normal cortical regions when compared to healthy controls. In the seizure onset regions, the cortical gray matter was showed decreased anisotropy in epileptic regions when compared to controls. Also, a marked increase in diffusivity was noted in the cortical gray matter, and this increase was most pronounced in the outer fraction of the gray matter. In addition, disrupted patterns of axonal connectivity such as increases in local connectivity [15] and decreases in long-range connectivity [16] have been consistently reported as DWIC makers for temporal lobe epilepsy. In contrast to the previous works, the current study found a striking difference in deep features (not raw features) between the ground-truth SOZ and non-SOZ, suggesting that there may be a fundamental difference in their hidden patterns of imaging abnormalities influencing DWIC connectivity edges (Fig. 6B). In other words, the abnormalities existing in SOZ sites may not be simply subtle in terms of raw MRI signals. Spatial convolution (or association) of local features may be more critical to discriminate the SOZ, as the last 256 elements of our deep features (from 1×7 convolution layer of Fig. 2) showed much higher Z-score and lower p-value than other elements of deep features. Indeed, the proposed msResNet could show notable changes in SOZ that are compatible with underlying abnormalities, and may assist in identifying the epileptogenicity likelihood from given multi-modal MRI data.

Fig. 6.

A. Predictive weight of individual input segments in x_i estimated from the trained layers of the proposed msResNet using saliency visualization, which computes the partial derivative of output: ${\mathbf{P}}_i^{{\mathbf{C}}_1}$ (i.e., probability of C₁: SOZ) with respect to the partial derivative of the input segment in x_i. B: Two-class separation (C₁: SOZ vs. C₂: non-SOZ) of the deep feature vector, k_i that was obtained from input instance x_i of C₁ and C₂. For each element of k_i, Z-score and p-value of a t-test were evaluated between the two classes, C₁ and C₂. Each red circle indicates an element providing statistically significant separation at a p-value < 0.05.

Recent deep learning network studies [55]–[57] have proposed CNN to detect the most common epileptogenic lesion: focal cortical dysplasia (FCD) using single modal MRI (i.e., T1-weighted MRI). High sensitivity was achieved to detect small patches of FCD (i.e., 74% in 35 healthy and 15 disease test group [56], 83% in 23 independent validation patients [57] and 90% in 10 FCD patients [55]). In contrast to these recent studies, the present study utilized multi-modal MRI features to localize heterogeneous SOZs with different types of epileptogenic lesions (FCD, tumor, tuber, gliosis, etc.) that are clinically prevalent and completely confirmed by iEEG recording, histopathological examination, and postoperative seizure control. Due to the different nature of study population, a direct comparison cannot be performed based on the reported values of sensitivity. However, taken together with the findings of the present work and other deep learning studies, it appears to be plausible that the laminar featuresof T1-weighted MRI, FA and ADC may be promising biomakers to non-invasively localize the epileptogenic foci associated with different types of morphological abnormalities.

Both histopathology data from resected specimens and neuroimaging studies in the past decades increasingly highlighted the importance of abnormal white matter in the epileptic brain, with specific changes in the vicinity of seizure foci (reviewed by [58]). White matter changes, including blurring of the gray-white matter interface often visualized on MRI and an increased number of heterotopic neurons in the subcortical white matter on histology have been common hallmarks both in temporal lobe epilepsy and neocortical epilepsies associated with developmental malformations [59]–[62]. Myelinated axons in the white matter create both short- and long-range fibers connecting various cortical regions, and these connections form a network whose connectivity undergoes distinct changes in the epileptic brain and participates in seizure generation and propagation [63]. Abnormal myelination may develop early as a result of repeated seizures or interictal spikes leading to increasing axonal electrical activity, which is one of the regulators of developmental myelination processes [64], [65]. Increased cell content in white matter and abnormal myelination can both alter tissue characteristics affecting diffusion properties, which show more severe abnormalities closer to the epileptic focus as detected by diffusion MRI [66]–[68]. Altogether these processes lead to microstructural white matter changes that are enhanced in and around epileptic foci, and their detection can contribute to a more accurate seizure focus localization.

Despite our effort to ensure methodological rigor with multiple cross-validations, the present study has unavoidable limitations. Our retrospective and observational study had a small sample size (n=41 patients who achieved postoperative seizure freedom to guarantee the practical reliability of our ground-truth SOZ and non-SOZ localization), complicating the ability of msResNet to learn the heterogeneous natures of multi-scale deep features in different lesional cases (e.g., cortical dysplasia, tumor, tuber etc.). Although artificial data augmentation such as the synthetic minority over-sampling technique [44] has been widely applied to alleviate this limitation, it is based on a randomly interpolated resampling procedure across the nearest neighbors, which may cause overfitting in the network output. Thus, the practical reproducibility of the msResNet to classify SOZ in lesional and non-lesional MRI should be re-evaluated at a large sample size, including multi-institutional data. The federated learning approach [69] was recently proposed to facilitate multi-institutional collaborations, where individual institutions do not share their data to train the network model but rather their trained network model to aggregate multi-institutional training in the shared model. Adopting this federated learning would allow the msResNet to train at a large sample size, making it possible for the msResNet to ultimately classify different subgroups of lesional and non-lesional SOZ sites controlled by heterogeneous epileptogenic mechanisms.

More importantly, the wide age range limits learning of deep features without considering age-related developmental trajectories of multi-modal MRI features, even though all imaging features were appropriately pre-scaled by their global maxima. Other clinical variables such as epilepsy duration, seizure severity, and the location of epileptogenic foci were not considered in our deep learning classification. Thus, further studies should consider the effect of these clinical variables on learning performance to better generalize and integrate our SOZ classification into clinical practice. It is anticipated that the addition of clinical variables to the input vector of the msResNet could further improve the classification accuracy in both lesional and non-lesional MRI cases. Finally, a future study should test the accuracy of the proposed msResNet approach in an unselected, more typical, epilepsy surgery population including patients with postoperative seizure recurrence and having multi-focal epilepsy.

V. CONCLUSION

The present study has demonstrated that imaging abnormalities exist in the SOZ on clinically acquired multi-modal MRI data, and that deep learning of these abnormalities via the multi-scale neural network can be effective to localize SOZ even if the MRI is negative (not localizing) on visual assessment. Our results could benefit such evaluations by revealing potential abnormalities that need more careful inspection. They also may help presurgical planning by providing secondary options to guide the placement of iEEG electrodes.