Skip to main content
NCBI home page
NeuroImage : Clinical logoLink to NeuroImage : Clinical
. 2014 Nov 27;7:122–131. doi: 10.1016/j.nicl.2014.11.013

Advanced [18F]FDG and [11C]flumazenil PET analysis for individual outcome prediction after temporal lobe epilepsy surgery for hippocampal sclerosis

J Yankam Njiwa a,⁎, KR Gray b, N Costes c, F Mauguiere d,e,f, P Ryvlin d,e, A Hammers a
PMCID: PMC4299974  PMID: 25610774

Abstract

Purpose

We have previously shown that an imaging marker, increased periventricular [11C]flumazenil ([11C]FMZ) binding, is associated with failure to become seizure free (SF) after surgery for temporal lobe epilepsy (TLE) with hippocampal sclerosis (HS). Here, we investigated whether increased preoperative periventricular white matter (WM) signal can be detected on clinical [18F]FDG-PET images. We then explored the potential of periventricular FDG WM increases, as well as whole-brain [11C]FMZ and [18F]FDG images analysed with random forest classifiers, for predicting surgery outcome.

Methods

Sixteen patients with MRI-defined HS had preoperative [18F]FDG and [11C]FMZ-PET. Fifty controls had [18F]FDG-PET (30), [11C]FMZ-PET (41), or both (21). Periventricular WM signal was analysed using Statistical Parametric Mapping (SPM8), and whole-brain image classification was performed using random forests implemented in R (http://www.r-project.org). Surgery outcome was predicted at the group and individual levels.

Results

At the group level, non-seizure free (NSF) versus SF patients had periventricular increases with both tracers. Against controls, NSF patients showed more prominent periventricular [11C]FMZ and [18F]FDG signal increases than SF patients. All differences were more marked for [11C]FMZ. For individuals, periventricular WM signal increases were seen at optimized thresholds in 5/8 NSF patients for both tracers. For SF patients, 1/8 showed periventricular signal increases for [11C]FMZ, and 4/8 for [18F]FDG. Hence, [18F]FDG had relatively poor sensitivity and specificity. Random forest classification accurately identified 7/8 SF and 7/8 NSF patients using [11C]FMZ images, but only 4/8 SF and 6/8 NSF patients with [18F]FDG.

Conclusion

This study extends the association between periventricular WM increases and NSF outcome to clinical [18F]FDG-PET, but only at the group level. Whole-brain random forest classification increases [11C]FMZ-PET's performance for predicting surgery outcome.

Keywords: Hippocampal sclerosis, Surgery outcome, FDG-PET, FMZ-PET, Periventricular white matter signal increases, Random forests

Graphical abstract

graphic file with name fx1.jpg

Open in a new tab

1. Introduction

Around one-third of patients with epilepsy are refractory to medical treatment, and epilepsy surgery is an efficient therapeutic alternative for some of these patients. Resection surgery is indicated in some focal epileptic syndromes, including mesial temporal lobe epilepsy (mTLE), a common form of adult epilepsy that is often resistant to anti-epileptic drugs (AEDs) and frequently associated with hippocampal sclerosis (HS) (Engel et al., 2003; Spencer and Huh, 2008). Nevertheless, about one-third of patients with HS are not seizure free (NSF) after epilepsy surgery. There is therefore a need of finding relevant biomarkers to anticipate the risk of suboptimal surgery outcome.

The presence of unilateral HS on presurgical magnetic resonance imaging (MRI) ipsilateral to surgery predicts a favourable, i.e. seizure free, outcome (Jeong et al., 2005), as does focal hypometabolism on [18F]fluorodeoxyglucose positron emission tomography (FDG-PET) ipsilateral to affected temporal lobe (Malmgren and Thom, 2012).

Other potential biomarkers have been investigated for the prediction of TLE-HS surgery outcome and findings are not always consistent (Aull-Watschinger et al., 2008; Janszky et al., 2005). For example, analysis of the ictal semiology as recorded by scalp video-EEG has shown that extra temporal features do not correlate with surgical outcome in patients with temporal lobe epilepsy and HS (Borelli et al., 2008). However, other studies have found extratemporal features as relevant prognostic factors for predicting outcome after TLE-HS surgery (Ferrari-Marinho et al., 2012; Najm et al., 2013; Ryvlin and Kahane, 2005). Hence the search of prognostic features is important for identifying reliable predictors of suboptimal outcome as a tool for patient management and counselling, including for avoiding invasive EEG prior to surgery when possible.

PET has long been used for helping to localize the seizure-onset area, especially in TLE (Engel et al., 1990). The most common PET tracer used is [18F]FDG, which measures regional glucose metabolism (Henry et al., 1993). [11C]flumazenil (FMZ) which binds to the benzodiazepine sites of GABAA receptors has been used for predicting surgery outcome in mTLE-HS. Previous studies have shown that increased preoperative periventricular white matter (WM) FMZ binding was associated with NSF outcome after temporal lobe resection for HS. This was shown at the group level using volume-of-distribution images (Hammers et al., 2005) and, with a larger control group, at the individual level using summed images (Yankam Njiwa et al., 2013). The tentative explanation for this association is that increased periventricular binding relates to heterotopic neurons, i.e. a mild form of migration disorder not visible on MRI (Hammers et al., 2003).

Furthermore, in a number of papers, investigation of WM [11C]FMZ volume-of-distribution (VT) by voxel-based analysis has shown a strong correlation between WM [11C]FMZ VT and the number of heterotopic neurons histologically determined in the WM (Hammers et al., 2001; Hammers et al., 2002). PET analysed with SPM detected periventricular [11C]FMZ VT increases in MRI-negative patients with extratemporal epilepsy which were analogous to such increases when analysing MRI visible periventricular heterotopia (Hammers et al., 2003).

In the present study, we first investigated whether increased preoperative WM FDG uptake could also be used for predicting epilepsy surgery outcome. We then assessed if an advanced machine learning technique (random forests (RF); Breiman, 2001) could predict seizure outcome based on voxel-wise signal intensities from preoperative FDG and FMZ whole brain images. Finally, both methods were compared.

2. Material and methods

2.1. Participants

Patients were retrospectively identified for the study and fulfilled the following criteria: 1) suffering from mTLE, 2) HS diagnosed on MRI (confirmed by histopathology for thirteen of the patients), 3) temporal lobe surgery with a postsurgical follow up for at least 12 months, 4) availability of preoperative [18F]FDG PET and [11C]FMZ PET on the Siemens/CT HR+ PET camera installed at the CERMEP imaging centre.

Patient details have been described in a previous study on [11C]FMZ PET alone (Yankam Njiwa et al., 2013). Briefly, sixteen patients (twelve women) fulfilled the criteria. Nine had left-sided HS and six right-sided HS, diagnosed on MRI by an experienced radiologist and confirmed by an experienced neurologist. Outcome was assessed according to Engel's classification (Engel, 1987) by an epileptologist blinded to the imaging results (PR), for the time point of assessment. Eight patients (50%) were entirely seizure free (SF; Engel class IA) and eight were not (NSF; not Engel class IA either continuously or at some stage during follow-up, at least 4 months after surgery) (Takahashi et al., 2012). Table 1 shows the clinical characteristics of the patients.

Table 1.

Clinical data for 16 patients with refractory mTLE and HS. Upper half: seizure free patients; lower half: non-seizure free patients. Imaging based predictors for seizure status are colour-coded, with green indicating that the particular marker correctly predicted the individual patient's outcome class according to the method used in that column (see text for details).

Patient Age/sex Onset/duration of epilepsy (years) HS Follow-up (months) Medication at operation. *: reduced. **: stopped (medication restarted) Outcome (Engel class) FDG hypometabolism on SPM [11C]FMZ-PET increases in periventricular areas (threshold = 0.01, z > 3.2, p < 0.46) [18F]FDG-PET increases in periventricular areas (threshold = 0.01, z > 3.2, p < 0.5) RF prediction (FMZ) RF prediction (FDG) Additional information
#1 24/f 6/18 L 31 LEV, LTG* IA Yes None L ant front lobe z = 3.89 (−14 28 2), 3.2 cm2 True False L temporal–occipital dysplasia
#2 22/f 2/20 L 58 CBZ, TPM, CLB** IA Yes None None False False
#3 27/m 4/23 L 35 CBZ, VPA, CLB** IA Yes None None True True
#4 36/f 3/33 L 98 CBZ, LTG, TPM** IA Yes None L ant front lobe z = 4.24 (−24 32 −2), 2.2 cm3
R ant front lobe z = 3.81 (16 32 −6), 1.7 cm3 		True True L temporal pole signal abnormalities + discrete dysgenesis of left amygdala
#5 46/f 12/34 L 52 PB, PRG, LEV* IA No None R ant front lobe z = 3.24 (4 22 −4), 2.4 cm3 True True
#6 26/f 5/21 R 112 CBZ, CLB, PB* IA No R lat of ventricle z = 4.18 (42 −36 8), 2.2 cm3 L lateral of ventricle z = 6.02 (−34−42 16), 5.3 cm3. R ant front lobe z = 4.66 (18 28 4), 10.4 cm3. R lateral of ventricle z = 3.7 (18–22 32), 2.2 cm3 True False R temporal pole signal abnormalities
#7 48/f 6/42 R 81 CBZ, CLB** IA Yes None None True True
#8 30/f 5/25 R 24 CBZ, TPM, CLB, LEV** IA Yes None None True False
#9 46/f 33/13 R 23 PHT, LEV, PB** IB Yes None Lateral of posterocentral portion of R lateral ventricle z = 3.76 (30–34 22), 1.6 cm2 True False
#10 36/f 3/33 R 116 CBZ, PB, CLB IVB No None None True True Seizure onset right temporal and right insular on SEEG
#11 29/f 4/25 R 44 CBZ, CLB** IIA Yes None None True True Small symmetric hippocampi and temporal pole signal abnormalities on MRI (FDG, SEEG unilateral R)
#12 32/m 5/27 L 119 PB, CLB** (LEV, CLB) IIA No Widespread WM increases with the maximum L ant of frontal horn z = 6.02 (−20 34 −4), 15.3 cm3 but extending bilaterally with other maxima anterior to both frontal horns and postero-lateral of right lateral ventricle. Lateral of posterocentral portion of L lateral ventricle z = 4.62 (−34 −44 14), 3.7 cm3. Lateral of posterior portion of R lateral ventricle z = 4.12 (30 –60 12), 6.5 cm3 Widespread WM increases with the maximum R lat ventricle z = 5.62 (30–34 20), 5.02 cm3 but extending bilaterally with other maxima to the lateral ventricle z = 4.92 (−36 −42 16), 7.0 cm3. R ant front lobe z = 3.73 (26 32 10), 2.2 cm3 True True
#13 32/f 11/21 L 89 CLB, CBZ** (CBZ, PB) IID No Lateral of anterocentral portion of R frontal horn z = 3.27 (24 12 16), 4.0 cm3. L anterior of frontal horn z = 3.82 (−18 36 −4), 5.0 cm3 R anterior of frontal horn z = 3.24 (24 −6 28), 1.9 cm3 False True
#14 22/m 9/13 L 72 CBZ, TPM, PB, CLB** ID Yes Lateral of posterocentral portion of L lateral ventricle z = 4.51 (−34 −48 16), 1.7 cm3. L anterior of frontal horn z = 3.96 (−14 38 −2), 1.7 cm3 L ant front lobe z = 5.79 (−12 36 −2), 5.5 cm3. L sup cella media post horn z = 4.7 (−32 −50 18), 3.6 cm3 True True
#15 39/m 1/38 L 67 TPM, PB* IIA No Lateral and inferior of L ventricle where temporal and occipital horns originate z = 4.01 (−30 −52 4), 3.0 cm3. Lateral and inferior of R ventricle z = 3.29 (10 6 4), 2.4 cm3 Lateral postorocentral portion of R lateral z = 5.11 (24 −38 4), 7.3 cm3. L occipital horn z = 4.648 (−28 −50 6), 4.4 cm3 True False
#16 41/f 3/38 R 78 LTG, CBZ, PB* IC No Lateral and inferior of R ventricle where temporal and occipital horns originate z = 3.22 (32 −48 4), 1.8 cm3 None True True Moderated gliosis R temporal pole, L ventricular enlargement
Open in a new tab

AP = amisulpride, AZL = alprazolam, PB = phenobarbital, CBZ = carbamazepine, CLB = clobazam, CP = codeine paracetamol, DHGT = dihidroergotamine, EXD = enrofloxacine+diclofenac Enromax, FA = folic acid, GBP = gabapentin, GVG = vigabatrin, HDZ = hydroxyzine, LEV = levetiracetam, LTG = lamotrigine, OXC = oxcarbazepine, PB = phenobarbital, PBA = phenobarbital with amphetamine, PHT = phenytoin, PPN = pipampérone, PVT = pravastatin, PX = paroxétine, PZ = Prozac, TGB = tiagabine, TPM = topiramate, VPA = valproic acid.

Postoperative outcomes for non-seizure free patients are given in the footnotes, including times when the outcome class changed. [11C]FMZ-PET results were previously presented (Yankam Njiwa, 2013 #6038) and are repeated here for comparison.Clusters are characterized by peak voxel z score (MNI coordinates), and cluster extent.

M = male, f = female, HS = hippocampal sclerosis.1IB at 5 months and IA at 17 months. 2Never seizure free. 3IB at 5, 10, 23 months, ID at 16 months and IA at 29 and 35 months. 4IA until 110 months. 5IA at 19 months and IID at 31 months. 6ID at 60 months. 7IIA at 67 months. 8IC at 12 and 21 months.

Data were available for the following non-imaging potential outcome predictors: age at onset, duration of epilepsy, age at operation.

Fifty control subjects with a mean age of 35 ± 10 years were available for this study. Forty-one had [11C]FMZ PET scans (20 men and 21 women), and 30 had [18F]FDG scans (21 men, 9 women). Twenty patients had both FMZ and FDG-PET scans. All images were acquired on the same camera. All controls had normal MRI and were not treated with any regular medication.

All subjects had signed written informed consent at the time of PET scanning in agreement with the French legislation and after approval by the local Ethics committee.

2.2. PET data acquisition

Three-dimensional PET scans were acquired on an Exact ECAT HR+ scanner (Siemens, Erlangen, Germany) with a full width half maximum (FWHM) spatial resolution of about 5 mm3 (Brix et al., 1997). A 68Ge transmission scan was acquired for attenuation correction.

FMZ (RO 15-1788) was radiolabelled with 11C via methylation (Maziere et al., 1984). Unlabelled FMZ (0.01 mg/kg) and [11C]FMZ (2.775 MBq/kg) were injected simultaneously (Delforge et al., 1995), and twelve frames of increasing length were acquired over 55 min (for details, see (Yankam Njiwa et al., 2013)). Summed [11C]FMZ radioactivity images were obtained from the attenuation and scatter corrected dynamic 3D series over the last five frames (20–55 min post injection), as these have been shown to yield particularly reliable measures (Bouvard et al., 2012).

The protocol for FDG-PET imaging consisted of a simultaneous injection of an 18.5 MBq load dose in addition to 1.5 MBq/kg dose of [18F]FDG followed by the acquisition of static images 40–50 min post injection.

2.3. PET data analysis

[11C]FMZ summed radioactivity images and static [18F]FDG images were first analysed using statistical parametric mapping (SPM8; Wellcome Department of Imaging Neuroscience, UCL, London, UK; http://www.fil.ion.ucl.ac.uk/spm) and specific periventricular masks (Hammers et al., 2005; Yankam Njiwa et al., 2013). Voxel-wise PET signal intensities across the whole brain were then used for RF classification.

For group analyses, the images of the seven right HS patients (3/8 SF and 4/8 NSF) were right–left reversed prior to spatial normalization to lateralize the epileptogenic side to the left for all patients.

In-house [11C]FMZ and [18F]FDG templates in the standard Montreal Neurological Institute (MNI) space based both on over thirty five healthy subjects scanned on the same scanner were available. Because a proportion of the raw images had been left–right reversed, these templates were symmetrized for group analyses and RF classification as described earlier (Didelot et al., 2010). All images of the study were spatially normalized (Ashburner and Friston, 1999) to the corresponding symmetric template for the appropriate tracer (either [11C]FMZ or [18F]FDG). The resulting spatially normalized images were smoothed using an isotropic Gaussian kernel of 10 mm FWHM.

For individual analyses with SPM8, the original, asymmetric [11C]FMZ and [18F]FDG templates were used, i.e. individual images were not flipped. The preprocessing was otherwise identical.

Global means, i.e. the average radioactivity across the brain determined within a mask consisting of the voxels with intensity above 1/8 of the mean of all voxels in the matrix, were compared between SF patients, NSF patients and controls using two-tailed student t-tests.

2.3.1. SPM analyses of periventricular WM increases

For SPM8 analysis, entire WM was extracted through combination of two masks and the assessment of periventricular regions was performed using a mask constructed with basic SPM8 functions, for more details see (Hammers et al., 2005; Yankam Njiwa et al., 2013). A covariance group specific analysis (ANCOVA) model was utilized for global radioactivity correction.

Periventricular increases were assessed within the mask introduced above in the group and individual analyses. We aimed to replicate the earlier finding of periventricular WM increases of [11C]FMZ binding in patients with suboptimal outcome relative to SF patients and controls, using the much more widely used PET tracer [18F]FDG.

At the individual level, each patient was compared against all 30 [18F]FDG controls in order to investigate whether preoperative periventricular increases can be used to predict postoperative outcome for a given patient. Several combinations of the characteristics of clusters with increased periventricular WM signal were evaluated for the best separation of SF and NSF patients, namely “uncorrected cluster probability” and “z value of peak voxel” (Yankam Njiwa et al., 2013).

A given search space between 0.00 and 0.99, with moving steps of 0.01, for cluster probability and a level of significance of the peak voxel z score set at any value above 3.2, step of 0.02 was set. The ability to optimally discriminate SF and NSF was tested for all combinations and after screening the cluster p values.

2.3.2. SPM analyses of mesial temporal [18F]FDG hypometabolism

Presence of focal hypometabolism on presurgical FDG-PET ipsilateral to affected temporal lobe has been shown to predict favourable surgical outcome (Malmgren and Thom, 2012). We have therefore investigated the significance of mesial temporal lobe [18F]FDG hypometabolism as a predictor of postsurgical outcome in this study. Each patient was compared against all 30 [18F]FDG controls and clusters with a peak voxel z score ≥3.2 and a p value ≤0.5 were considered significant.

2.3.3. Random forest classification based on whole-brain PET signal intensities

We investigated the capabilities of random forest (RF) classifiers to distinguish patients who became SF and those who remained NSF from each other and from controls based on voxel-wise signal intensities across the full brain volumes for both [11C]FMZ and [18F]FDG PET images.

Each subject served as a test dataset for the training set composed of the remaining subjects acquired with each tracer. Feature data from images acquired with each tracer were independently used as inputs to an RF classifier.

RF is an ensemble classifier, consisting of many decision trees, which outputs the class that is the result of combining the prediction by individual trees (Breiman, 2001). Random forests combine bootstrap aggregation (bagging) (Breiman, 1996) and random feature selection (Amit and Geman, 1997; Ho, 1998) to construct a collection of decision trees exhibiting controlled variations. The training set for each individual tree in a random forest is constructed by sampling N examples (subjects) at random with replacement from the N available examples in the dataset. After drawing N bootstrap samples from the original data, an ‘unpruned’ decision tree is grown as follows: at each node, rather than choosing the best split among all predictors, randomly sample mtry of the predictors and choose the best split from among those variables. New data are classified by aggregating the predictions of the Ntree trees, in the case of classification considering the majority of votes.

Tree nodes are split based on the Gini index (Breiman et al., 1984), which measures the likelihood of an example to be incorrectly labelled if it was randomly classified according to the distribution of labels within the node. A measure of the importance of an individual feature for classification can be computed by summing the decreases in the Gini index occurring at all nodes in the forest, which are partitioned based on that feature.

For the classification results presented in this study, the implementation of random forests in the statistical software package R, a port of Leo Breiman and Adele Cutler's original Fortran code by Andy Liaw and Matthew Wiener, version 4.6–6 (http://cran.r-project.org/web/packages/randomForest), was used.

At each forest were grown [1000:step:10,000] trees in steps of 1000. The datasets were first tuned, with the specific class implemented in R (rftune), to seek for the optimal number of features (computing the smallest error value) to be selected at each tree node according to the number of available features. The number of features thus computed at each node was 714 for FMZ and 15 for FDG.

3. Results

3.1. Global values

The global values of the summed radioactivity images differed significantly between the NSF and controls groups before spatial normalization for FDG. After spatial normalization, i.e. on the images used for the present analyses, these values did not differ significantly (pmin = 0.11). Table 2 illustrates the results.

Table 2.

Global values (mean ± SD), in Bq/ml, computed for summed radioactivity images (FMZ) and FDG static images, before and after normalization. In brackets are the student test values of comparison of different patient groups and control group. The absolute difference in values before and after normalization is due to matrix size differences.

FDG
 FMZ
NSF SF Controls NSF SF Controls
Before normalization 2064 ± 946 (p = 0.03) 2290 ± 986 (p = 0.09) 2998 ± 652 1492 ± 300 (p = 0.03) 1562 ± 207 (p = 0.02) 1778 ± 256
After normalization 3268 ± 1807 (p = 0.22) 3233 ± 1360 (p = 0.11) 4154 ± 932 2183 ± 395 (p = 0.17) 2202 ± 279 (p = 0.09) 2409 ± 335
Open in a new tab

3.2. Group analyses

3.2.1. NSF versus SF patients restricted to WM periventricular signal increases

Higher periventricular ligand uptake was visualized in NSF patients in comparison to SF patients for both tracers. Areas of high [18F]FDG binding were observed around the posterior horn of the ventricles contralateral to the seizure focus (zmax = 3.37, 0.4 cm3) and ipsilaterally (zmax = 3.66, 1.1 cm3). [18F]FDG uptake was increased at very similar locations as [11C]FMZ uptake (Yankam Njiwa et al., 2013); see Fig. 1a.

Fig. 1.

Fig. 1

Open in a new tab

Periventricular increases and unmasked statistical results overlaid onto the SPM8 T1-weighted template. FDG results obtained for this study; FMZ results from Yankam Njiwa et al. (2013) and shown for direct comparison. (a) Increases in non-seizure free patients (NSF; not Engel IA) compared to seizure-free patients; ipsilateral, left side, (zmax = 2.69, CE = 0.56 cm3) and contralateral (zmax = 4.2, CE = 0.9 cm3) for FMZ and ipsilateral (zmax = 3.66, CE = 1.1 cm3), contralateral (zmax = 3.37, CE = 0.4 cm3) for FDG. (b) Seizure free (SF; Engel class IA) patients, compared to controls (zmax = 3.31, CE = 0.25 cm3, ipsilateral) for FMZ and (zmax = 4.16, CE = 2.12 cm3, ipsilateral) for FDG. The apparent increase near the superior sagittal sinus is a normalization artefact appearing in both modalities. (c) NSF patients compared to controls (zmax = 5.25, CE = 5.25 cm3, ipsilateral) for FMZ and (zmax = 4.68, CE = 5.2 cm3, contralateral) for FDG. (d) Periventricular signal increases in an individual patient (#15), with a suboptimal outcome, for FMZ (zmax = 4.01, CE = 3 cm3, ipsilateral), and for FDG (zmax = 5.11, CE = 7.3 cm3, contralateral). Significant clusters are pointed with green arrows.

There were no significant differences between SF and NSF patients for other potential predictors of postoperative outcome. Results are illustrated in Table 3.

Table 3.

Evaluation of the age at seizure onset, the duration of epilepsy and the age at operation as potential non-imaging predictors for epilepsy surgery outcome, in terms of median, interquartile distance, and Mann–Whitney U test (MWUT) values.

Mean[interquartile distance]
Predictor Age at onset Duration of epilepsy Age at operation
NSF 4.5 [6.5] 26 [15.3] 34 [8.3]
SF 5.0 [2.3] 24 [12.5] 28.5 [13]
MWUT (two tailed) 32.5 (p = 0.96) 32.5 (p = 0.96) 38.5 (p = 0.5)
Open in a new tab

3.2.2. Patient groups against controls restricted to WM periventricular signal increases

As expected, SF patients compared with controls showed a smaller number of periventricular clusters, which were also less significant, than NSF patients for [18F]FDG. [18F]FDG showed a bigger cluster with a higher z score (z = 4.16, 2.1 cm3) than [11C]FMZ had in our previous study (Fig. 1b).

The group of NSF patients compared with controls showed increased FDG uptake in four clusters with higher z scores than SF patients: two ipsilaterally (8.6 and 1.3 cm3, zmax = 4.19) and two contralaterally (5.2 and 1.2 cm3, zmax = 4.68). Similarly, using [11C]FMZ, more clusters of periventricular signal increases with higher z scores had been found in NSF patients versus controls, compared with SF patients against controls (Yankam Njiwa et al., 2013, Fig. 1c).

3.2.3. Random forest classifier analyses of the entire brain volume

Random forest classifiers were independently applied to the available feature data corresponding to all available voxels (513,340/vol) for both radiotracers.

3.2.4. NSF versus SF patient groups

For both FDG-PET and FMZ-PET, RF classification was not able to distinguish between NSF and SF patients. Performance results are illustrated in Table 4. FMZ better performed than FDG for the classification of subjects, except for the NSF/SF comparison in terms of sensitivity.

Table 4.

Test performance of the SPM-based assessment of periventricular increases. SF, seizure free (n = 8); NSF, not seizure free (n = 8); FMZ [11C]flumazenil PET; FDG [18F]FDG PET. The cells with a grey background indicate patients correctly classified.

3.2.4.

Open in a new tab

3.2.5. Patient groups against controls

For the comparison between the group of patients and controls, RF classifiers well correctly predicted 7/8 NSF using FMZ images and 6/8 with FDG. FMZ binding was more sensitive in differentiating SF to from controls (7/8 well correctly predicted) than FDG (6/8 well correctly predicted).

With FMZ data, important features for distinguishing NSF patients from controls consisted of more widespread clusters detected principally in the hippocampus, whereas for SF patients those were somewhat more concentrated in the amygdala and in the temporal lobe. Results are shown in Fig. 2.

Fig. 2.

Fig. 2

Open in a new tab

Feature importances for differentiating between three clinical groups using voxel based FDG-PET (bottom) and FMZ-PET (top). Voxels important for distinguishing patients from controls are overlaid onto an MNI-space FMZ-PET template (top) and FDG-PET template (bottom). Left column, non-seizure free patients; right column, seizure free patients.

3.3. Individual analyses

3.3.1. NSF versus SF patients: WM periventricular signal increases

After screening the cluster p values as mentioned in the Materials and methods section, it resulted that considering clusters in the periventricular mask fulfilling the condition of zmin = 3.2/pmax = 0.46 for [11C]FMZ, and zmin = 3.2/pmax = 0.5 for [18F]FDG best differentiate SF and NSF patients, see Table 1.

Periventricular increases in individual patients occurred in similar regions as in the group analyses. Under the optimal threshold conditions described in the Materials and methods section and for the same patients, five of eight NSF patients (63%) showed periventricular WM signal increases with [18F]FDG, the same number as previously found for [11C] FMZ (Yankam Njiwa et al., 2013, Table 5).

Table 5.

Performance results of the RF classifier for binary classifications. Mean ± SD accuracy, sensitivity and specificity for summed radioactivity images (FMZ) and FDG static images. Performances are averaged over the number of trees for each RF (1000:1000:10,000). NSF = not seizure free, SF = seizure free, C = controls. Performance values of the RF classifier in classifying NSF/SF patients versus controls and in identifying NSF patients versus SF patients.

FDG
 FMZ
NSF/SF NSF/C SF/C NSF/SF NSF/C SF/C
Accuracy 0.36 ± 0.03 0.66 ± 0.001 0.53 ± 0.02 0.40 ± 0.06 0.83 ± 0.01 0.82 ± 0.01
Sensitivity 0.33 ± 0.06 0.74 ± 0.04 0.5 ± 0.00 0.20 ± 0.06 0.76 ± 0.04 0.76 ± 0.04
Specificity 0.40 ± 0.05 0.63 ± 0.00 0.53 ± 0.03 0.54 ± 0.08 0.85 ± 0.01 0.83 ± 0.01
Open in a new tab

Four of eight SF patients showed periventricular signal increases with [18F]FDG, whereas only one of eight SF patients had shown periventricular increases with [11C]FMZ (Yankam Njiwa et al., 2013, Table 5). In the context of this study, sensitivity corresponds to the fraction of NSF patients correctly predicted to remain NSF, i.e. having periventricular increases, and specificity corresponds to the fraction of SF correctly predicted to become SF after the operation, i.e. patients without periventricular increases. Hence, the test performed had 63% sensitivity / 50% specificity for [18F]FDG, compared with 63% sensitivity / 88% specificity for [11C]FMZ. Table 1 and Table 5 show the corresponding results and Fig. 1d illustrates areas of increased periventricular [11C]FMZ binding and [18F]FDG uptake in an individual NSF patient (Patient # 15).

3.3.2. NSF versus SF patients: mesial temporal [18F]FDG hypometabolism

Six out of eight SF patients did have mesial temporal lobe hypometabolism, and five out of eight NSF patients did not (Table 1). The presence of ipsilateral mesial temporal lobe hypometabolism did hence to a certain degree predict SF outcome, and its absence to a degree predicted NSF outcome.

3.3.3. Random forest classifier analyses of the entire brain volume

For FMZ, the leave one out test performed with 1000 trees led to five out of eight true positive NSF predictions in NSF patients against 35 out of 41 controls correctly predicted. The test also well predicted five SF patients as SF and failed to predict six out of 30 controls.

In contrast, with FDG, the test predicted five out of eight true positive NSF and failed to well predict eleven controls. Fourteen controls were not correctly predicted against four true positive SF patients.

4. Discussion

This study shows an association between periventricular WM increases of [18F]FDG binding with NSF status after temporal resection for HS. These results corroborate the observations previously made for [11C]FMZ (Yankam Njiwa et al., 2013). For large numbers of control datasets available, periventricular [11C]FMZ has previously been shown to usefully predict individual postoperative outcome; here we show that this is less the case for [18F]FDG uptake regarding periventricular white matter increases.

Accuracies for distinguishing patients from controls were larger than 0.8 for [11C]FMZ using RF classification. However, this distinction is not clinically meaningful. For the clinically important SF/NSF distinction, [18F]FDG as well as [11C]FMZ performed roughly at chance level.

The previous study (Yankam Njiwa et al., 2013) showed that summed [11C]FMZ radioactivity images, not requiring metabolite corrected arterial plasma input functions, were sufficient for predicting surgery outcome in candidates with TLE based on periventricular WM changes. An important finding of the current study is that the method could in principle, with caution, also be applied to the more clinically widely used FDG.

Increased [11C]FMZ/[18F]FDG signal likely indicates an increased concentration of heterotopic neurons in the sense of a developmental abnormality, i.e. MRI-invisible subependymal heterotopia (Hammers et al., 2003) or a consequence of recurrent seizures. This may imply the existence of a more widespread epileptogenic network beyond the hippocampus and boundaries of the temporal lobes.

As illustrated in the Results section, the presence of ipsilateral mesial temporal lobe hypometabolism to a certain degree predicted SF outcome status, and its absence to a degree predicted NSF outcome. However, the FMZ-based methods performed somewhat better. Interestingly, the different methods failed to predict outcome in different patients, suggesting that the different tracers assess different aspects of these patients' epilepsies.

According to the classification results computed with random forests classifiers, [11C]FMZ binding may refine the prediction of surgery outcome for temporal lobe epilepsy with HS, whilst this was not the case with [18F]FDG uptake. The feature importance during RF analysis allows the determination of voxels that have been useful for classification. For differentiating between the three clinical groups using voxel based [18F]FDG-PET and [11C]FMZ-PET, hippocampal voxels were more important for the classification of NSF patients against controls using FMZ images. In contrast, voxels located in the amygdala were more important for the classification between SF patients and controls compared to NSF patients. These findings are in line with the regions expected to be affected in temporal lobe epilepsy (Reid and Staba, 2014). Nevertheless the lesser sensitivity of [18F]FDG may reflect the failure to detect signal differences in brain structures and could also explained by the absence of local cluster representing important voxels.

These feature importances were highly interesting because voxels were detected outside the periventricular areas, thus opening the door to merge the results of the two methods. Combining RF classification with periventricular masking improved the prediction accuracy for FMZ: If both were used for predicting NSF status and if either test predicted NSF status (data in Table 1), FMZ would correctly predict NSF status in all NSF patients, but assign NSF prognosis incorrectly to 2/8 SF patients (sensitivity 100%, specificity 75%). However, combining both predictors for FDG would not be useful, as all patients except two SF patients would be given the NSF prognosis.

Several studies have shown that temporal lobe hypometabolism has postsurgical prognostic significance; patients with at least 15% hypometabolism were likely to become SF (Theodore et al., 1992; Theodore et al., 1994). It has also been illustrated that mean lateral but not mesial temporal asymmetry was significantly higher in patients who did become seizure free than those who did not (Theodore et al., 1992). Here, we investigated relative hypermetabolism in the periventricular white matter. For this feature, NSF patients had more clusters than SF patients (Fig. 2). However, they were seen bilaterally without marked asymmetry. This may reflect a mechanism acting on both sides, either a developmental abnormality of migration (Gleeson and Walsh, 2000) or a response to frequent seizures (Parent et al., 2002). Other attempts to predict postsurgical outcome from pre-surgical imaging data have also been recently discussed (Chassoux et al., 2010; Feis et al., 2013; Yankam Njiwa et al., 2013). In addition, atypical histological HS patterns, as for example damage restricted to the hilar region, may predict poorer outcome (Blumcke et al., 2007; de Lanerolle et al., 2003; Sagar and Oxbury, 1987). The same held true when HS patterns were analysed quantitatively (Thom et al., 2010). The usefulness of the potential predictors is dependent on the assessment time according to the operation and prognostic factors are also interdependent (Janszky et al., 2005; Najm et al., 2013). Seizure characteristics like high seizure frequency and a history of generalized seizures have also been described as predictors of non-seizure free outcome (Asztely et al., 2007; Janszky et al., 2005). In our study, seizure frequencies had not been prospectively noted at the time of the scan in all patients included, and given the long follow-up, it was not possible to meaningfully reconstruct this information.

Previous studies have sometimes found that “mild” malformations of cortical development, e.g. cortical lamination abnormalities in the temporal lobe associated with HS (focal cortical dysplasia type IIIa according to the 2011 ILAE classification (Blumcke et al., 2011)) may not influence postsurgical outcome in mTLE (Kuba et al., 2012). It is important to note that such findings refer to abnormalities in surgical specimens and hence, by definition, to subtle pathology that has been removed and can therefore no longer contribute to ictogenesis, e.g. via the “short-circuiting” described in animal models associated with heterotopic neurons (Chevassus-au-Louis et al., 1999). This is consistent with our own previous studies, where temporal lobe white matter increases of flumazenil had been seen bilaterally, but were only associated with suboptimal outcome in the contralateral temporal lobe (Hammers et al., 2001). Similarly, in (Yankam Njiwa et al., 2013) and the present manuscript we describe the same in vivo biomarker in a remote area not removed during surgery.

Both tracers and methods did not always overlap in their prediction of seizure status for individual patients. Periventricular WM signal increases are not always seen at the same locations for both tracers. [18F]FDG uptake assesses metabolism, a more general aspect of brain function than the more specifically neuron-linked [11C]FMZ binding; this could explain the fact that [18F]FDG uptake was less predictive of outcome (see Table 1).

Other studies showed that the complete postoperative absence of seizures widely varies with the considered postoperative time point (Aull-Watschinger et al., 2008; Elsharkawy et al., 2009; Kelemen et al., 2006). For example, in a study of 153 patients with MRI-diagnosed HS, the gender, the seizure frequency before operation, and the unilaterality of interictal EEG discharges were used as variables in a logistic regression equation for predicting outcome after surgery (Aull-Watschinger et al., 2008). Complete seizure freedom (Engel class IA) could not be predicted from conventional preoperative variables; our study provides another element allowing such a prediction even if its accuracy remains limited.

In summary, classification based on RF performed slightly better than the previous method based on periventricular WM signal increases for both used tracers. The accuracy may be sufficient to aid in individual preoperative counselling. However, the classifier constructed will need to be tested on an independent sample.

Acknowledgement

We would like to thank Pr. Rolf. A Heckemann for his help in using R. Dr. Sandrine Bouvard and Dr. Hélène Catenoix for their help regarding the database and data collection.

References

  1. Amit Y., Geman D. Shape quantization and recognition with randomized trees. Neural Computation. 1997;9(7):1545–1588. [Google Scholar]
  2. Ashburner J., Friston K.J. Nonlinear spatial normalization using basis functions. Hum. Brain Mapp. 1999;7(4):254–266. doi: 10.1002/(SICI)1097-0193(1999)7:4&#x0003c;254::AID-HBM4&#x0003e;3.0.CO;2-G. 10408769 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Asztely F. Long term follow-up of the first 70 operated adults in the Göteborg epilepsy surgery series with respect to seizures, psychosocial outcome and use of antiepileptic drugs. J. Neurol. Neurosurg. Psychiatry. 2007;78(6):605–609. doi: 10.1136/jnnp.2006.098244. 17237145 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aull-Watschinger S. Outcome predictors for surgical treatment of temporal lobe epilepsy with hippocampal sclerosis. Epilepsia. 2008;49(8):1308–1316. doi: 10.1111/j.1528-1167.2008.01732.x. 18754943 [DOI] [PubMed] [Google Scholar]
  5. Blümcke I. A new clinico-pathological classification system for mesial temporal sclerosis. Acta Neuropathol. 2007;113(3):235–244. doi: 10.1007/s00401-006-0187-0. 17221203 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Blümcke I. The clinicopathologic spectrum of focal cortical dysplasias: a consensus classification proposed by an ad hoc task Force of the ILAE Diagnostic Methods Commission. Epilepsia. 2011;52(1):158–174. doi: 10.1111/j.1528-1167.2010.02777.x. 21219302 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Borelli P. Extratemporal ictal clinical features in hippocampal sclerosis: their relationship to the degree of hippocampal volume loss and to the outcome of temporal lobectomy. Epilepsia. 2008;49(8):1333–1339. doi: 10.1111/j.1528-1167.2008.01694.x. 18557777 [DOI] [PubMed] [Google Scholar]
  8. Bouvard S. Test–retest reliability of [11C]flumazenil data acquired using the Delforge partial saturation method. J. Cereb. Blood Flow Metab. 2012;32:S167. [Google Scholar]
  9. Breiman L. Classification and Regression Trees. Wadsworth, Belmont; 1984. [Google Scholar]
  10. Breiman L. Bagging predictors. Mach. Learn. 1996;24(2):123–140. [Google Scholar]
  11. Breiman L. Random forests. Mach. Learn. 2001;45(1):5–32. [Google Scholar]
  12. Brix G. Performance evaluation of a whole-body PET scanner using the NEMA protocol. National Electrical Manufacturers Association. J Nucl Med. 1997;38:1614–1623. 9379202 [PubMed] [Google Scholar]
  13. Chassoux F. FDG-PET improves surgical outcome in negative MRI Taylor-type focal cortical dysplasias. Neurology. 2010;75(24):2168–2175. doi: 10.1212/WNL.0b013e31820203a9. 21172840 [DOI] [PubMed] [Google Scholar]
  14. Chevassus-au-Louis N., Represa A. The right neuron at the wrong place: biology of heterotopic neurons in cortical neuronal migration disorders, with special reference to associated pathologies. Cell. Mol. Life Sci. 1999;55(10):1206–1215. doi: 10.1007/s000180050367. 10487203 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. De Lanerolle N.C. A retrospective analysis of hippocampal pathology in human temporal lobe epilepsy: evidence for distinctive patient subcategories. Epilepsia. 2003;44(5):677–687. doi: 10.1046/j.1528-1157.2003.32701.x. 12752467 [DOI] [PubMed] [Google Scholar]
  16. Delforge J. Quantification of benzodiazepine receptors in human brain using PET, [11C]flumazenil, and a single-experiment protocol. J. Cereb. Blood Flow Metab. 1995;15(2):284–300. doi: 10.1038/jcbfm.1995.34. 7860662 [DOI] [PubMed] [Google Scholar]
  17. Didelot A. Voxel-based analysis of asymmetry index maps increases the specificity of 18F-MPPF PET abnormalities for localizing the epileptogenic zone in temporal lobe epilepsies. J. Nucl. Med. 2010;51(11):1732–1739. doi: 10.2967/jnumed.109.070938. 21051649 [DOI] [PubMed] [Google Scholar]
  18. Elsharkawy A.E. Long-term outcome after temporal lobe epilepsy surgery in 434 consecutive adult patients. J. Neurosurg. 2009;110(6):1135–1146. doi: 10.3171/2008.6.JNS17613. 19025359 [DOI] [PubMed] [Google Scholar]
  19. Engel J., Jr. Presurgical evaluation for partial epilepsy: relative contributions of chronic depth-electrode recordings versus FDG-PET and scalp-sphenoidal ictal EEG. Neurol. 1990;40(11):1670–1677. doi: 10.1212/wnl.40.11.1670. 2122275 [DOI] [PubMed] [Google Scholar]
  20. Engel J., Jr. Practice parameter: temporal lobe and localized neocortical resections for epilepsy: report of the Quality Standards Subcommittee of the American Academy of Neurology, in association with the American Epilepsy Society and the American Association of Neurological Surgeons. Neurol. 2003;60(4):538–547. doi: 10.1212/01.wnl.0000055086.35806.2d. 12601090 [DOI] [PubMed] [Google Scholar]
  21. Engel J.J. Outcome with respect to seizures. In: Engel J.J., editor. Surgical Treatment of the Epilepsies. Raven; New York: 1987. pp. 553–572. [Google Scholar]
  22. Feis D.L. Prediction of post-surgical seizure outcome in left mesial temporal lobe epilepsy. Neuroimage Clin. 2013;2:903–911. doi: 10.1016/j.nicl.2013.06.010. 24179841 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Ferrari-Marinho T. Auras in temporal lobe epilepsy with hippocampal sclerosis: relation to seizure focus laterality and post surgical outcome. Epilepsy Behav. 2012;24(1):120–125. doi: 10.1016/j.yebeh.2012.03.008. 22520586 [DOI] [PubMed] [Google Scholar]
  24. Gleeson J.G., Walsh C.A. Neuronal migration disorders: from genetic diseases to developmental mechanisms. Trends Neurosci. 2000;23(8):352–359. doi: 10.1016/s0166-2236(00)01607-6. 10906798 [DOI] [PubMed] [Google Scholar]
  25. Hammers A. Neocortical abnormalities of [11C]-flumazenil PET in mesial temporal lobe epilepsy. Neurology. 2001;56(7):897–906. doi: 10.1212/wnl.56.7.897. 11294927 [DOI] [PubMed] [Google Scholar]
  26. Hammers A. Abnormalities of grey and white matter [11C]flumazenil binding in temporal lobe epilepsy with normal MRI. Brain. 2002;125(10):2257–2271. doi: 10.1093/brain/awf233. 12244083 [DOI] [PubMed] [Google Scholar]
  27. Hammers A. Grey and white matter flumazenil binding in neocortical epilepsy with normal MRI. A PET study of 44 patients. Brain. 2003;126(6):1300–1318. doi: 10.1093/brain/awg138. 12764053 [DOI] [PubMed] [Google Scholar]
  28. Hammers A. Periventricular white matter flumazenil binding and postoperative outcome in hippocampal sclerosis. Epilepsia. 2005;46(6):944–948. doi: 10.1111/j.1528-1167.2005.30904.x. 15946336 [DOI] [PubMed] [Google Scholar]
  29. Henry T.R., Mazziotta J.C., Engel J., Jr. Interictal metabolic anatomy of mesial temporal lobe epilepsy. Arch. Neurol. 1993;50(6):582–589. doi: 10.1001/archneur.1993.00540060022011. 8503794 [DOI] [PubMed] [Google Scholar]
  30. Ho T. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 1998;20(8):832–844. [Google Scholar]
  31. Janszky J. Temporal lobe epilepsy with hippocampal sclerosis: predictors for long-term surgical outcome. Brain. 2005;128(2):395–404. doi: 10.1093/brain/awh358. 15634733 [DOI] [PubMed] [Google Scholar]
  32. Jeong S.W. Prognostic factors for the surgery for mesial temporal lobe epilepsy: longitudinal analysis. Epilepsia. 2005;46(8):1273–1279. doi: 10.1111/j.1528-1167.2005.33504.x. 16060939 [DOI] [PubMed] [Google Scholar]
  33. Kelemen A. [Predictive factors for the results of surgical treatment in temporal lobe epilepsy] Ideggyogy. Sz. 2006;59(9-10):353–359. 17165378 [PubMed] [Google Scholar]
  34. Kuba R. Grey-white matter abnormalities in temporal lobe epilepsy associated with hippocampal sclerosis: inter-observer analysis, histopathological findings, and correlation with clinical variables. Epilepsy Res. 2012;102(1-2):78–85. doi: 10.1016/j.eplepsyres.2012.05.002. 22738717 [DOI] [PubMed] [Google Scholar]
  35. Malmgren K., Thom M. Hippocampal sclerosis—origins and imaging. Epilepsia. 2012;53(Suppl. 4):19–33. doi: 10.1111/j.1528-1167.2012.03610.x. 22946718 [DOI] [PubMed] [Google Scholar]
  36. Maziere M. Synthesis of ethyl 8-fluoro-5,6-dihydro-5-[11C]methyl-6-oxo-4H-imidazo [1,5-a] [1,4]benzodiazepine-3-carboxylate (RO 15.1788-11C): a specific radioligand for the in vivo study of central benzodiazepine receptors by positron emission tomography. Int. J. Appl. Radiat. Isot. 1984;35(10):973–976. doi: 10.1016/0020-708x(84)90215-1. 6094361 [DOI] [PubMed] [Google Scholar]
  37. Najm I. Temporal patterns and mechanisms of epilepsy surgery failure. Epilepsia. 2013;54(5):772–782. doi: 10.1111/epi.12152. 23586531 [DOI] [PubMed] [Google Scholar]
  38. Parent J.M., Valentin V.V., Lowenstein D.H. Prolonged seizures increase proliferating neuroblasts in the adult rat subventricular zone-olfactory bulb pathway. J. Neurosci. 2002;22(8):3174–3188. doi: 10.1523/JNEUROSCI.22-08-03174.2002. 20026296] [Pubmed: 11943819] [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Reid A.Y., Staba R.J. Limbic networks: clinical perspective. Int. Rev. Neurobiol. 2014;114:89–120. doi: 10.1016/B978-0-12-418693-4.00005-4. 25078500 [DOI] [PubMed] [Google Scholar]
  40. Ryvlin P., Kahane P. The hidden causes of surgery-resistant temporal lobe epilepsy: extratemporal or temporal plus? Curr. Opin. Neurol. 2005;18(2):125–127. doi: 10.1097/01.wco.0000162852.22026.6f. 15791141 [DOI] [PubMed] [Google Scholar]
  41. Sagar H.J., Oxbury J.M. Hippocampal neuron loss in temporal lobe epilepsy: correlation with early childhood convulsions. Ann. Neurol. 1987;22(3):334–340. doi: 10.1002/ana.410220309. 3674798 [DOI] [PubMed] [Google Scholar]
  42. Spencer S., Huh L. Outcomes of epilepsy surgery in adults and children. Lancet Neurol. 2008;7(6):525–537. doi: 10.1016/S1474-4422(08)70109-1. 18485316 [DOI] [PubMed] [Google Scholar]
  43. Takahashi M. Voxel-based comparison of preoperative FDG-PET between mesial temporal lobe epilepsy patients with and without postoperative seizure-free outcomes. Ann. Nucl. Med. 2012;26(9):698–706. doi: 10.1007/s12149-012-0629-9. 22810894 [DOI] [PubMed] [Google Scholar]
  44. Theodore W.H. Temporal lobectomy for uncontrolled seizures: the role of positron emission tomography. Ann. Neurol. 1992;32(6):789–794. doi: 10.1002/ana.410320613. 1471870 [DOI] [PubMed] [Google Scholar]
  45. Theodore W.H. Positron emission tomographic measurement of cerebral blood flow and temporal lobectomy. Ann. Neurol. 1994;36(2):241–244. doi: 10.1002/ana.410360220. 8053663 [DOI] [PubMed] [Google Scholar]
  46. Thom M., Liagkouras I., Elliot K.J. Reliability of patterns of hippocampal sclerosis as predictors of postsurgical outcome. Epilepsia. 2010;51:1801–1808. doi: 10.1111/j.1528-1167.2010.02681.x. [DOI] [PubMed] [Google Scholar]
  47. Yankam Njiwa J. Periventricular [11C]flumazenil binding for predicting postoperative outcome in individual patients with temporal lobe epilepsy and hippocampal sclerosis. Neuroimage Clin. 2013;3:242–248. doi: 10.1016/j.nicl.2013.07.008. 24273709 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from NeuroImage : Clinical are provided here courtesy of Elsevier

RESOURCES