How much are clinical fMRI reports influenced by standard postprocessing methods? An investigation of normalization and region of interest effects in the medial temporal lobe

Abstract

Recent evidence has indicated that standard postprocessing methods such as template‐based region of interest (ROI) definition and normalization of individual brains to a standard template may influence final outcome of functional magnetic resonance imaging investigations. Here, we provide the first comprehensive investigation into whether ROI definition and normalization may also change the clinical interpretation of patient data. A series of medial temporal lobe epilepsy patients were investigated with a clinical memory paradigm and individually delineated as well as template‐based ROIs. Different metrics for activation quantification were applied. Results show that the application of template‐based ROIs can significantly change the clinical interpretation of individual patient data. This relates to sensitivity for brain activation and hemispheric dominance. We conclude that individual ROIs should be defined on nontransformed functional data and that use of more than one metric for activation quantification is beneficial. Hum Brain Mapp, 2010. © 2010 Wiley‐Liss, Inc.

INTRODUCTION

Several studies have demonstrated the clinical applicability of functional magnetic resonance imaging (fMRI) for investigation of memory activation in the medial temporal lobe (MTL) [Baxter et al.;2007; Beisteiner et al.;2008; Dickerson and Sperling, 2008; Frings et al., 2008; Golby et al., 2002; Jansen et al., 2009; Jokeit et al., 2001; Killgore et al., 1999; Kircher et al., 2007; Koenig et al., 2008; Trivedi et al., 2008]. A common clinical application is the presurgical localization of memory areas in medial temporal lobe epilepsy (MTLE) and tumour patients. Besides precise localization of memory areas within the MTL, hemispheric dominance is a major clinical issue. Results of comprehensive presurgical diagnostics (including fMRI) influence decisions whether and how to conduct surgery.

A common procedure with fMRI investigations involves normalization of individual brains to a standard template [Fox, 1995; Friston et al., 1995] and the evaluation of activation measures in standardized regions of interest (ROIs) in relevant MTL areas. For ROI definition, the Wake Forest University PickAtlas (WFU) [Maldjian et al., 2003, 2004] or Anatomical Automatic Labeling (AAL) atlas [Tzourio‐Mazoyer et al., 2002] are most popular [Branco et al., 2006; Dickie et al., 2008; Frings et al., 2008; Hurt et al., 2008; Ischebeck et al., 2006; Tsukiura and Cabeza, 2008; Watanabe et al., 2008]. Recently, several authors argued that such automated techniques of localizing functional activation to neuroanatomical structures might be problematic [Devlin and Poldrack, 2007; Fadiga, 2007; Gartus et al., 2007; Geissler et al., 2005; Hoeksma et al., 2005; Krishnan et al., 2006; Mitsis et al., 2008; Rodionov et al., 2009; Vandenbroucke et al., 2004]. For instance, Rodionov et al. [2009] compared automatic ROI definition techniques for the hippocampus based on three brain atlases (Pick Atlas Brodmann areas, AAL atlas, frequency‐based Hammers atlas). Comparing with an individual hippocampus delineation as gold standard, they found considerable errors with the automatically defined ROIs: volumetric errors of up to 107%, false‐positive volume classifications of up to 60%, and false‐negative volumes of up to 72%. In line with this investigation, Mitsis et al. [2008] have demonstrated that the use of ROIs defined differently may generate considerably different experimental results in the same data set. The authors compared various anatomical and functional ROIs, defined for each subject, at group level or using a Talairach‐like atlas. An investigation concerning functional‐anatomical coregistration procedures in healthy and pathological brains found errors of centimeters with pathological brains [Gartus et al., 2007]. Clinically relevant mislocalization of activation has also been shown to arise from smoothing [Geissler et al., 2005]. Specific problems in temporal lobe patients concern distortions and signal voids due to susceptibility artifacts close to the mastoids, and pathology‐related signal changes, which are often asymmetric. Both factors complicate image registration processes. For instance, Krishnan et al. [2006] investigated the effect of normalization procedures in memory‐impaired subjects and described decreased normalization accuracy as a “potentially important confounder of template‐based fMRI analyses in the hippocampus and MTL”. Vandenbroucke et al. [2004] investigated face encoding in healthy elderly men. They found MTL activity in 18 of 29 subjects when analyzing data in individual native space. After normalization of the data to standard space [Talairach and Tournoux, 1988], no region was significantly activated on a group level, related to intersubject variability. A further study showed that the accuracy of registration may depend on the age of the subject population [Hoeksma et al., 2005].

Another clinically important issue—not yet thoroughly investigated—concerns correct definition of true positive and true negative activations. With current clinical fMRI investigations, various kinds of thresholded data analysis are commonly used [Chmayssani et al., 2009; Matsuo et al., 2007; Sanchez‐Carrion et al., 2008; Schoning et al., 2009; Tie et al., 2008; Werner et al., 2009]. Since the medical report requires a decision where borders between active and inactive brain areas are, there is often no alternative to thresholding data. The major problem with application of standard threshold conventions is that they result in large variability of single session results [Beisteiner et al., 1997, 2000; McGonigle et al., 2000]. One solution to this problem is to maximize functional information during a single session by the use of many repeats of runs. By evaluating such data at different thresholds, the “true response” of a subject may be detected with increased certainty, since threshold‐related map variability is considerably reduced [Beisteiner et al., 2000, 2001; Roessler et al., 2005]. Following these early investigations, such interrelations have been increasingly recognized and have also been nicely demonstrated by a recent reanalysis of the McGonigle et al. [2000] data by Smith et al. [2005]. An alternative approach avoids thresholding of functional signals by calculation of representative activation values over all voxels within a preselected ROI.

All these issues are of major importance for one of the largest groups of candidates for presurgical memory fMRI — patients with MTLE planned for selective amygdalohippocampectomies. Presurgical fMRI is increasingly performed with such patients to define memory localization and standard postprocessing methods are often used (see above). However, currently it is not known whether the problems just described may influence individual fMRI reports in a clinically significant way. For instance, it would not be a significant clinical problem if systematic inaccuracies exist which do not influence final conclusions about hemispheric dominance or activated structures. On the contrary, errors changing such conclusions could be disastrous for the patient.

Here, we provide the first comprehensive investigation into whether different postprocessing methods may generate a clinically relevant change of fMRI results in the MTL. Partly based on the outcome of previous studies, we concentrated on three important factors: the choice of ROIs, the application of normalization procedures, and the metric for activation quantification. The specific questions for this investigation were as follows: Does the sensitivity for memory‐related brain activation or lateralization change depending on (1) the technique for ROI delineation, (2) normalization of individual brains to a standard template, or (3) the metric for activation quantification? If sensitivity changes, is the change clinically and statistically significant?

Our work extends previous studies by investigating possible consequences for the clinical handling of individual patients. Clinical handling depends on the localization of memory activations and determination of which hemisphere dominates. Furthermore, this study of normalization and ROI effects investigates all relevant MTL areas comprehensively and evaluates different approaches for definition of true positive voxels. To allow a clinically realistic assessment of relevant effects, we investigated a series of patients with MTLE, as the patient group for whom fMRI memory reports were most often requested.

METHODS

Patients

The study was approved by the local ethics committee, and all patients gave written informed consent. Thirty‐three patients, consecutively transferred for presurgical memory localization and suffering from medically refractory unilateral MTLE, were included (for patient characteristics see Table I). Handedness was assessed via the modified Edinburgh Inventory Test [Salmaso and Longoni, 1985]. We intended to investigate a consecutive series of patients with MTLE as they were transferred, but not a homogenous patient population. Therefore, the only inclusion criterion was that clinicians needed fMRI memory mapping and decided to request this fMRI report from the authors (Study Group Clinical fMRI). These requests were in no way influenced by the authors of this study. This was done to see how the techniques work when applied in a realistic clinical environment — where every patient has to be investigated independently of the status of the disease. Therefore, we do not intend to draw any conclusions about advantages or disadvantages of the various methods with respect to homogeneous subgroups of patients with epilepsy. MTLE was defined by unitemporal interical and ictal EEG changes, as well as by unilateral hippocampal atrophy or sclerosis on high‐resolution MRI. All patients were able to perform the given task and showed no claustrophobic and/or unexpected reactions.

Table I.

Demographics of the 33 patients with MTLE

	Mean age	Age range	Patient characteristics
			Handedness		Side of pathology
			Right	Left	Right	Left
Male	38	21–65	15	2	9	8
Female	36	16–54	10	6	4	12

Patient	Side of brain pathology	Sex	Age	Handedness
BA	Left	m	42	r
BAU	Left	f	22	l
CO	Left	f	46	r
DE	Left	f	41	r
DI	Left	f	37	l
DO	Left	m	37	l
HA	Left	f	25	l
HE	Left	f	25	r
HO	Left	m	50	r
KA	Left	f	37	r
KAU	Left	f	23	r
KE	Left	f	39	l
KER	Left	m	34	r
KO	Left	f	27	r
KOR	Left	m	39	r
LU	Left	m	31	r
ME	Left	m	21	r
SC	Left	m	60	l
SE	Left	f	16	r
TU	Left	f	42	l
AM	Right	f	54	r
AW	Right	m	22	r
DOR	Right	m	36	r
FR	Right	m	37	r
GR	Right	m	45	r
HAI	Right	m	32	r
PE	Right	f	40	l
PO	Right	m	41	r
SA	Right	m	34	r
ST	Right	m	65	r
UL	Right	f	51	r
WA	Right	f	53	r
ZN	Right	m	24	r

Task

Each patient performed a hometown‐walking task [Beisteiner et al., 2008] similar to previous descriptions [Jokeit et al., 2001]. The task comprised mental walking along an individually generated set of 15 selected routes in their hometown, presented in a pseudorandomized order. Corresponding to previous literature, subjects were asked to imagine as many details as possible in a three‐dimensional manner while navigating. Subjects had a similar level of familiarity with the routes under consideration and had been acquainted with the localities for a number of years. Patients were trained in the task by a neuropsychologist several days before the fMRI investigation. Silent backward counting [instead of odd number counting in Jokeit et al., 2001] was used as a baseline condition, which is easily accomplishable by patients. The timing for the task onset and offset was controlled by oral commands. Subjects were asked to minimize head movements. Correct execution of the memory task was assessed by asking questions about the navigated route after each fMRI run with the scanner stopped (e.g. “Did you pass the red signpost?”). Patients had to give yes/no responses that were scored.

MRI Acquisition

Each session included 15 block‐designed BOLD fMRI runs with four baseline and three activation periods of 20 s each. To minimize head motion artifacts, individually constructed plaster cast helmets [Edward et al., 2000] were used for optimized and secure head fixation. All scans were performed on a 3‐T BRUKER Medspec scanner with a standard birdcage head coil. High‐resolution sagittal T₁‐weighted MR images were acquired with an MPRAGE sequence (TE: 3.9 ms, TR: 21.6 ms, flip angle: 15°, matrix size: 256 × 256 × 128, FOV: 230 mm, axial slices: 1.5 mm thickness) to determine the imaging plane for the functional measurements and to provide a T₁‐weighted anatomical brain volume for neuroanatomical analyses. For functional imaging, a phase‐corrected blipped single shot gradient echo EPI sequence was used, optimized for the local scanner characteristics (FOV: 230 mm, matrix size: 128 × 128, 35 coronal slices of 3 mm thickness, no gap, TE: 55 ms, TR: 5 s, receiver bandwidth per pixel in phase encoding direction (PE) 20 Hz, PE direction head–foot (BRUKER convention)). Two dummy/preparation scans prefaced each run to ensure quasi‐equilibrium in longitudinal magnetization. The hippocampus and parahippocampal structures were completely covered by 35 coronal slices along the longitudinal hippocampal axis. During each 20 s block, four volumes were recorded, summarizing to 28 volumes per run (7 blocks) and 420 volumes per patient (15 runs).

fMRI Data Processing

Imaging data were analyzed with Statistical Parametric Mapping 5 (SPM5) (Wellcome Department of Imaging Neuroscience, London, UK; http://www.fil.ucl.ac.uk/spm). To account for residual small‐scale motion, motion correction was carried out for each patient (default settings, except: “Quality: 0.95,” “Separation: 2,” “Num passes: register to first,” to improve correction quality). All functional images (voxel size: 1.8 mm × 1.8 mm × 3 mm = 9.72 mm³) were individually normalized (voxel size of normalized functional images: 2 mm × 2 mm × 2 mm = 8 mm³). We strictly applied the default SPM5 settings to keep the normalization procedure as standardized as possible and compared the results of a single‐step normalization to the SPM5 EPI template with a double‐step normalization to the skull stripped MNI 152 anatomical template (1 mm resolution). The latter requires a registration of the individual functional images to the individual (skull stripped) anatomical images and then normalization of individual anatomical images to the anatomical template [Brett et al., 2002b]. To support both the normalization and coregistration algorithm finding the global optimum, each subject's dataset was coarsely aligned to the corresponding template as a starting estimate. Skull stripping was done using “bet2” [Smith2002]. During later data analysis, ROI evaluations were performed with normalized and nonnormalized data sets. Functional data (normalized and nonnormalized) were smoothed by a Gaussian‐smoothing kernel (4 mm × 4 mm × 6 mm FWHM). Statistical analysis (1st level) was performed by means of a mixed‐effects analysis using SPM's default settings — this includes generation of a signal intensity‐based brain mask, high‐pass filtering (cut‐off period 128 s), and correction for serial correlations using an autoregressive AR(1) model. BOLD responses were modeled by a fixed response boxcar function convolved with a hemodynamic response. All 15 runs performed by a patient were submitted to a combined analysis by appropriate contrast settings. Individual brain activation was established with voxel‐wise t‐tests to generate individual SPM t‐maps (first‐level analysis).

ROI Definitions

Four types of ROI definitions were used for comparison. ROI structures in the MTL were selected according to the functional results of previous studies applying the hometown‐walking. The most relevant structures are the parahippocampal gyrus and fusifom gyrus which comprise the collateral sulcus.

Individual nonnormalized ROIs

Individual ROIs within the MTL were defined as in [Beisteiner et al., 2008] by a qualified neuroanatomical expert. To ease delineation of neuroanatomical structures on functional images, the original functional EPI images were interpolated to a matrix of twice the size of the original, via “manualreslice” which is part of the AIR 3.08 software package [Woods et al., 1998a, b]. In a first step, the hippocampus, fusiform gyrus, and parahippocampal gyrus were outlined on anatomical images using MRIcro [Rorden and Brett, 2000] and neuroantomical atlases [Duvernoy, 1999; Ono et al., 1990]. In a second step, a comprehensive ROI was manually delineated on the interpolated version of the functional images (Fig. 1). This single ROI included the hippocampal formation (including the hippocampus, dentate gyrus, and the beginning of the subiculum), the parahippocampal gyrus, and the fusiform gyrus. On each of the two hemispheres one such single ROI was independently outlined. At last, these ROIs were retransformed to the original matrix resolution for quantitative analyses.

Figure 1.

Example of delineation of individual nonnormalized ROIs on nonnormalized functional EPI images. Two adjacent coronal slices are shown (Patient GR) in radiological convention. (A) Anatomical images, (B) interpolated nonnormalized functional images, (C) nonnormalized functional images with individual ROIs (not interpolated), and (D) Same as (C) but with MTL activation overlaid (thresholded ROI analysis).

Individual normalized ROIs

All individual nonnormalized ROIs as defined earlier were normalized using the subject‐specific transformation parameters obtained during the normalization procedure as described earlier (Fig. 2). This included a matrix transformation to a voxel‐size of 2 mm × 2 mm × 2 mm. After normalization, all ROI's were smoothed using a 2 mm × 2 mm × 2 mm FWHM kernel to even discontinuities. This kernel size produced an acceptable trade‐off between eliminating ROI chipping and ROI enlargement.

Figure 2.

Comparison of normalized ROIs (patient KO). From left to right: coronal section of normalized functional EPI image (radiological convention), individual normalized ROIs shown in green, mean group ROIs shown in purple, standard ROI shown in blue. The dotted line indicates the upper border of the sub‐ROIs used to analyze neuroanatomical coverage of basal MTL (see text).

Mean group ROI

All ROIs generated in individual normalized ROIs were used to define a mean group ROI. For this, a superposition of all individual normalized ROI volumes was generated. Then, every voxel within the superposition volume was checked for overlap of individual ROIs. The criterion for including a voxel in the mean group ROI was an overlap of individual ROIs of at least 50% (Fig. 2).

Standard ROI

Standard ROIs were defined using the widely applied AAL package [Tzourio‐Mazoyer et al., 2002], also available for SPM5 (http://www.cyceron.fr/web/software.html). The AAL library is part of the MarsBaR [Brett et al., 2002a] MATLAB toolbox and the WFU, PickAtlas [Maldjian et al., 2003, 2004]. The AAL package includes a digital human brain atlas that can be used autonomously from within SPM and MATLAB. This atlas identifies neuroanatomical ROIs by numbers (e.g. left/right hippocampus = 4101/4102, left/right parahippocampal gyrus = 4111/4112, left/right fusiform gyrus = 5401/5402). Corresponding to the individual ROI definitions, we performed a combined ROI analysis of the left and right hippocampus, parahippocampal gyrus, and fusiform gyrus ROIs. This was done by analyzing each AAL ROI and then combining the values to form a comprehensive left hemispheric or right hemispheric standard ROI (Fig. 2).

Metric for Activation Quantification

To assess the influence of alternative techniques for definition of functional activation, we compared two thresholded analysis techniques and one nonthresholded ROI analysis technique. As a measure of functional activation, activity indices for the left‐hemispheric (A _L) and right‐hemispheric (A _R) ROIs were calculated considering all voxels within a ROI included in the SPM signal intensity‐based brain mask generated during first‐level statistical analysis.

Thresholded whole brain analysis

Activity indices A _L,R were derived from the SPM5 whole brain first level data analysis (P ≤ 0.05, family wise error (FWE) correction). The activity indices A_L,R were determined for each patient as the number of supra‐threshold voxels in each ROI, for left and right hemispheric ROIs separately (given as voxel counts in Table II).

Table II.

Effect of ROI definition on the activity indices (A _L and A _R taken together)

Statistical differences between ROI activity indices (P‐values, two‐sided paired t‐test)
	Thresholded whole brain analysis (voxel count)	Thresholded ROI analysis (t‐sum)	Nonthresholded ROI analysis (–log(p) − sum)
Individual nonnormalized ROI vs. individual normalized ROI	0.024	6 × 10 −10	9 × 10 −9
Individual nonnormalized ROI vs. mean group ROI	0.118	4 × 10 −7	2 × 10 −8
Individual nonnormalized ROI vs. standard ROI	2 × 10 −4	0.223	0.313
Individual normalized ROI vs. mean group ROI	0.064	0.003	0.04
Individual normalized ROI vs. standard ROI	1 × 10 −6	2 × 10 −9	1 × 10 −6
Mean group ROI vs. standard ROI	2 × 10 −6	6 × 10 −11	6 × 10 −9
Mean activity indices
	Mean ± SD	Mean ± SD	Mean ± SD
Individual nonnormalized ROI	690 ± 490	5,700 ± 3,300	11,200 ± 7,700
Individual normalized ROI	820 ± 640	8,700 ± 4,000	19,800 ± 1,205
Mean group ROI	770 ± 590	7,600 ± 3,400	18,300 ± 9,400
Standard ROI	470 ± 340	5,200 ± 2,200	12,000 ± 5,600

Group results comparing the activity indices between ROIs using a two‐sided paired t‐test.

Significant values in bold (Bonferroni corrected statistical significance level: P ≤ 0.0167).

Thresholded ROI analysis

Individually defined and ROI‐specific thresholds were calculated as in [Fernandez et al., 2001]. The t‐values of all voxels within an ROI were determined from the SPM5 whole brain first‐level data analysis. Then, the top 5% t‐values were determined and their mean calculated for each ROI (top 5% t‐mean). Corresponding top 5% t‐means of the left and the right hemisphere ROIs were then averaged. The ROI‐specific threshold for active voxels was set as 50% of the averaged top 5% t‐means. This threshold calculation was done for every ROI definition (individual nonnormalized ROI, individual normalized ROI, mean group ROI, standard ROI). The procedure results in individual and ROI‐specific threshold settings adapted according to the patients general activation level. As a last step, for every ROI the t‐values of all suprathreshold voxels were summed up, generating the t‐sums given in Table II.

Nonthresholded ROI analysis

To calculate nonthresholded activity indices, the following procedure was applied [Benson et al., 1999]. The P‐values of all voxels within an ROI were determined from the SPM5 whole brain first‐level data analysis. All these P‐values were transformed to −log(p). The rationale for the procedure is that numeric P‐values (range 0–1) are very small and they differ only marginally. Thus, a −log(p) transformation expands the range and emphasizes small P‐values (the full range becomes 0–∞). Voxels with high P‐values (noise) are suppressed. For every ROI, the −log(p) values of all voxels were added. This sum was calculated for all left hemispheric ROIs representing A_L and all right hemispheric ROIs representing A_R and is given as −log(p) − sum in Table II.

Calculation of Lateralization Index

An individual lateralization index LI was calculated, defined as

This was performed for all three metrics for activation quantification (thresholded whole brain analysis, thresholded ROI analysis, nonthresholded ROI analysis) and for each ROI definition separately. The lateralization index LI can range between −1 for only right‐hemispheric activation and +1 for only left‐hemispheric activation.

Statistical Analysis

All activity indices A _L, A _R and lateralization indices were calculated for each ROI, activation quantification metric and patient separately. Since the values of A _L,R were normally distributed (established by Kolmogorov–Smirnov and Shapiro–Wilk tests), t‐tests were used for further statistical evaluations using SPSS 10 (http://www.spss.com). The only exception concerned the distribution of active voxels in the analysis shown in Table IV. Here, statistical evaluation was done with a Wilcoxon test.

Table IV.

Effect of ROI definition on coverage of active MTL voxels

Statistical differences between the number of active voxels covered (thresholded whole brain analysis, P‐values, two‐sided paired t‐test)
Functional ROI vs. standard ROI	2 × 10 −5
Functional ROI vs. individual normalized ROI	0.288
Functional ROI vs. mean group ROI	0.131
Mean number of active voxels covered
	Mean ± SD
Functional ROI	544 ± 454
Standard ROI	376 ± 286
Individual Normalized ROI	542 ± 453
Mean Group ROI	538 ± 452

Group results comparing the functional ROI (comprising all active MTL voxels of the thresholded whole brain analysis) and the neuroanatomical ROIs using a two‐sided paired t‐test. Significant values in bold.

ROI Differences Concerning the Activity Indices A _L,R

To evaluate whether the four ROI definition techniques generated significantly different activation results, individual activity indices were submitted to a statistical evaluation. All four ROI definitions were compared (Table II).

ROI Differences Concerning Coverage of Relevant Neuroanatomical Structures

Coverage of all relevant neuroanatomical structures is evidently the goal of manual delineation of ROIs on original functional EPI data by the individual nonnormalized ROIs. However, it is not clear whether complete coverage is also achieved with other ROI definition techniques (standard ROIs, mean group ROI, individual normalized ROIs). To evaluate possible differences in structural coverage, ROIs defined on normalized data (individual normalized ROI, mean group ROI, and standard ROI) were visually assessed to establish if target structures were fully encompassed. This revealed major differences in the individual coverage of basal MTL structures (Fig. 2). To establish if the differences were systematic and significant, we compared the basal extension of these ROIs quantitatively. We concentrated on functionally relevant structures by analyzing only the basal parts of the ROIs (sub‐ROIs). The anterior‐posterior borders of the sub‐ROIs were defined by the extension of the individual normalized ROI of each individual. This avoided inclusion of anterior and posterior neuroanatomical nontarget areas. In addition, the upper ROI boundary was defined at 50% of the vertical extension of the standard ROI. The lower boundary was not limited. These sub‐ROIs overlay the most relevant neuroanatomical structures (i.e. the basal MTL structures). The volumes of these sub‐ROIs were determined (voxel counts) and statistically tested (Student's t‐test, Table III).

Table III.

Effect of ROI definition on neuroanatomical coverage of basal MTL

Statistical differences between the number of neuroanatomical voxels covered by the sub‐ROIs (P‐values, two‐sided paired t‐test)
Individual normalized ROI vs. mean group ROI	0.194
Individual normalized ROI vs. standard ROI	1 × 10 −13
Mean group ROI vs. standard ROI	2 × 10 −16
Mean voxel count
	Mean ± SD
Individual normalized ROI	4,700 ± 1,800
Mean group ROI	4,300 ± 750
Standard ROI	2,200 ± 500

Group results comparing the number of neuroanatomical voxels in the sub‐ROIs described above.

Significant values in bold (two‐sided paired t‐test, Bonferroni corrected statistical significance level: P ≤ 0.0167).

ROI Differences Concerning Coverage of Relevant Activations

Even if normalized ROIs were to provide different coverage of neuroanatomical structures, this would not necessarily lead to relevant differences in functional assessments, that is, whether activated voxels are omitted. To test for this explicitly, it must be established which voxels are active and which are inactive independent of the ROI. A functional ROI was defined for every patient, consisting of all MTL activations generated with the thresholded whole brain analysis (normalized data) (Fig. 3). Then, the amount of functional ROI activity missed by the three normalized ROI definitions was determined on a group level (Table IV) and on an individual level. To check whether the coverage of relevant activations differed significantly between the three neuroanatomical ROIs, a pair‐wise ROI comparison was also performed (Table V). In addition, we tested the spatial overlap of activations detected by the neuroanatomical ROIs to investigate whether ROIs included spatially different parts of memory activations. For this a pairwise ROI comparison was performed (Table VI). We compared the number of overlapping active voxels with the number of nonoverlapping active voxels. This was only done for that ROI of the ROI pair, which covered less activation (compare Table IV).

Figure 3.

Comparison of MTL activation covered by the different ROIs (patient KO). Same conventions as in Figure 2. Individual MTL activations identified in the thresholded whole brain analysis are shown (and formed the functional ROI). Activity outside ROIs is shown in brown.

Table V.

Effect of ROI definition on coverage of active MTL voxels

Statistical differences between the number of active voxels covered (thresholded whole brain analysis, P‐values, two‐sided paired t‐test)
Individual normalized ROI vs. standard ROI	3 × 10 −5
Mean group ROI vs. standard ROI	3 × 10 −5
Individual normalized ROI vs. mean group ROI	0.105
Mean difference of active voxels covered (group mean)
	Mean ± SD
Individual normalized ROI vs. standard ROI	166 ± 195
Mean group ROI vs. standard ROI	162 ± 194
Individual normalized ROI vs. mean group ROI	4 ± 14

Group results comparing the neuroanatomical ROIs using a two‐sided paired t‐test. Significant values in bold.

Table VI.

Spatial overlap of activations detected by the neuroanatomical ROIs

Overlap of activations detected by the ROIs (thresholded whole brain analysis, P‐values, two‐sided paired Wilcoxon test)
	Overlapping active voxels vs.nonoverlapping active voxels
Overlap of standard ROI activations with individual normalized ROI activations	1 × 10 −6
Overlap of standard ROI activations with mean group ROI activations	1 × 10 −6
Overlap of mean group ROI activations with individual normalized ROI activations	1 × 10 −6
Mean values of overlapping and nonoverlapping voxels
	Mean ± SD
Number of active voxels detected by the standard ROI and the individual normalized ROI	375 ± 285
Number of active voxels detected by the standard ROI but not by the individual normalized ROI	1 ± 4
Number of active voxels detected by the standard ROI and the mean group ROI	373 ± 284
Number of active voxels detected by the standard ROI but not by the mean group ROI	3 ± 11
Number of active voxels detected by the mean group ROI and the individual normalized ROI	536 ± 451
Number of active voxels detected by the mean group ROI but not by the individual normalized ROI	6 ± 22

ROI Differences Concerning Lateralization Results

Lateralization indices (LI) were calculated for all three activation quantification methods (thresholded whole brain analysis, thresholded ROI analysis, and nonthresholded ROI analysis). For each result, a two‐sided paired t‐test was performed to test the patient group for significant differences between the four methods of ROI definition (Table VII). To evaluate the effects on individual patients, LIs for all patients and all analysis methods were also calculated.

Table VII.

Effect of ROI definition on the lateralization index LI

Statistical differences between ROI lateralization indices (P‐values, two‐sided paired t‐test)
	Thresholded whole brain analysis (voxel count)	Thresholded ROI analysis (t‐sum)	Nonthresholded ROI analysis (P‐sum)
Individual nonnormalized ROI vs. individual normalized ROI	0.59	0.176	0.14
Individual nonnormalized ROI vs. mean group ROI	0.699	0.344	0.303
Individual nonnormalized ROI vs. standard ROI	0.01	2 × 10 −5	0.001
Individual normalized ROI vs. mean group ROI	0.873	0.843	0.729
Individual normalized ROI vs. standard ROI	0.002	2 × 10 −6	0.002
Mean group ROI vs. standard ROI	0.003	4 × 10 −6	2 × 10 −4
Mean lateralization indices
	Mean ± SD	Mean ± SD	Mean ± SD
Individual Non‐Normalized ROI	0.065 ± 0.3	0.094 ± 0.25	0.076 ± 0.17
Individual Normalized ROI	0.053 ± 0.24	0.075 ± 0.18	0.058 ± 0.14
Mean Group ROI	0.055 ± 0.24	0.076 ± 0.17	0.059 ± 0.11
Standard ROI	−0.031 ± 0.25	−0.014 ± 0.16	0.005 ± 0.12

Group results comparing the LIs between ROIs using a two‐sided paired t‐test. Significant values in bold (Bonferroni corrected statistical significance level: P ≤ 0.0167). A positive LI value means left hemispheric dominance.

Lateralization Differences Depending on the Metric for Activation Quantification

To evaluate the extent to which the metric for activation quantification influences a clinical report about hemispheric dominance, a two‐sided paired t‐test was performed comparing the three methods of activation quantification. This was done for each ROI definition separately (Table VIII).

Table VIII.

Effect of the activation quantification method on the lateralization index LI

Statistical differences between activation quantification methods concerning lateralization Indices (P‐values, two‐sided paired t‐test)
	Individual nonnormalized ROI	Individual normalized ROI	Mean group ROI	Standard ROI
Thresholded whole brain analysis vs. nonthresholded ROI analysis	0.71	0.88	0.90	0.34
Thresholded ROI analysis vs. nonthresholded ROI analysis	0.91	0.90	0.91	7 × 10 −4
Thresholded whole brain analysis vs. thresholded ROI analysis	0.7	0.91	0.84	0.45

Group results comparing the LIs between ROIs using a two‐sided paired t‐test. Significant values in bold (Bonferroni corrected statistical significance level: P ≤ 0.0167).

RESULTS

Subject Perfomance

All patients completed the tasks and the control questions were answered correctly.

General Findings Comparing the Two Normalization Procedures

For all ROI specific analyses, we found a similar behavior of the two normalization procedures. However, there was also a general tendency for increased differences between the Standard ROI and the other ROIs when normalizing to the anatomical template (instead of the EPI template). In the following, detailed results are only given for the EPI template normalization (including tables). This is done for simplification and since the EPI template findings showed smaller ROI differences. Therefore, ROI results based on the EPI template indicate the minimum ROI differences which have to be expected with our data. Results found with the anatomical template normalization are described in the text only.

ROI Differences Concerning the Activity Indices A _L,R

The effect of ROI definition on the activity indices A _L,R is shown in Table II. An analysis of the mean group ROI showed that it excluded about 20% of the individual normalized ROI voxels (mean ± standard deviation: 23.14% ± 10.25%). Pair‐wise comparison of ROIs (activity indices of left and right hemispheric ROIs taken together) showed significant differences in 12/18 comparisons (with both normalization procedures: EPI template normalization, anatomical template normalization). These comparisons include both, effects of the normalization procedure and effects of the ROI delineation. Normalization includes spatial smoothing and a change of the voxel size (matrix size). Comparing the individual nonnormalized and individual normalized ROIs allows an evaluation of the effects of the normalization procedure on a certain ROI delineation. The thresholded ROI analysis and the nonthresholded ROI analysis showed significant normalization effects, while the thresholded whole brain analysis did not (Table II).

Analyzing only normalized ROIs (individual normalized ROI, mean group ROI, standard ROI) allows the effects of ROI definition to be elicited on identically normalized data sets. Here, the largest differences were found for the standard ROI (6/6 comparisons showed significantly lower brain activity with the standard ROI).

Both normalization procedures showed comparable ROI effects, but mean activity indices of normalized ROIs were lower with the anatomical template normalization.

ROI Differences Concerning Coverage of Relevant Neuroanatomical Structures

A qualitative analysis of ROI differences concerning coverage of neuroanatomical target structures showed lower coverage of basal MTL structures with the standard ROI than all other ROIs (Fig. 2). Because of predominant activation of basal parahippocampal structures, such a lack of neuroanatomical coverage is critical for the given and related memory tasks. Consequently, we perfomed a quantitative analysis with sub‐ROIs focusing on the basal MTL structures (parahippocampal gyrus and fusifom gyrus). Table III and Figures 2 and 5 show that the standard ROI comprised a significantly lower part of the relevant volume compared to all other ROIs. This was confirmed by an analysis of individual patients: the standard ROI missed parts of the basal structures in 32/33 cases. At maximum, a single patient ROI missed 68% of relevant neuroanatomical structures (patient AW). The individual normalized ROI and mean group ROI did not differ significantly. These findings were comparable for both normalization procedures although the mean number of neuroanatomical voxels covered was lower with the anatomical template normalization.

Figure 5.

Mismatch in basal extension of the Standard ROI (upper row, blue color), basal extension of the temporal lobe of the MNI 152 anatomical template (upper row) and basal extension of the temporal lobe of the EPI template (lower row). Corresponding transversal sections.

ROI Differences Concerning Coverage of Relevant Activations

The critical question of whether differences in neuroanatomical ROI coverage are functionally relevant is answered in Table IV and Figure 3. Results show that the standard ROI significantly misses brain activation (31% on average (= 168 voxels), Table IV). There was no significant omission of activated MTL voxels with the individual normalized ROI and mean group ROI. Group results are supported by the individual results: in 29/33 patients the standard ROI missed brain activation (individual normalized ROI: 6/33, mean group ROI: 9/33). In the worst case, 57% of activated voxels was missed (patient ZN). A qualitative control of the individual nonnormalized ROIs showed only minor omissions of isolated single voxels due to the smoothing‐related activation extensions. This control was done based on the thresholded whole brain analysis and the outcome corresponds to the findings with the individual normalized ROI.

Between‐ROI differences concerning the number of active voxels detected are shown in Table V. The standard ROI detected significantly less activation than the other two ROIs. On the other hand, there was no significant difference between the individual normalized ROI and the mean group ROI (542/538 active voxels detected, Table IV). Tests in Table VI show that all ROIs overlapped significantly and therefore detected comparable parts of the brain activation.

Both normalization procedures showed comparable results, with one exception. All normalized ROIs generated with the anatomical template normalization significantly missed some brain activation (corresponding P‐values in Table IV: standard ROI, 6 × 10⁻⁷, individual normalized ROI: 0.003, mean group ROI: 0.024).

ROI Differences Concerning Lateralization Results

The effect of ROI definition on the lateralization index LI is shown in Table VII. Mean LI values indicate a varying group lateralization depending on the ROI definition. Again, the Standard ROI differs significantly from all other ROIs. This was true for all three activation quantification methods (thresholded whole brain analysis, thresholded ROI analysis, nonthresholded ROI analysis). With the thresholded analyses, there is even a change in the hemisphere identified as being dominant: applying standard ROIs, the clinical conclusion would be that the right hemisphere is most important for the given memory task, with all other ROIs the conclusion would be that the left hemisphere dominates. On the individual level, many LIs did not indicate significant lateralization of function (defined as 0.2 < LI < −0.2 according to literature [Springer et al., 1999]).

However, a dependence of the LI on the ROI definition was evident in most participants (Table IX). The lateralization tendency changed for several patients from left to right or vice versa. When accepting only significant changes we found up to nine patients per activation quantification metric, who changed lateralization significantly. In the latter context “significant” means change of the dominant hemisphere (0.2 < LI < −0.2) or change from dominant to bilateral with a minimum LI difference of 0.2. The maximum change was found with patient ZN (thresholded whole brain analysis, Fig. 4, Table IX), from whom LI changed from left hemispheric dominance (LI = 0.2, individual normalized ROI) to right hemispheric dominance (LI = −0.32, standard ROI).

Table IX.

Variation of individual lateralization results depending on the ROI

Individual lateralization index LI based on thresholded whole brain analysis
Subject	Side of brain pathology	Individual nonnormalized ROI	Individual normalized ROI	Mean group ROI	Standard ROI
BA	Left	0.53	0.25	0.28	0.13
BAU	Left	–0.21	–0.22	–0.32	–0.15
CO	Left	0.10	0.23	0.22	0.31
DE	Left	0.90	0.42	0.44	0.10
DI	Left	0.06	0.03	0.11	0.10
DO	Left	0.21	0.43	0.43	0.33
HA	Left	–0.17	–0.12	–0.09	–0.15
HE	Left	0.27	0.32	0.31	0.07
HO	Left	0.06	0.04	0.09	–0.05
KA	Left	0.06	0.04	0.06	–0.16
KAU	Left	*	*	*	*
KE	Left	*	*	*	*
KER	Left	0.06	0.10	0.18	0.08
KO	Left	0.28	0.20	0.20	0.14
KOR	Left	0	–0.17	–0.25	0
LU	Left	–0.11	–0.15	–0.09	–0.10
ME	Left	–0.71	–0.62	–0.40	–0.76
SC	Left	–0.45	–0.38	–0.40	–0.51
SE	Left	–0.09	–0.15	–0.11	–0.18
TU	Left	0.03	0.08	–0.04	–0.09
AM	Right	0.16	0.07	0.03	0.02
AW	Right	0.05	0.12	0.10	0.12
DOR	Right	0.03	0.04	0.15	0.12
FR	Right	0.06	0.06	0.08	–0.02
GR	Right	0.15	0.08	0.14	0.05
HAI	Right	0.23	0.22	0.21	0.15
PE	Right	–0.33	–0.22	–0.47	–0.47
PO	Right	0.41	0.40	0.32	0.35
SA	Right	–0.26	–0.22	–0.19	–0.37
ST	Right	0.04	0.15	0.13	0.21
UL	Right	0.09	0.06	0.04	–0.11
WA	Right	0.45	0.38	0.35	0.17
ZN	Right	0.14	0.20	0.17	–0.32

Data are shown for the thresholded whole brain analysis. With patients KAU and KE not every ROI comprised significant voxels with this activation quantification metric.

Figure 4.

Example of the influence of ROI definitions on lateralization results (Patient ZN). Same ROI conventions as in Figure 2. Brain activation as detected by the ROIs is shown (thresholded whole brain analysis). A significant left dominance changes to a significant right dominance with the standard ROI (blue).

Lateralization results with the anatomical template differed from results with the EPI template normalization. With anatomical template normalization the standard ROI shows a right hemispheric dominance also for the nonthresholded ROI analysis (compare Table VII, mean lateralization indices).

Lateralization Differences Depending on the Metric for Activation Quantification

Concerning the influence of the metric for activation quantification on clinical lateralization reports, Table VIII shows only the standard ROI to be vulnerable for the chosen quantification method. All other ROIs generated stable group lateralizations (with both normalization procedures). On the individual level, significant lateralization changes were found with every ROI and concerned up to 7/33 patients. The largest effect was found with patient PE (individual nonnormalized ROI) changing from a significant right dominance (LI = −0.33, thresholded whole brain analysis) to a significant left dominance (LI = 0.32, nonthresholded ROI analysis).

DISCUSSION

Our study investigated the influence of ROI definition and normalization methods on clinical fMRI results concerning presurgical memory localization. Despite the evidence described in the introduction, that standard methods might pose problems, detailed patient investigations are absent from the literature. Consequently, automated ROI definitions on normalized data sets are regularly applied in recent clinical fMRI literature [Bettus et al., 2008; Lehn et al., 2009; Rametti et al., 2009; Sanchez‐Carrion et al., 2008; Schoning et al., 2009; Tie et al., 2008]. For comprehensive judgment of possible ROI effects, we investigated a rather large spectrum of ROI definitions. On one end of the spectrum, we defined ROIs on original EPI data (individual nonnormalized ROI), on the other end we defined ROIs according to unrelated anatomical models (standard ROI). This strategy allowed us to separate data normalization and ROI definition effects. On the basis of previously published results of the present memory paradigm, our ROIs comprised all functionally relevant neuroanatomical structures. Since correct definition of true positive and true negative activations is a major problem with clinical fMRI, we also evaluated thresholded and nonthresholded metrics for activation quantification. In this context, it is important to note that fMRI data analysis with application of ROIs introduces an additional threshold, namely an anatomical threshold. This is justified, if the integration of neurophysiological and neuroanatomical knowledge is correct and results in functionally homogeneous ROIs.

A major result of our investigation indicates that the application of standard ROIs may indeed influence fMRI results in a clinically significant way. This was the case both for individual patients and group results. It is best illustrated by two findings. The first finding is that the neuroanatomical coverage of the individual MTLs by the standard ROI was found to be incomplete (Table III, Figs. 2 and 5). There seem to be two reasons for this: (1) with both normalization procedures a systematic error occurred at basal MTL borders indicating imperfect normalization results. The mismatch between patient brains and template was larger with anatomical template normalization. Note, that for the other ROIs investigated in our study, this normalization problem does not exist (since the ROIs are linked to the individual patient data and not to the external template). (2) There is a mismatch concerning basal extension of the EPI template, the MNI 152 anatomical template and the standard ROI (Fig. 5). As a consequence, the standard ROI omits relevant brain activity (Table IV, V, VI and Fig. 3) and reduces total calculated MTL activity (Table II). This may be problematic with pathological brains which generate only weak functional signals.

The second finding illustrating the clinically relevant influence of standard ROIs concerns hemispheric dominance. Dominant activation occurring within the same hemisphere as the pathological region to be operated on indicates an increased risk for specific postoperative functional deficits. Conversely, dominant activation in the nonpathological hemisphere may indicate a minor functional risk. Our results show that with the standard ROI group lateralization changed significantly against all other ROIs. Instead of a left hemispheric dominance, the standard ROI generated a right hemispheric dominance (Table VII and Fig. 4). Inferior performance of the standard ROI was true with respect to all other ROIs. Note, that the nonstandard ROIs share the characteristic, that they are based on individual delineations performed on the original functional EPI data (Fig. 1). Of course, the expression “inferior performance of the standard ROI” relates to its power to detect known fMRI brain activations. We did not try to correlate ROI results with independent clinical measures of brain activity (e.g. wada test, postoperative surgical outcome). However, it seems unlikely, that a technique which misses relevant brain activations produces adequate fMRI results for presurgical memory localization. This issue needs to be addressed in a comprehensive study that correlates a battery of clinical parameters (isolated wada results seem insufficient due to the inherent problems with the interpretation of test results) with fMRI results.

Concerning the influence of the metric for activation quantification, such effects may be inferred from Table VIII. Group results show that only with the standard ROI the LI changed with the metric for activation quantification. Results for all other ROIs were consistent. In individual patients, however, some of the other ROIs did also show effects of the metric for activation quantification. This underlines the presence of considerable intersubject variation. It argues for comprehensive analysis of individual patient data, with application of different methodologies [Foki and Beisteiner, 2010].

Altogether, our results did not show large differences between the three nonstandard ROIs (individual nonnormalized ROI, individual normalized ROI, mean group ROI). This indicates that the decisive factor is the selection of nontransformed individual functional images as a basis for neuroanatomical ROI delineation. Nontransformed individual functional images have been subject to the lowest degree of manipulation with respect to secondary model assumptions and data processing steps. It is evident that ROI delineation on nontransformed functional images requires EPI images with adequate resolution and contrast. As demonstrated by our success rate for covering the functional ROI with individual ROI delineation, this approach seems feasible (Table IV). It is clear that the validity of the individually defined ROIs may be rater dependent and intra‐ and interrater variability studies concerning individual MTL delineations have yet to be done. Note however, that the major problem of the standard ROI was a clearly visible lack to cover basal MTL. For experienced fMRI experts, it is not difficult to detect basal MTL borders and include them in the individual ROI. Therefore, it is likely that our major result is not much rater dependend. One might even speculate, whether mean group ROIs based on individual ROI delineations might allow the generation of disease specific standard templates (e.g. MTL pathology template, brain atrophy template). Concerning all our results on ROI effects, it is important to note that no conclusions are possible about the numerous AAL ROIs not investigated here. However, since it is possible, that similar coverage problems exist with other AAL ROIs, a simple procedure with AAL ROI studies might be to check normalization results and the neuroanatomical ROI coverage with every single patient and exclude patients with relevant coverage errors. Of course, it would also be interesting to find out whether and how the standard ROI procedures could be improved. Unfortunately, this is a rather complex enterprise. The dominant problems—and possible points of action—with the AAL standard ROI technique may concern every single of numerous data processing steps. For example, a listing of modifiable steps related to normalization of individual brains to a standard template includes: skull stripping, data resampling, manual optimization of initial brain position, signal intensity normalization, selection of adjustable software settings, masking of pathologies—all these steps may influence the final AAL results. Importantly, the influence may vary with the signal‐to‐noise ratio and artifact level of the individual patient data. Selection of an alternative normalization package may again produce different final AAL results [Klein et al., 2009]. A similar list of possible points of action could be generated for other parts of the AAL procedure (e.g. a list related to manipulation of the predefined AAL ROIs (e.g. smoothing, extending)). We therefore restricted our investigation to two types of standard ROI definitions, one based on EPI template normalization and one based on anatomical template normalization. Both normalization procedures were implemented as “standard” as possible and with both we found some differences between normalized than nonnormalized data. In general, differences between normalized than nonnormalized data increased with anatomical template normalization, which includes more postprocessing steps than EPI template normalization. To our opinion, this argues for minimizing the number of postprocessing steps with clinical data to reduce the probability for adding processing errors related to application of insufficient models [Gartus et al., 2007].

The larger activity indices with normalized data (Table II) seem to be related to the matrix size change performed during our normalization process: because of smaller volumes of our normalized voxels (8 mm³ instead of 9.72 mm³), the number of ROI voxels is increased to 122%. Correspondingly, the mean number of active voxels is also increased (820 instead of 690) despite comparable activation volumes (6707 mm³ vs. 6560 mm³, all values for the individual normalized ROI). This matrix effect on absolute activation levels should be regarded when clinical results are evaluated (and matrices differ between investigations). A further analysis of normalization effects is possible via LI evaluations. We did not find significant LI differences between the individual nonnormalized ROI and the individual normalized ROI. Both results indicate that normalization of an individually defined ROI volume has limited effects on final results. Note however, that this conclusion must be strictly separated from our finding, that normalization to a standard brain may well influence final results with predefined standard ROIs.