Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Aug 1.
Published in final edited form as: Neuropsychologia. 2010 Jun 4;48(10):2912–2921. doi: 10.1016/j.neuropsychologia.2010.05.033

A Virtual Reality-Based FMRI Study of Reward-Based Spatial Learning

Rachel Marsh 1, Xuejun Hao 1, Dongrong Xu 1, Zhishun Wang 1, Yunsuo Duan 1, Jun Liu 1, Alayar Kangarlu 1, Diana Martinez 3, Felix Garcia 1, Gregory Tau 1, Shan Yu 1, Mark G Packard 2, Bradley S Peterson 1
PMCID: PMC2914178  NIHMSID: NIHMS211247  PMID: 20570684

Abstract

Although temporoparietal cortices mediate spatial navigation in animals and humans, the neural correlates of reward-based spatial learning are less well known. Twenty-five healthy adults performed a virtual-reality fMRI task that required learning to use extra-maze cues to navigate an 8-arm radial maze and find hidden rewards. Searching the maze in the spatial learning condition compared to the control conditions was associated with activation of temporoparietal regions, albeit not including the hippocampus. The receipt of rewards was associated with activation of the hippocampus in a control condition when using the extra-maze cues for navigation was rendered impossible by randomizing the spatial location of cues. Our novel experimental design allowed us to assess the differential contributions of the hippocampus and other temporoparietal areas to searching and reward processing during reward-based spatial learning. This translational research will permit parallel studies in animals and humans to establish the functional similarity of learning systems across species; cellular and molecular studies in animals may then inform the effects of manipulations on these systems in humans, and fMRI studies in humans may inform the interpretation and relevance of findings in animals.

Keywords: Memory Systems, fMRI, spatial learning, reward

INTRODUCTION

Previous fMRI studies of spatial navigation within virtual reality (VR) environments have consistently identified a network of regions -- including the hippocampus, parahippocampus, retrosplenial cortex, and posterior parietal cortex -- that are engaged during virtual navigation relative to non-navigational control tasks (Aguirre, Detre, Alsop, & D’Esposito, 1996; Hartley, Maguire, Spiers, & Burgess, 2003; Hassabis et al., 2009; Iaria, Fox, Chen, Petrides, & Barton, 2008; Spiers & Maguire, 2006b; Wolbers & Buchel, 2005). However, many of these control tasks differed from the navigation tasks on salient features, such as the form of visual stimuli, motor demands, task instructions, or cognitive effort required for navigation, thereby precluding full isolation of the neural correlates of spatial learning in standard functional imaging subtraction paradigms. In addition, none of the prior fMRI studies have directly assessed the role of reward in spatial learning.

Our VR task for spatial learning is directly analogous to the standard “win-shift” radial arm maze paradigm that has been used to study spatial learning and memory in animals. This paradigm involves navigation of a maze with 8 identically appearing arms extending outward from a central platform that has extra-maze objects visible from within the maze. Rodents obtain food rewards by visiting each arm of the maze once, with re-entries into maze arms previously visited scored as errors. Because performance on this task requires rodents to remember those arms that have been previously visited, it is regarded as a prototypical test of spatial memory or cognitive mapping. Extensive evidence from animal studies employing lesions (Olton & Samuelson, 1976; Packard, Hirsh, & White, 1989) and pharmacological stimulation (Packard & White, 1991) have demonstrated that the hippocampus contributes to performance on the win-shift radial maze task.

Analogous fMRI studies in humans can complement these radial arm maze studies of spatial learning in animals. For example, fMRI studies can more readily and simultaneously assess the differential involvement of brain regions other than the hippocampus in spatial learning. In addition, fMRI studies can disentangle the roles of reward and learning in reward-based spatial learning, and they more readily can study of de novo learning, without requiring prior training as in animal studies.

We report an fMRI study of reward-based spatial learning in healthy adults performing a novel VR task. Participants used an MRI-compatible joystick to navigate through an 8-arm radial maze that was surrounded by a naturalistic landscape consisting of mountains, trees, and flowers that had to be used for spatial navigation if participants were to avoid reentries into previously visited arms. Hidden at the end of each arm of the maze was a reward that participants could earn and keep only if they did not re-enter an arm already visited. To isolate the neural correlates of spatial learning, we included a rigorously defined control condition that shared all salient features with the active learning task except for the use of the extra-maze cues for navigation. Our a priori hypothesis was that participants would engage medial temporal and parietal areas during spatial learning in the active compared with the control condition. Based on prior findings of reward-based learning in animals and humans (Adcock, Thangavel, Whitfield-Gabrieli, Knutson, & Gabrieli, 2006; Ito, Robbins, Pennartz, & Everitt, 2008; Vanni-Mercier, Mauguiere, Isnard, & Dreher, 2009), we also hypothesized that activation of mesolimbic areas would be associated with the receipt of rewards in the active compared to the control condition. In addition, we explored whether mesolimbic activation was associated with the anticipation of rewards, and whether it depended on the actual receipt of rewards or on the involvement of learning processes. Finally, to ensure that any confusion or additional cognitive effort in the control condition was not producing activation in the active condition, we compared brain activity during spatial learning and the receipt of rewards in the active condition with brain activity in a ‘trail following’ condition that eliminated the attempted use of the extra-maze cues for navigation.

METHODS

Participants

Participants were recruited through flyers posted in the local community to serve as healthy control participants for a neuroimaging study of the neural basis of substance dependence. The Structured Clinical Interview for DSM-IV-TR Axis I Disorders (First, Spitzer, Gibbon, & Williams, 2002) was administered to all participants. Any who met DSM-IV criteria for a current Axis I disorder, or who had a lifetime history of substance abuse disorder, neurological illness, past seizures, head trauma with loss of consciousness, mental retardation, or pervasive developmental disorder were excluded from participating. The Institutional Review Board of NYSPI approved this study, and all participants gave informed consent prior to participating.

Spatial Learning Paradigm

Creation of the virtual environments was accomplished through use of software that was built upon C ++ and OpenGL, an API (Application Programming Interface) of graphics. The virtual environments were composed of an eight-arm radial maze with a central starting location and a low outer perimeter wall. The maze was surrounded by a naturalistic landscape (e.g., mountains, trees and flowers) that constituted the extra-maze cues that could be used for spatial navigation (Fig. 1A). Prior to scanning, participants underwent a 10-minute training session on a desktop computer to practice using a joystick to move and navigate freely around a virtual maze that was similar in appearance to the experimental environment used for scanning, but with different extra-maze cues.

Figure 1. Virtual Reality Environment.

Figure 1

Open in a new tab

Shown here is (A) a view of the VR environment, (B) the U.S. dollar signs denoting rewards that were hidden at the ends of the maze arms, and (C) a view of the ‘trail-following’ control condition C.

Stimuli during scanning were presented through non-magnetic goggles (Resonance Technology, Inc.). Participants used an MRI-compatible joystick (Current Designs Inc.) to navigate the maze. Before entering the scanner, participants were informed that they would find themselves in the center of a virtual maze with eight identical runways extending outwards, and that hidden rewards (U.S. $, Fig. 1B) would be available at the very end of the runways. They were instructed to try to find the rewards and were told that they could earn and keep as many as they found, but that they would loose money if they revisited arms they had already visited. They were told that they would complete several sessions of the task. There were not told that the conditions of the sessions would differ from one another, and therefore they believed that they would be performing the same task multiple times. Participants were not given actual monetary rewards for their performance on the task, but rather paid a set amount for their participation in the study.

The first session (condition A) required spatial learning. To find and retain all 8 rewards, participants had to learn to use the extra-maze cues to avoid revisiting previously visited arms. Each trial began at the center platform. After reaching the end of an arm, participants reappeared automatically in the middle of the center platform to initiate a new trial, with the initial viewing perspective randomly oriented across the entire 360 degrees of the circular maze, which compelled participants to use extra-maze cues to orient themselves for subsequent navigation. The randomized orientation at the start of each trial also prevented use of S-R strategies, such as “chaining” rules (e.g. exploring arms positioned successively to the left or right of the last arm entered, regardless of extra-maze cues), when performing the task.

After obtaining all 8 rewards in condition A, participants saw a black screen with white written text indicating that a new session was about to begin. In this first control condition (B), the same extra-maze cues as used in condition A were randomized among locations after each trial: the mountains that surrounded the maze were cut into several pieces that were then shuffled and re-stitched into whole mountains, while the individual trees and flowers were themselves shuffled among locations. This randomization of the cues together with the randomized directional orientation of participants with respect to the various arms of the maze at the start of each trial destroyed any possibility of using the spatial layout of the cues (spatial learning) to perform the task. As in condition A, the randomized orientation at the start of each trial also prevented the use of S-R or other procedural learning strategies. To control for the frequency of reward and punishment with the spatial learning task, however, participants were rewarded at the same frequency as in the previous spatial learning condition, though now without regard to any characteristics of their actual performance. Participants were not told that the cues would be randomized in this session or that anything else would differ from the previous session. This control condition thus shared all salient features with the spatial learning task (A), including all the lower-order stimulus features, such as color, texture, hue, luminance, the form and movement of visual stimuli, as well as many higher-order task features, such as the motor demands, task instructions, cognitive effort in searching for rewards, and reward and punishment experiences. Condition B was pre-programmed to terminate according to the number of trials that a given participant needed to obtain all 8 rewards in condition A. For instance, if a participant required 10 trials to find all 8 rewards in condition A (i.e. 8 correct and 2 error trials), they would be given 2 unbaited trails randomly in condition B. After completion of condition B, another black screen appeared that instructed participants to “follow the arrows” during the next session (condition C).

Condition C was a second control condition to contrast with condition A. Condition C was added to ensure that the use of the cues or the additional confusion or cognitive effort in condition B was not producing any observed activation in condition A (when compared to B). Condition C was a “trail following” condition in which the extra-maze cues were randomized among locations in the same manner as in control condition B, but a red arrow at eye level indicated the path to follow (the arm to be traversed, Fig. 1C). Because participants were told simply to follow the arrow on each trial, they were presumably not trying to use the extra-maze cues for navigation, and in fact they were unable to use those cues because of the randomization of the extra-maze cues upon the initiation of each trial. Because the allocation of rewards was randomized, however, the arrow did not always point to the correct arm (where a reward was hidden). This control condition shared many salient features with both the spatial learning task (A) and control condition B, including the form and movement of visual stimuli, motor demands, and reward and punishment experiences. It did not require as much cognitive effort in searching for rewards as did condition B, the task instruction (“follow the arrow”) differed from conditions A and B, and the task generated less confusion than did condition B when trying to use and remember the extra-maze cues for navigation when the locations of those cues were randomly interspersed.

Each participant underwent two scanning runs, each run containing one presentation of condition A, B, and C presented in that order, for a total of 2 runs of each condition. Because the use of A to establish the rate of reward and punishment in conditions B and C required presenting A first for all participants, the order of conditions was not counterbalanced across participants. In addition, testing our a priori hypotheses required comparing conditions A and B. Condition C was added only to ensure that any confusion or additional cognitive effort in condition B was not producing activation when contrasted with condition A. The spatial learning paradigm contained 48 rewarded navigation events (8 rewards per each of 3 conditions x 2 runs). The number of unrewarded events varied for each participant. The 3D coordinates of each participant’s movements through the virtual environment were recorded with a temporal resolution of 50 msec. Performance on the task was scored automatically using the same criteria as in animal studies (e.g. the arms entered and order of entry, overall latency to visit baited arms, total number of errors, number of choices before the first error were all recorded).

Image Acquisition & Preprocessing

Acquisition

Head positioning in the magnet was standardized using the canthometal line. Images were acquired on a GE Signa 3 Tesla LX scanner (Milwaukee, WI) and a standard quadrature GE head coil. A T1-weighted sagittal localizing scan was used to position the axial functional images parallel to anterior commissure-posterior commissure (AC-PC) line. In all participants, a 3D spoiled gradient recall (SPGR) image was acquired for coregistration with the axial functional images and with the standard Talairach coordinate system. The functional images were obtained using a T2*-sensitive gradient-recalled, single-shot, echo-planar pulse sequence having a TR=2,800msec, TE=25msec, 90° flip angle, single excitation per image, 24×24cm FOV, a 64×64 matrix, 43 slices 3mm thick, no gap, covering the entire brain. We collected a maximum of 322 EPI volumes for each of two run, with the actual number of volumes determined by the performance of each participant during the active conditions of each run.

Preprocessing

Our preprocessing procedures were run in batch mode and implemented using subroutines in SPM2. Functional images were first corrected for timing differences between slices using a windowed Fourier interpolation to minimize their dependence on the reference slice (Jazzard, Matthews, & Smith, 2002). Images were then motion-corrected and realigned to the middle image of the middle scanning run. Images were discarded if the estimates for peak motion exceeded 2-mm displacement or 28 degrees of rotation (Friston, Williams, Howard, Frackowiak, & Turner, 1996). The corrected images were normalized and coregistered to a standard MNI template by warping each participant’s SPGR image to the MNI template avg152T1 (resolution=2×2×2 mm per voxel) and then warping each functional image to the SPGR image for that person. Images were then spatially smoothed using a Gaussian-kernel filter with a full width at half maximum of 8 mm.

Behavioral Analysis

We tested whether participants would improve in spatial learning over scan runs using a mixed models analysis (PROC MIXED) with repeated measures over runs performed in SAS version 8.0 (SAS Institute Inc, Cary, NC). Participants who demonstrate spatial learning on the VR paradigm should be faster at obtaining all 8 rewards (i.e., they should have shorter average latency scores) during the second compared with the first active, spatial learning condition (e.g., condition A in run 2 versus condition A in run 1). With latency defined as the time to complete each learning condition (A) for each individual, latency was entered as the dependent variable in a linear mixed model. Run (1 or 2) was entered as the within-subjects factor. Covariates included age, sex, and minority status. The hypothesized improvement in spatial learning was tested by assessing statistical significance of the main effect of run, representing the change in latency from run 1 to run 2. Participants who engaged in spatial learning should have required fewer trials to obtain all 8 rewards over the 2 runs. The rate of learning (8/the number of trials) was calculated for each person and entered as the dependent variable in another linear mixed model with run as the within-subjects factor and age, sex, and race as covariates. The improvement in the rate of spatial learning was assessed by testing the significance of the main effect of run in this model, representing the change in rate from run 1 to run 2. We also considered for inclusion in both of these models all interactions of latency or rate of learning with age and run. Terms that were not statistically significant were eliminated via backward stepwise regression, with the constraint that the models had to be hierarchically well-formulated at each step (i.e., all possible lower-order terms had to be included in the model, regardless of their statistical significance) (Morrell, Pearson, & Brant, 1997).

Image Analysis

First-level parametric analyses were conducted individually for each participant using GLM in a modified version of SPM2 run in batch mode. The GLM included the following 4 regressors, corresponding to specific events that occurred in each condition and in both scan runs: (1) ‘searching,’ began at the start of a trial and lasted until 10% of the length of an arm had been traversed, (2) ‘anticipation of reward,’ began after the first 10% of an arm was traversed and extended until reaching its baited area, (3) ‘receipt of reward,’ was defined as comprising the images at the end of the trial when a reward was present, and (4) ‘no receipt of reward’ was the time when the end of an arm was reached but no reward was received (Fig. 2). All regressors were convolved with a canonical hemodynamic response function (HRF), with the durations of the regressors for each participant modeled according to the durations of these events during performance in condition A. Low frequency noise from scanner drift was removed using a high-pass filter based on discrete cosine transform (DCT).

Figure 2. Study Design.

Figure 2

Open in a new tab

Shown here is a schematic diagram of the event-related design, including the 4 events that were modeled as regressors. These events were defined based on the participants’ behavioral performance, consistent with the performance of rats in the original radial arm maze studies.

Twelve contrast images were created for each person by applying a t contrast vector to the parameters (beta_j) that were estimated for voxel j. Thus, beta_j = b1j, b2j, b3j, b4j, b5j,…,bnj, where n = 12 because 4 regressors corresponded to 4 events (searching, anticipation, receipt of reward, and no receipt of reward) for each of 3 conditions (A, B, and C). Table 1 lists the contrast images that were used to identify neural activity associated with spatial learning and with attempted spatial learning during each of the 4 events. Below we describe the specific information provided by each of these contrasts and how each contrast was used to test our a priori and exploratory hypotheses. Because spatial learning was rendered impossible by the randomization of extra-maze cues in condition B, our comparison of neural activity associated with events in condition B with neural activity associated with the same events during spatial learning (in condition A) allowed us to disentangle components of the learning process.

Table I.

Contrast images used to assess neural activity associated with and without spatial learning during each event.

Event Spatial Learning Attempted Learning
Searching A vs. B1 B vs. C
Anticipation A vs. B B vs. C
Receipt of Reward A vs. B2 B vs. C
No Receipt of Reward A vs. B2 B vs. C
No Reward vs. Reward A3 B4
Open in a new tab

The following contrasts were used to identify neural activity associated with

1

spatial learning;

2

reward experience during spatial learning;

3

the effects of reward when spatial learning was possible; and

4

the effects of reward when learning was attempted but rendered impossible (e.g. reward disentangled from learning). In exploratory analyses, the A vs. C contrast was used to identify neural activity associated with the same processes as the A vs. B contrast.

Group composite activation maps were generated by calculating t statistics at each pixel of the image using a random effects analysis, covarying for age and sex. We report voxels that were identified on these maps using a p-value threshold <0.025 together with the requirement that the activation occurred in a spatial cluster of at least 20 adjacent pixels, a conjoint requirement which, based on an approximation formula (Friston, Worsley, Frackowiak, Mazziotta, & Evans, 1994), yields a conservative effective p-value < 0.000005. The combined application of a statistical threshold and cluster filter reduces substantially the false-positive identification of activated pixels at any given threshold (Forman et al., 1995).

A Priori Hypothesis Testing

We tested whether participants engaged medial temporal areas during reward-based spatial learning on the task. Analyses of covariance (ANCOVAs) with centered age and sex as covariates were performed on the following contrasts to identify neural activity during searching and reward experiences in the active condition A compared to control condition B:

  • Neural Activity During Searching in Spatial Learning (A vs. B): identified neural activity associated with spatial learning (i.e., using the extra-maze cues to navigate the maze), regardless of whether a trial was rewarded or not.

  • Neural Activity During the Receipt or Non-Receipt of Reward When Learning (A vs. B): isolated the neural correlates of reward experiences during spatial learning by comparing neural activity associated with the receipt or non-receipt of reward at the end of trials in condition A with neural activity associated with the same reward experience at the end of trials in condition B.

Exploratory Analyses

To ensure that any confusion or additional cognitive effort in condition B was not producing activation in condition A, neural activity during searching and the receipt of reward in condition A was also compared to neural activity during the same events in control condition C (A vs. C). We also tested (1) whether participants engaged medial temporal areas when attempting to learn, (2) whether they engaged mesolimbic areas during the anticipation of reward, and (3) whether mesolimbic engagement depended on reward and condition. Thus, ANCOVAs were performed on the following contrasts:

  • Searching When Attempts at Spatial Learning Were Rendered Impossible (B vs. C): identified neural activity associated with attempting to use the extra-maze cues for spatial learning when learning was experimentally rendered impossible by the randomization of cues in condition B.

  • Reward Anticipation (A vs. B): identified neural activity associated with reward anticipation during spatial learning. This contrast was appropriate because learning, and therefore reward anticipation, was only possible in condition A. Rewards were not based on performance in condition B since cue randomization made learning impossible and reward frequency was based only on performance in condition A.

  • Reward-Specific Activation (No Reward vs. Reward) in Condition A: identified the effects of reward on neural activity when spatial learning was possible (i.e., comparing activity during no receipt of reward at the end of trials with the receipt of reward and the end of trials in condition A).

  • Reward-Specific Activation (No Reward vs. Reward) in Condition B: identified the effects of reward on neural activity when spatial learning was attempted but rendered impossible (i.e., comparing activity during no reward at the end of trials with reward and the end of trials in condition B).

Finally, to ensure that our findings reported for the primary contrast (A vs. B) were not driven by the fixed order of conditions across participants, paired t-tests were used to compare neural activity associated with searching and with reward experiences at the end of trials during the first and second presentation of these conditions.

RESULTS

Participants

One person moved excessively during scanning and was therefore excluded from further analysis. The participants who remained for statistical analyses included 25 healthy adults, 21 men and 4 women, 21–44 (mean = 32.5±7.6) years of age; 10 were Caucasian, 10 African American, 4 Hispanic, and 1 was Asian American. They were from households of primarily medium socioeconomic status (SES = 33.7±14, possible range 0–64) (Hollingshead, 1975). All but one of the participants were right-handed (Oldfield, 1971).

Behavioral Performance

Age, sex, and minority status did not account for significant a proportion of variance in spatial learning in either model (Ps > .1) and were therefore eliminated from the models. A significant main effect of Run in the first model (t(23)=8.16, P<.01) indicated that participants required less time to complete run 2 (8011±3797 msec) compared to run 1 (11059±6911). A significant main effect of Run in the second model (t(23)=9.62, P<.01) indicated that participants required fewer trials (attempts) to obtain all 8 rewards in run 2 (mean rate = 0.64±0.3) compared to run 1 (0.48±0.2). Thus, spatial learning, defined by a decline in latency and an increased rate of reward, improved over the two scanning runs. Across both scan runs, however, an average of 45% of the trials were not rewarded (range: 0 – 80%), prompting our assessment of neural activity associated with both the receipt and non-receipt of reward (below).

A Priori Hypothesis Testing

Neural Activity During Searching in Spatial Learning (A vs. B)

Searching in condition A relative to condition B, regardless of whether the trial was rewarded or not, activated temporal and parietal cortices (Fig. 3a, Table 2), including bilateral superior temporal gyrus (STG; P < .001, BA 22), bilateral parahippocampal gyrus (P < .001, BA 36), and right lateral inferoparietal cortex (LIP, P = .01, BA 40). Searching also activated the posterior cingulate (PCC, BA31), precuneus (P < .001, BA 7/31, Fig 3a), and primary motor cortices (PMC, P < .001, BA 4, Figs. 3a).

Figure 3. Average Brain Activity During Searching.

Figure 3

Open in a new tab

These are t-maps for FMRI signal during searching in condition A compared to control conditions B (A) and C (B). Increases in signal during condition A are shown in red and increases during the control conditions are shown in blue. Searching in condition A relative to condition B, regardless of whether the trial was rewarded or not, activated medial temporal and parietal areas involved in spatial learning. (C) A t-map for FMRI signal during searching in condition B compared to C. Increases in signal during B are in red and increases during C are in Blue.

STG, superior temporal gyrus; LIP, lateral inferoparietal area; PCC, posterior cingulate cortex; PHi, parahippocampus; Cd, caudate nucleus; RSC, retrosplenial cortex.

Table 2.

Activations associated with spatial learning during searching.

Activated Regions Location
Side BA # of voxels Peak Location Significance Testing Contrast
x y z T statistic p
Searching
Superior temporal gyrus L 22 156 −44 −21 1 4.21 < 0.001 A > B
R 699 65 −26 16 5.15 < 0.001 A > B
L 274 −55 −27 11 5.46 < 0.001 A > C
R 340 63 −25 12 4.60 < 0.001 A > C
Lateral parietal area R 40 84 50 −36 48 4.45 0.01 A > B
R 15 40 −50 47 3.37 0.01 A > C
Parahippocampal gyrus L 36 146 −34 −30 −15 3.45 < 0.001 A > B
R 157 38 −26 −19 3.93 < 0.001 A > B
L 191 −28 −22 −21 3.75 < 0.001 A > C
R 216 36 −24 −24 3.47 < 0.001 A > C
Precuneus/Posterior cingulate cortex L/R 7/31 613 8 −69 20 4.95 < 0.001 A > B
L/R 1002 6 −69 26 7.49 < 0.001 A > C
Posterior cingulate/Retrosplenial cortex L 30/31 707 −18 −64 5 7.17 < 0.001 A > C
Primary motor cortex R 4 170 46 −12 37 3.47 < 0.001 A > C
Mid-cingulate cortex L/R 24 105 6 −16 36 2.33 < 0.01 A > C
Caudate nucleus (head) L −14 21 −1 2.49 0.01 A < C
 (body) L −16 14 10 2.72 0.006 A < C
Open in a new tab

Neural Activity During the Receipt and Non-Receipt of Reward When Learning (A vs. B)

The receipt of reward in condition A vs. B deactivated the left hippocampus, which was equivalent to activating this region more during the receipt of reward in condition B, when learning was impossible, than in condition A, when learning occurred (P = .007, Fig. 4a, Table 3). However, a small portion of the right hippocampus was activated in association with the failure to receive reward at the end of trials in condition B vs. A (P = .008, Fig. 4d, Table 3). The relatively greater activation of the hippocampus in condition B relative to condition A suggests that at the end of trials, the confusion, effort, or difficulty of condition B relative to condition A could have activated the hippocampus, independent of the receipt of reward. Alternatively, the effects of reward and non-reward on hippocampus activation at the end of trials in condition B could be lateralized, with unanticipated reward activating the left hippocampus and the failure to receive a reward (unrewarded trials) activating the right hippocampus.

Figure 4. Average Brain Activity During the Receipt of Reward and No Reward.

Figure 4

Open in a new tab

These are t-maps for fMRI signal during the receipt of reward in condition A compared to control conditions B (A) and C (B), and for fMRI signal during no reward in condition A compared to conditions B (D) and C (E). Increases in signal during the control conditions are shown in blue. The hippocampus was activated during the receipt of reward (A) and no receipt of reward (D) in condition B and therefore associated with the reward experience itself during spatial learning. Also shown are t-maps for fMRI signal during the receipt of reward (C) and no reward (F) in condition B compared to C. Increases in signal during B are in red and increases during C are in Blue.

Hi, hippocampus; Cd, caudate nucleus; Put, putamen.

Table 3.

Activations associated with spatial learning during the receipt of reward, and no reward.

Activated Regions Location
Side # of voxels in cluster Peak Location Significance Testing Contrast
x y z T statistic p
Receipt of Reward
Hippocampus L 34 −32 −16 −13 2.62 0.007 A < B
Caudate nucleus L 25 −8 14 9 2.11 0.01 A < C
Lenticular nucleus (or putamen) L 64 −20 4 5 2.36 0.01 A < C
Failure
Hippocampus R 43 32 −14 −18 2.59 0.008 A < B
Putamen L 4900 −22 4 2 3.99 < 0.001 A < C
R 490 28 −5 9 3.02 0.001 A < C
Caudate nucleus L −4 16 −1 3.16 0.001 A < C
Thalamus L/R 0 0 7 3.43 0.001 A < C
Failure vs. Receipt of Reward in B
Hippocampus L 2022 −20 −12 −16 4.08 < 0.001 Reward ↑
Amygdala R 24 −1 −13 3.01 0.003 Reward ↑
Open in a new tab

Exploratory Analyses

Neural Activity During Searching in Spatial Learning (A vs. C)

Similar to findings for the contrast of condition A with B, searching in condition A relative to condition C also activated temporal and parietal cortices (Fig. 3b, Table 2), including bilateral superior temporal gyrus (STG; P < .001, BA 22), bilateral parahippocampal gyrus (P < .001, BA 36), and right lateral inferoparietal cortex (LIP, P = .01, BA 40). This contrast also revealed greater activation of the posterior cingulate and restrosplenial cortices (RSC) (P < .001, BA 30/31, Fig. 3b), and greater activation of the mid-cingulate cortex (MCC, P < .01, BA 31, Fig. 3b).

Neural Activity During Searching When Attempts at Spatial Learning Were Rendered Impossible (B vs. C)

Searching in condition B relative to C activated the PMC, MCC, and PCC bordering on the precuneus and RSC. In addition, searching deactivated the body and head of the left caudate nucleus (Ps < .01) and left nucleus accumbens (Ps < .01) or, equivalently, activated these regions in condition C relative to conditions A (Fig. 3b) and B (Fig. 3c).

Neural Activity During the Receipt of Reward When Spatial Learning Was Possible (A vs. C)

Activity in the left caudate nucleus and putamen was greater at the end of the trial in condition C than in condition A during both reward (Ps = .01, Fig. 4b) and no reward (Ps = .001, Fig 4e).

Reward-Specific Activation (No Reward vs. Reward)

Maps of neural activity during no reward compared to reward within conditions A and B revealed greater activity in the left hippocampus and right amygdala when not rewarded in condition B (Ps ≤ .003, Fig. 5b), but not in condition A (Fig. 5a). Thus, hippocampus activation was greater when not receiving reward than when receiving reward in condition B, when learning was rendered impossible. In addition, an interaction of condition (A or B) with reward (rewarded or unrewarded trials) was detected in the left hippocampus (P < .01, Fig. 5c), confirming statistically that activation of this region was indeed specific to condition B when participants were not rewarded and learning was impossible. Finally, a main effect of reward was detected in premotor cortices bilaterally, indicating greater activation of these regions during non-rewarded trials compared to rewarded ones in conditions A (Fig. 3a) and B (Fig. 3b) – i.e., regardless of whether learning was possible and whether rewards were anticipated.

Figure 5. Average Brain Activity During the No Receipt Compared to the Receipt of Reward.

Figure 5

Open in a new tab

These are t-maps for fMRI signal during no receipt compared to the receipt of reward in conditions A (A) and B (B). Increases in signal associated with no receipt of reward are shown in red and increases during the receipt of reward are shown in blue. The hippocampus and amygdala were activated during no receipt of reward at the end of trials compared to the receipt of reward at the end of trials in condition B, when learning was rendered impossible.

Hi, hippocampus; Amy, Amygdala; PMC, primary motor cortex.

Reward Anticipation (A vs. B)

The anticipation of reward (beginning after the first 10% of an arm was traversed and extending until participants reached its baited area) was not associated with activity in mesolimbic regions or in any other brain areas in the active learning (A) or control (B) conditions (maps not shown).

Order Effects

A comparison of contrast maps (A vs. B) of neural activity during searching and during the receipt and non-receipt of reward in run 1 with neural activity during these events in run 2 suggested that order and practice did not contribute to our findings of temporo-parietal activation during searching in spatial learning and of hippocampal activation during the receipt and non-receipt of reward in condition B (maps not shown).

DISCUSSION

We used a novel VR paradigm to investigate the neural correlates of spatial learning in healthy individuals. In the active learning condition (A), participants had to use extra-maze cues to navigate the maze and find hidden rewards. In control condition B, randomization of the elements of the extra-maze scene prevented participants from using the extra-maze cues to learn the location of rewards. As expected, participants demonstrated spatial learning during the active conditions by taking less time and making fewer attempts to find all 8 rewards on the second compared to the first of two runs.

Neural Activity During the Search Phase of Spatial Learning

Also as predicted, imaging results revealed that searching the maze during spatial learning in condition A relative to control condition B, when learning was impossible, activated large expanses of temporal and parietal cortices that participate in spatial navigation (Aguirre et al., 1996; Ekstrom et al., 2003; Hartley et al., 2003; Hassabis et al., 2009; Iaria et al., 2008; Spiers & Maguire, 2006b; Wolbers & Buchel, 2005), including bilateral STG (BA 22), bilateral parahippocampal gyrus (BA 36), and right LIP (BA 40), as well as the PCC and RSC. Exploratory analyses revealed the same patterns of temporo-parietal activation during searching in condition A relative to control condition C, when an arrow indicating the path to follow minimized effort and confusion associated with the attempted use of extra-maze cues to navigate and find rewards. The similarity in findings across these contrasts (A vs. B and A vs. C) indicates that participants were likely engaging the same strategies across the task conditions. Although we did not counterbalance the order of conditions across participants, these similar patterns of temporo-parietal activation during searching, along with our findings of non- significant order effects, suggest that order and practice did not contribute to these findings. Finally, we found that the PCC and RSC activated more in condition B relative to condition C, indicating that these regions activated when attempting to use spatial cues to navigate, even when success was rendered impossible in condition B by the randomization of extra-maze cues.

Our findings suggest that the parahippocampal cortex may play a greater role than the hippocampus in processing spatial information, in contrast to findings from previous fMRI studies that have implicated the hippocampus in spatial navigation (Gron, Wunderlich, Spitzer, Tomczak, & Riepe, 2000; Hartley et al., 2003; Iaria, Chen, Guariglia, Ptito, & Petrides, 2007). The tasks used in these prior studies differed considerably from ours, in that some included separate learning and retrieval conditions, and all included less rigorously specified control conditions. One study (Iaria et al., 2007), for example, required participants to learn the spatial relationships between landmarks in a VR environment before the experiment and then compared activity during retrieval of their learned, mental representation of the maze environment when navigating to those landmarks with activity during a control condition that was similar to our condition C (i.e., participants followed arrows along a path). The landmarks used in the control condition, however, differed from those used during learning and retrieval. This comparison produced activation of the anterior hippocampus during learning and of the posterior hippocampus and parahippocampus during retrieval. In contrast, our participants did not undergo separate learning and retrieval procedures, they were not instructed to learn the spatial relationships between the cues, and they experienced identical elements of the extra-maze environment during the active and control tasks. They were also likely attending to the larger extra-maze environment and to the spatial relationships among cues, rather than to the details of the particular cues and landmarks that were uncontrolled in the previous study. When searching the maze, our participants activated the parahippocampus rather than the hippocampus.

Our finding that the parahippocampus activates during the search phase of spatial learning is consistent with prior fMRI findings suggesting that the parahippocampal cortex responds preferentially to scenes or a spatial layout, but not to single objects (Epstein, Higgins, Jablonski, & Feiler, 2007), that it processes contextual associations (Aminoff, Gronau, & Bar, 2007), and is activated when participants discriminate between two virtual reality environments.(Hassabis et al., 2009) Consistent with our findings, single-unit recording studies have reported that neurons in the rodent homolog of the parahippocampus strongly encode spatial relationships among visual stimuli (Burwell & Hafeman, 2003). In addition, lesion studies have demonstrated a role for the parahippocampal cortex in binding objects to a particular environmental context rather than in memory for particular objects themselves (Norman & Eacott, 2005). Finally, our finding of parahippocampal activation during spatial learning is consistent with findings that lesions to parahippocampal areas, including the entorhinal cortex, impair spatial learning in rodents on the Morris water maze task (Kopniczky et al., 2006).

Our finding that spatial navigation in humans engages the parahippocampus rather than the hippocampus may at first seem to differ from findings of the radial-arm maze studies in rodents that implicate the hippocampus in spatial learning (Becker, Walker, & Olton, 1980; Packard et al., 1989). These animal studies showed that lesions of the hippocampus impair the ability of rodents to navigate a radial-arm maze accurately and find hidden rewards. In addition, the hippocampus is known to participate in path integration, the updating of one’s position in the environment based on self-motion cues (Etienne & Jeffery, 2004), self-motion cues determine the firing properties of hippocampal place cells (Wiener, Paul, & Eichenbaum, 1989), and hippocampal-dependent spatial learning by rodents within a radial-arm maze has been reported to require movement (White, 2004). Thus, movement is likely required to engage the hippocampus during navigation. The ‘searching’ phase in our task included very brief periods of passive scanning without movement (i.e., participants only spent 0.003% of the total search time motionless in the center of the maze before traversing an arm), possibly contributing to our failure to detect hippocampus activation during that portion of the task.

The hippocampus likely works in concert with anatomically connected regions, including parahippocampal areas and retrosplenial cortices, to create a map of the environment (Sharp, 1999). Single-cell recordings indicate that the retrosplenial cortex contains head-direction cells that fire when an animal’s head points in a specific direction in an open field (Cho & Sharp, 2001). Moreover, the regions that most commonly activate in human imaging studies of spatial navigation, including virtual reality-based fMRI studies, are the parahippocampal and retrosplenial cortices (Ekstrom et al., 2003; Hassabis et al., 2009; Maguire et al., 1998; Spiers & Maguire, 2006a; Wolbers, Weiller, & Buchel, 2004). Patients with damage to the RSC can recognize landmarks, but they have trouble finding their way around familiar environments (Ino et al., 2007), suggesting that the RSC plays an important role in processing the directional information between landmarks. These findings from lesion studies are consistent with our imaging findings that the RSC activated during the searching phase of the task, particularly when contrasted with activity during condition C, when participants were simply following an arrow and not using internally generated processing of directional information.

Participants also engaged the LIP area during the searching phase of the task. The LIP, a subdivision of the inferior parietal lobe, is involved in visual attention (Corbetta & Shulman, 2002). Unilateral lesions of the LIP produce hemispatial neglect, a condition that prevents responding to stimuli in the contralateral visual space (Driver & Vuilleumier, 2001) and interferes with the maintenance of attention directed toward spatial information (Malhotra, Coulthard, & Husain, 2009). In addition, electrophysiological studies in animals suggest that neurons in the LIP integrate spatial information with information concerning self-motion, including shifting attention to salient or task-relevant objects (Balan & Gottlieb, 2006), representing the location of cues and associated motor responses (Oristaglio, Schneider, Balan, & Gottlieb, 2006), and encoding navigational epochs in a maze (Nitz, 2006). The lateral parietal cortex is anatomically connected to the retrosplenial cortex (Kobayashi & Amaral, 2003) and reciprocally connected with the parahippocampus and hippocampus (Blatt, Pandya, & Rosene, 2003; Clower, West, Lynch, & Strick, 2001). Together, these considerations suggest that the LIP plays a crucial role in supporting the sensory and attentional requirements of spatial navigation.

Participants also activated the STG during searching in the active compared to the control conditions. The STG supports higher order visual processing (Kandel, Schwartz, & Jessell, 1991), including the perception of biological motion (Servos, Osu, Santi, & Kawato, 2002). In addition, fMRI studies have shown that the STG is involved in imagined locomotion (Jahn et al., 2004), spatial orienting, and exploration (Himmelbach, Erb, & Karnath, 2006). Finally, participants in our study activated mid-cingulate and premotor cortices during the searching phase of the task in conditions A and B, but not in condition C, likely reflecting the greater requirements to construct a motor plan for navigation in conditions A and B than in condition C (Nakayama, Yamagata, Tanji, & Hoshi, 2008).

Neural Activity Associated with Reward

Our VR paradigm allowed us to assess brain activity associated with reward when spatial learning was possible in the active learning condition (A) and when it was attempted but rendered experimentally impossible by the randomization of extra-maze cues in condition B (Table 1). By comparing brain activity associated reward and no reward at the end of trials in condition A with brain activity during the same reward experiences at the end of trials in condition B, we were able to identify the neural correlates of reward experiences during spatial learning. By comparing brain activity during the receipt of reward at the end of trials vs. no receipt of reward at the end of trials in condition A (when spatial learning was possible) with brain activity associated with the same contrast in condition B (when spatial learning was attempted but impossible), we were able to disentangle the effects of reward on learning from the effects of reward itself, in the absence of learning.

Activation of the left hippocampus was greater when participants were rewarded at the end of trials in control condition B, when learning was impossible, relative to condition A (Fig. 4), but activation was greater when not rewarded than when rewarded at the end of trials in condition B (Fig. 5). These findings suggest that hippocampus activation was associated with the confusion, effort, or difficulty of condition B and not associated with spatial learning which was rendered impossible in this condition by the randomization of the extra-maze cues and the participants’ orientation at the start of each trial. Finally, exploratory analyses revealed that activity of the caudate nucleus when rewarded and not rewarded at trial completion was greater in condition C compared with conditions A and B, likely indicating the formation of a stimulus-response association between the arrow and an approach response, and consistent with extensive evidence indicating a role for the dorsal striatum in S-R habit learning in lower animals and humans (Packard & Knowlton, 2002).

Activation of the left hippocampus and the right amygdala was greater when not rewarded compared to when rewarded at the end of trials specifically within condition B, but not condition A (Fig. 5). The hippocampus and amygdala are components of the mesolimbic reward processing system (Heimer & Van Hoesen, 2006). Although the roles of these regions in reward processing are not well understood, accumulating evidence from single-unit recordings in animals suggests that the hippocampus is functionally connected to the ventral tegmental area (VTA) (Lisman & Grace, 2005; Thierry, Gioanni, Degenetais, & Glowinski, 2000), which projects dopamine signals most strongly to the ventral striatum (nucleus accumbens) but also to the hippocampus and amygdala (Wise, 2004). Dopamine is intimately associated with reward processing and reward-based learning (Schultz, 2007; Wise, 2004). Findings from neurophysiological studies suggest that reward unpredictability determines the phasic responses of ventral striatal dopaminergic neurons to rewarding stimuli (Schultz, 2007). These neurons fire when rewards occur without being predicted, but their firing drops below baseline when predicted rewards are omitted. Activation of the left hippocampus, but not the amygdala, was also associated with the receipt of reward at the end of trials in condition B compared to condition A (Fig. 4a). Reward receipt in condition B was unexpected because of the randomization of the extra-maze cues. Although lesion, neurophysiological, and fMRI studies typically implicate the nucleus accumbens in processing reward prediction errors (Cardinal, Pennicott, Sugathapala, Robbins, & Everitt, 2001; Delgado, 2007; Schoenbaum & Setlow, 2003), the nucleus accumbens did not activate with the hippocampus during the unexpected receipt of rewards in condition B or deactivate (i.e., drop below baseline) when participants were not rewarded at the end of trials in condition B. Thus, the receipt and non-receipt of rewards in condition B likely activated the hippocampus via a system or mechanism that is independent of the ventral striatum (Bunzeck & Duzel, 2006; Lisman & Grace, 2005; Wittmann, Bunzeck, Dolan, & Duzel, 2007) and independent of learning in general. Indeed, novel events that are devoid of behavioral relevance, such as the receipt of rewards in condition B, are known to activate the hippocampus but not the substantia nigra or, presumably, the ventral striatum (Stoppel et al., 2009). Moreover, the hippocampus contains neurons that respond to reward delivery (Watanabe & Niki, 1985), and human electrophysiological studies demonstrate that the hippocampus encodes and computes an uncertainty signal (Vanni-Mercier et al., 2009), consistent with our findings and with the possibility that hippocampal encoding of uncertainty and unexpected rewards may be independent of dopaminergic firing and the more classical system that subserves prediction error.

Alternatively, greater hippocampal engagement during the receipt of rewards in condition B compared to condition A may have reflected encoding of the novel (random) configuration of the cues in condition B, consistent with the role of the hippocampus in detecting associative novelty – when familiar stimuli are newly arranged (Nyberg, 2005). The hippocampus was perhaps acting as a match-mismatch detector in condition B, generating signals (e.g., activating) when predictions derived from previous experiences (e.g., the arrangement of the cues in condition A) were violated by current inputs (e.g., the random configuration of the cues in condition B, (Kumaran & Maguire, 2007). However, hippocampal activation was only observed during the reward experiences at the end of trials in condition B, rather than when participants were searching the maze. Thus, we believe that at the end of trials, the confusion, effort, or difficulty of condition B relative to condition A activated the hippocampus, and that this activation was not associated with learning, which was rendered experimentally impossible by the randomization of extra-maze cues in condition B.

Conclusions

Our findings have important implications for understanding the neural correlates of reward-based spatial learning. Our inclusion of two different control conditions and our isolation of different events that occurred during maze navigation allowed for the precise study and disentangling of the neural mechanisms that support spatial navigation and reward processing. Participants engaged mesial temporal, parietal, and retrosplenial areas when searching the maze, consistent with findings from prior studies of spatial navigation. Unlike tasks used in other studies, however, our task did not require participants to process the details of the spatial cues but rather required them to process and remember the spatial layout of the extra-maze environment. Thus, participants activated the parahippocampus rather than the hippocampus during the searching phase of the task. Unlike prior studies, we assessed the neural correlates of reward receipt during spatial learning and found that participants engaged the hippocampus when learning was rendered impossible in condition B. Although we did not offer the participants actual monetary rewards, which may have increased motivation and therefore performance on the task, previous studies have shown that virtual and abstract rewards also engage reward circuits (O’Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001). Our novel design also allowed us to assess the differential contributions of the hippocampus and other temporo-parietal areas to the searching and reward processing aspects of reward-based spatial learning. Finally, this translational research paradigm will facilitate parallel studies in animals and humans to establish the processes that are common and unique to each of the two species. Animal studies using the task will then permit cellular and molecular investigations of learning processes that are not possible in humans, but that may nevertheless inform experimental manipulations, such as the development and testing of novel medications, during use of the identical task in humans.

Acknowledgments

This work was supported in part by NIMH grants K02-74677 and K01-MH077652, a NIH/NIBIB grant 1R03EB008235, a grant from the National Alliance for Research on Schizophrenia and Depression (NARSAD), by funding from the Sackler Institute for Developmental Psychobiology, Columbia University and from the Opening Project of Shanghai Key Laboratory of Functional Magnetic Resonance Imaging, East China Normal University. We thank Tiago V. Maia for his valuable comments.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Adcock RA, Thangavel A, Whitfield-Gabrieli S, Knutson B, Gabrieli JD. Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron. 2006;50(3):507–517. doi: 10.1016/j.neuron.2006.03.036. [DOI] [PubMed] [Google Scholar]
  2. Aguirre GK, Detre JA, Alsop DC, D’Esposito M. The parahippocampus subserves topographical learning in man. Cerebral Cortex. 1996;6(6):823–829. doi: 10.1093/cercor/6.6.823. [DOI] [PubMed] [Google Scholar]
  3. Aminoff E, Gronau N, Bar M. The parahippocampal cortex mediates spatial and nonspatial associations. Cerebral Cortex. 2007;17(7):1493–1503. doi: 10.1093/cercor/bhl078. [DOI] [PubMed] [Google Scholar]
  4. Balan PF, Gottlieb J. Integration of exogenous input into a dynamic salience map revealed by perturbing attention. Journal of Neuroscience. 2006;26(36):9239–9249. doi: 10.1523/JNEUROSCI.1898-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Becker JT, Walker JA, Olton DS. Neuroanatomical bases of spatial memory. Brain Research. 1980;200(2):307–320. doi: 10.1016/0006-8993(80)90922-1. [DOI] [PubMed] [Google Scholar]
  6. Blatt GJ, Pandya DN, Rosene DL. Parcellation of cortical afferents to three distinct sectors in the parahippocampal gyrus of the rhesus monkey: an anatomical and neurophysiological study. Journal of Comparative Neurology. 2003;466(2):161–179. doi: 10.1002/cne.10866. [DOI] [PubMed] [Google Scholar]
  7. Bunzeck N, Duzel E. Absolute coding of stimulus novelty in the human substantia nigra/VTA. Neuron. 2006;51(3):369–379. doi: 10.1016/j.neuron.2006.06.021. [DOI] [PubMed] [Google Scholar]
  8. Burwell RD, Hafeman DM. Positional firing properties of postrhinal cortex neurons. Neuroscience. 2003;119(2):577–588. doi: 10.1016/s0306-4522(03)00160-x. [DOI] [PubMed] [Google Scholar]
  9. Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science. 2001;292(5526):2499–2501. doi: 10.1126/science.1060818. [DOI] [PubMed] [Google Scholar]
  10. Cho J, Sharp PE. Head direction, place, and movement correlates for cells in the rat retrosplenial cortex. Behavioral Neuroscience. 2001;115(1):3–25. doi: 10.1037/0735-7044.115.1.3. [DOI] [PubMed] [Google Scholar]
  11. Clower DM, West RA, Lynch JC, Strick PL. The inferior parietal lobule is the target of output from the superior colliculus, hippocampus, and cerebellum. Journal of Neuroscience. 2001;21(16):6283–6291. doi: 10.1523/JNEUROSCI.21-16-06283.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience. 2002;3(3):201–215. doi: 10.1038/nrn755. [DOI] [PubMed] [Google Scholar]
  13. Delgado MR. Reward-related responses in the human striatum. Annals of the New York Academy of Science. 2007;1104:70–88. doi: 10.1196/annals.1390.002. [DOI] [PubMed] [Google Scholar]
  14. Driver J, Vuilleumier P. Perceptual awareness and its loss in unilateral neglect and extinction. Cognition. 2001;79(1–2):39–88. doi: 10.1016/s0010-0277(00)00124-4. [DOI] [PubMed] [Google Scholar]
  15. Ekstrom AD, Kahana MJ, Caplan JB, Fields TA, Isham EA, Newman EL, et al. Cellular networks underlying human spatial navigation. Nature. 2003;425(6954):184–188. doi: 10.1038/nature01964. [DOI] [PubMed] [Google Scholar]
  16. Epstein RA, Higgins JS, Jablonski K, Feiler AM. Visual scene processing in familiar and unfamiliar environments. Journal of Neurophysiology. 2007;97(5):3670–3683. doi: 10.1152/jn.00003.2007. [DOI] [PubMed] [Google Scholar]
  17. Etienne AS, Jeffery KJ. Path integration in mammals. Hippocampus. 2004;14(2):180–192. doi: 10.1002/hipo.10173. [DOI] [PubMed] [Google Scholar]
  18. First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Non-Patient Edition (SCID-I/NP) New York: Biometrics Research, New York State Psychiatric Institute; 2002. [Google Scholar]
  19. Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): Use of a cluster-size threshold. Magnetic Resonance Medicine. 1995;33:636–647. doi: 10.1002/mrm.1910330508. [DOI] [PubMed] [Google Scholar]
  20. Friston KJ, Williams S, Howard R, Frackowiak RS, Turner R. Movement-related effects in fMRI time-series. Magnetic Resonance Medicine. 1996;35:346–355. doi: 10.1002/mrm.1910350312. [DOI] [PubMed] [Google Scholar]
  21. Friston KJ, Worsley KJ, Frackowiak RSJ, Mazziotta JC, Evans AC. Assessing the significance of focal activations using their spatial extent. Human Brain Mapping. 1994;1:214–220. doi: 10.1002/hbm.460010306. [DOI] [PubMed] [Google Scholar]
  22. Gron G, Wunderlich AP, Spitzer M, Tomczak R, Riepe MW. Brain activation during human navigation: gender-different neural networks as substrate of performance. Nature Neuroscience. 2000;3(4):404–408. doi: 10.1038/73980. [DOI] [PubMed] [Google Scholar]
  23. Hartley T, Maguire EA, Spiers HJ, Burgess N. The well-worn route and the path less traveled: distinct neural bases of route following and wayfinding in humans. Neuron. 2003;37(5):877–888. doi: 10.1016/s0896-6273(03)00095-3. [DOI] [PubMed] [Google Scholar]
  24. Hassabis D, Chu C, Rees G, Weiskopf N, Molyneux PD, Maguire EA. Decoding neuronal ensembles in the human hippocampus. Current Biology. 2009;19(7):546–554. doi: 10.1016/j.cub.2009.02.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Heimer L, Van Hoesen GW. The limbic lobe and its output channels: implications for emotional functions and adaptive behavior. Neuroscience and Biobehavioral Reviews. 2006;30(2):126–147. doi: 10.1016/j.neubiorev.2005.06.006. [DOI] [PubMed] [Google Scholar]
  26. Himmelbach M, Erb M, Karnath HO. Exploring the visual world: the neural substrate of spatial orienting. Neuroimage. 2006;32(4):1747–1759. doi: 10.1016/j.neuroimage.2006.04.221. [DOI] [PubMed] [Google Scholar]
  27. Hollingshead AB. Four-factor index of social status. New Haven, CT: Yale University Press; 1975. [Google Scholar]
  28. Iaria G, Chen JK, Guariglia C, Ptito A, Petrides M. Retrosplenial and hippocampal brain regions in human navigation: complementary functional contributions to the formation and use of cognitive maps. European Journal of Neuroscience. 2007;25(3):890–899. doi: 10.1111/j.1460-9568.2007.05371.x. [DOI] [PubMed] [Google Scholar]
  29. Iaria G, Fox CJ, Chen JK, Petrides M, Barton JJ. Detection of unexpected events during spatial navigation in humans: bottom-up attentional system and neural mechanisms. European Journal of Neuroscience. 2008;27(4):1017–1025. doi: 10.1111/j.1460-9568.2008.06060.x. [DOI] [PubMed] [Google Scholar]
  30. Ino T, Doi T, Hirose S, Kimura T, Ito J, Fukuyama H. Directional disorientation following left retrosplenial hemorrhage: a case report with fMRI studies. Cortex. 2007;43(2):248–254. doi: 10.1016/s0010-9452(08)70479-9. [DOI] [PubMed] [Google Scholar]
  31. Ito R, Robbins TW, Pennartz CM, Everitt BJ. Functional interaction between the hippocampus and nucleus accumbens shell is necessary for the acquisition of appetitive spatial context conditioning. Journal of Neuroscience. 2008;28(27):6950–6959. doi: 10.1523/JNEUROSCI.1615-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Jahn K, Deutschlander A, Stephan T, Strupp M, Wiesmann M, Brandt T. Brain activation patterns during imagined stance and locomotion in functional magnetic resonance imaging. Neuroimage. 2004;22(4):1722–1731. doi: 10.1016/j.neuroimage.2004.05.017. [DOI] [PubMed] [Google Scholar]
  33. Jazzard P, Matthews PM, Smith SM. Functional MRI—An Introduction to Methods. New York: Oxford University; 2002. [Google Scholar]
  34. Kandel ER, Schwartz JH, Jessell TM, editors. Principles of neural science. 3. East Norwalk, CT: Appleton & Lange; 1991. [Google Scholar]
  35. Kobayashi Y, Amaral DG. Macaque monkey retrosplenial cortex: II. Cortical afferents. Journal of Comparative Neurology. 2003;466(1):48–79. doi: 10.1002/cne.10883. [DOI] [PubMed] [Google Scholar]
  36. Kopniczky Z, Dochnal R, Macsai M, Pal A, Kiss G, Mihaly A, et al. Alterations of behavior and spatial learning after unilateral entorhinal ablation of rats. Life Science. 2006;78(23):2683–2688. doi: 10.1016/j.lfs.2005.10.014. [DOI] [PubMed] [Google Scholar]
  37. Kumaran D, Maguire EA. Match mismatch processes underlie human hippocampal responses to associative novelty. Journal of Neuroscience. 2007;27(32):8517–8524. doi: 10.1523/JNEUROSCI.1677-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lisman JE, Grace AA. The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron. 2005;46(5):703–713. doi: 10.1016/j.neuron.2005.05.002. [DOI] [PubMed] [Google Scholar]
  39. Maguire EA, Burgess N, Donnett JG, Frackowiak RS, Frith CD, O’Keefe J. Knowing where and getting there: a human navigation network. Science. 1998;280(5365):921–924. doi: 10.1126/science.280.5365.921. [DOI] [PubMed] [Google Scholar]
  40. Malhotra P, Coulthard EJ, Husain M. Role of right posterior parietal cortex in maintaining attention to spatial locations over time. Brain. 2009 doi: 10.1093/brain/awn350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Morrell CH, Pearson JD, Brant LJ. Linear transformations of linear mixed-effects models. American Statistician. 1997;51:338–343. [Google Scholar]
  42. Nakayama Y, Yamagata T, Tanji J, Hoshi E. Transformation of a virtual action plan into a motor plan in the premotor cortex. Journal of Neuroscience. 2008;28(41):10287–10297. doi: 10.1523/JNEUROSCI.2372-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Nitz DA. Tracking route progression in the posterior parietal cortex. Neuron. 2006;49(5):747–756. doi: 10.1016/j.neuron.2006.01.037. [DOI] [PubMed] [Google Scholar]
  44. Norman G, Eacott MJ. Dissociable effects of lesions to the perirhinal cortex and the postrhinal cortex on memory for context and objects in rats. Behavioral Neuroscience. 2005;119(2):557–566. doi: 10.1037/0735-7044.119.2.557. [DOI] [PubMed] [Google Scholar]
  45. Nyberg L. Any novelty in hippocampal formation and memory? Current Opinion in Neurology. 2005;18(4):424–428. doi: 10.1097/01.wco.0000168080.99730.1c. [DOI] [PubMed] [Google Scholar]
  46. O’Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neuroscience. 2001;4(1):95–102. doi: 10.1038/82959. [DOI] [PubMed] [Google Scholar]
  47. Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9:97–113. doi: 10.1016/0028-3932(71)90067-4. [DOI] [PubMed] [Google Scholar]
  48. Olton DS, Samuelson RJ. Remembrance of places past: Spatial memory in rats. Journal of Experimental Psychology. 1976;2:97–115. [Google Scholar]
  49. Oristaglio J, Schneider DM, Balan PF, Gottlieb J. Integration of visuospatial and effector information during symbolically cued limb movements in monkey lateral intraparietal area. Journal of Neuroscience. 2006;26(32):8310–8319. doi: 10.1523/JNEUROSCI.1779-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Packard MG, Hirsh R, White NM. Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. Journal of Neuroscience. 1989;9(5):1465–1472. doi: 10.1523/JNEUROSCI.09-05-01465.1989. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Packard MG, Knowlton BJ. Learning and memory functions of the Basal Ganglia. Annual Reviews of Neuroscience. 2002;25:563–593. doi: 10.1146/annurev.neuro.25.112701.142937. [DOI] [PubMed] [Google Scholar]
  52. Packard MG, White NM. Dissociation of hippocampus and caudate nucleus memory systems by posttraining intracerebral injection of dopamine agonists. Behavioral Neuroscience. 1991;105:295–306. doi: 10.1037//0735-7044.105.2.295. [DOI] [PubMed] [Google Scholar]
  53. Schoenbaum G, Setlow B. Lesions of nucleus accumbens disrupt learning about aversive outcomes. Journal of Neuroscience. 2003;23(30):9833–9841. doi: 10.1523/JNEUROSCI.23-30-09833.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Schultz W. Behavioral dopamine signals. Trends in Neuroscience. 2007;30(5):203–210. doi: 10.1016/j.tins.2007.03.007. [DOI] [PubMed] [Google Scholar]
  55. Servos P, Osu R, Santi A, Kawato M. The neural substrates of biological motion perception: an fMRI study. Cerebral Cortex. 2002;12(7):772–782. doi: 10.1093/cercor/12.7.772. [DOI] [PubMed] [Google Scholar]
  56. Sharp PE. Complimentary roles for hippocampal versus subicular/entorhinal place cells in coding place, context, and events. Hippocampus. 1999;9(4):432–443. doi: 10.1002/(SICI)1098-1063(1999)9:4<432::AID-HIPO9>3.0.CO;2-P. [DOI] [PubMed] [Google Scholar]
  57. Spiers HJ, Maguire EA. Spontaneous mentalizing during an interactive real world task: an fMRI study. Neuropsychologia. 2006a;44(10):1674–1682. doi: 10.1016/j.neuropsychologia.2006.03.028. [DOI] [PubMed] [Google Scholar]
  58. Spiers HJ, Maguire EA. Thoughts, behaviour, and brain dynamics during navigation in the real world. Neuroimage. 2006b;31(4):1826–1840. doi: 10.1016/j.neuroimage.2006.01.037. [DOI] [PubMed] [Google Scholar]
  59. Stoppel CM, Boehler CN, Strumpf H, Heinze HJ, Hopf JM, Duzel E, et al. Neural correlates of exemplar novelty processing under different spatial attention conditions. Human Brain Mapping. 2009;30(11):3759–71. doi: 10.1002/hbm.20804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Thierry AM, Gioanni Y, Degenetais E, Glowinski J. Hippocampo-prefrontal cortex pathway: anatomical and electrophysiological characteristics. Hippocampus. 2000;10(4):411–419. doi: 10.1002/1098-1063(2000)10:4<411::AID-HIPO7>3.0.CO;2-A. [DOI] [PubMed] [Google Scholar]
  61. Vanni-Mercier G, Mauguiere F, Isnard J, Dreher JC. The hippocampus codes the uncertainty of cue-outcome associations: an intracranial electrophysiological study in humans. Journal of Neuroscience. 2009;29(16):5287–5294. doi: 10.1523/JNEUROSCI.5298-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Watanabe T, Niki H. Hippocampal unit activity and delayed response in the monkey. Brain Research. 1985;325(1–2):241–254. doi: 10.1016/0006-8993(85)90320-8. [DOI] [PubMed] [Google Scholar]
  63. White NM. The role of stimulus ambiguity and movement in spatial navigation: a multiple memory systems analysis of location discrimination. Neurobioly of Learning and Memory. 2004;82(3):216–229. doi: 10.1016/j.nlm.2004.05.004. [DOI] [PubMed] [Google Scholar]
  64. Wiener SI, Paul CA, Eichenbaum H. Spatial and behavioral correlates of hippocampal neuronal activity. Journal of Neuroscience. 1989;9(8):2737–2763. doi: 10.1523/JNEUROSCI.09-08-02737.1989. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Wise RA. Dopamine, learning and motivation. Nature Reviews Neuroscience. 2004;5(6):483–494. doi: 10.1038/nrn1406. [DOI] [PubMed] [Google Scholar]
  66. Wittmann BC, Bunzeck N, Dolan RJ, Duzel E. Anticipation of novelty recruits reward system and hippocampus while promoting recollection. Neuroimage. 2007;38(1):194–202. doi: 10.1016/j.neuroimage.2007.06.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Wolbers T, Buchel C. Dissociable retrosplenial and hippocampal contributions to successful formation of survey representations. Journal of Neuroscience. 2005;25(13):3333–3340. doi: 10.1523/JNEUROSCI.4705-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Wolbers T, Weiller C, Buchel C. Neural foundations of emerging route knowledge in complex spatial environments. Brain Research. Cognitive Brain Research. 2004;21(3):401–411. doi: 10.1016/j.cogbrainres.2004.06.013. [DOI] [PubMed] [Google Scholar]

RESOURCES