Functional contributions and interactions between the human hippocampus and subregions of the striatum during arbitrary associative learning and memory

Abstract

The hippocampus and striatum are thought to have different functional roles in learning and memory. It is unknown under what experimental conditions their contributions are dissimilar or converge, and the extent to which they interact over the course of learning. In order to evaluate both the functional contributions of as well as the interactions between the human hippocampus and striatum, the present study used high-resolution functional magnetic resonance imaging (fMRI) and variations of a conditional visuomotor associative learning task that either taxed arbitrary associative learning (Experiment 1) or stimulus-response learning (Experiment 2). In the first experiment we observed changes in activity in the hippocampus and anterior caudate that reflect differences between the two regions consistent with distinct computational principles. In the second experiment we observed activity in the putamen that reflected content specific representations during the learning of arbitrary conditional visuomotor associations. In both experiments the hippocampus and ventral striatum demonstrated dynamic functional coupling during the learning of new arbitrary associations, but not during retrieval of well-learned arbitrary associations using control variants of the tasks that did not preferentially tax one system versus the other. These findings suggest that both the hippocampus and subregions of the dorsal striatum contribute uniquely to the learning of arbitrary associations while the hippocampus and ventral striatum interact over the course of learning.

Introduction

The hippocampus and striatum are two brain regions critical for learning and memory (Poldrack and Packard, 2003). The hippocampus is important for the learning of arbitrary configural or relational information (Sutherland and Rudy, 1989; Cohen & Eichenbaum, 1993; Ranganath, 2010), while the striatum, comprised of functionally complementary subregions, contributes to goal-directed and habitual behavior through feedback based learning and memory (Shohamy et al., 2004; Yin and Knowlton, 2006; Balleine and O'Doherty, 2010). These two regions are thought to compete during learning (Schroeder et al., 2002; Poldrack and Packard, 2003), however direct anatomical connections between the ventral striatum and hippocampus (Groenwegen et al., 1987) as well as increased functional coupling during learning (Mattfeld and Stark, 2010) support evidence for cooperative processes. This study aims to investigate the learning related activity of the hippocampus and functionally distinct subregions of the striatum and their interactions under different experiments that systematically vary the computational compatibility between task conditions.

We used two conditional visuomotor associative learning tasks (Canavan et al., 1989; Murray and Wise, 1996; Brasted et al., 2003; Wirth et al., 2003; Hadj-Bouziane and Boussaoud, 2003; Buch et al., 2006; Grol et al., 2006; Haruno and Kawato, 2006; Williams and Eskandar, 2006; Brovelli et al., 2008) that manipulated either the probability of receiving valid feedback (Experiment 1) or the consistency of stimulus-response mappings (Experiment 2) to investigate the contributions and interactions of the hippocampus, the anterior caudate, the putamen, and the ventral striatum during conditional visuomotor associative learning.

Tasks that utilize probabilistic feedback reliably recruit anterior regions of the caudate and reductions in hippocampal activations (Poldrack et al., 2001; Delgado et al., 2005). In contrast, the hippocampus is important for tasks that require the learning of episode unique information (Eichenbaum, 2004). Thus, in the first experiment, we focused our analyses on these two regions and hypothesized that activations in the anterior caudate would correlate with learning under stochastic feedback task conditions while the hippocampus would correlate with learning when participants received deterministic feedback.

In the second experiment, participants were required to learn arbitrary stimulus-color associations that either remained in the same location across the experiment (static) or randomly changed location on every trial (dynamic). The putamen has been shown to be important for the learning and retention of stimulus response associations (Nixon et al., 2004; Tricomi et al., 2009), however in the current experiment no stable stimulus response associations were available to learn. Therefore, we hypothesized that activity in the putamen of participants in the dynamic color location condition would not correlate with learning. In contrast, the hippocampus has been shown to be critical for paired associate learning (Yoon et al., 2011), thus the functional role of the hippocampus is well suited for encoding the association between arbitrary pairs of stimuli. In both experiments we evaluated learning related changes in functional coupling between the ventral striatum and hippocampus. We posited that functional correlations would increase during learning for task conditions that did not preferentially impact one learning system versus the other.

Materials and Methods

Participants

A total of 97 right-handed volunteers participated, 51 in the first experiment (28 Female, mean age 20.1, age range 18-25) and 46 in the second experiment (24 Female, mean age 21.9, age range 18-27). All participants had no history of neurological disease or psychiatric illness and normal or corrected to normal vision. Participants were recruited from the University of California, Irvine community, gave written informed consent, and were paid for their participation. In the first experiment three participants were dropped from further analysis – one for excessive movement and two others following scanner difficulties, leaving a total of 48 participants. In the second experiment two participants were dropped from further analysis due to scanner related artifacts, leaving a total of 44 participants.

Materials

Stimuli were computer-generated kaleidoscopic images (Miyashita et al., 1991) presented in the center of the screen against a black background. A total of 200 kaleidoscopic images were generated prior to the experiment. The last four images from the set served as the reference stimuli for all participants. New associative learning stimuli were randomly selected from the remaining 196 available images.

Procedure

The experiments reported here are variants of a previously published CVAL task (Law et al., 2005; Kirwan et al., 2007; Mattfeld and Stark, 2010). Each experiment was designed to make learning more or less dependent on distinct learning and memory processes. CVAL tasks consist of learning to associate arbitrary stimuli with arbitrary responses through trail and error. During both experiments participants saw one of three trial types: 1) new learning trials; 2) reference trials; or 3) perceptual baseline trials. Reference stimuli were included in each experiment to compare the BOLD activity of well-learned associations to those that were currently being learned. Perceptual baseline trials were presented to establish a reference for the fMRI signal and further induce jitter between trial types aiding in the estimation of the hemodynamic response.

Procedure: CVAL deterministic vs. stochastic feedback (Experiment 1)

In the first experiment participants were randomly assigned to either a deterministic or stochastic feedback condition. Stochastic feedback was used to make the learning of arbitrary associations less dependent on the hippocampus and more dependent on the anterior caudate. Participants were informed that each kaleidoscopic image was associated with a specific response corresponding to one of the squares on the screen and their job was to learn through trial-and-error which response was paired with each image. Each trial began with the presentation a kaleidoscopic image and four horizontal superimposed square outlines were presented for 500 ms. A brief delay (700 ms) followed the presentation of the stimulus. The kaleidoscopic image was removed during the delay period leaving only the fixation cross and square outlines on the screen. The response period (700 ms) began when the fixation cross was replaced by the cue “Go!”. During the response period, participants selected one of the four square outlines on the screen with an MR-compatible button box. The selected square outline filled with white, indicating which response had been recorded. Feedback (800 ms) was provided following the response period: a green “yes” if they were correct, a red “no” if they were incorrect, and a “?” if they failed to respond in time. The inter-trial-interval consisted of a central fixation cross presented for 300 ms (Figure 1A). Participants in the deterministic feedback version received 100% valid feedback on each trial. In contrast, participants in the stochastic feedback condition received valid feedback 80% of the time (a green “yes” when they were correct and a red “no” when they were incorrect) and invalid feedback 20% of the time (a red “no” when they were correct and a green “yes” when they were incorrect).

Figure 1.

Example kaleidoscopic image and trial structure for Experiment 1, deterministic versus probabilistic learning conditional visuomotor associative learning task (A), Experiment 2, static versus dynamic stimulus color associative learning task (B), and baseline trials (C).

Procedure: CVAL static versus dynamic motor component (Experiment 2)

The second experiment required participants to learn arbitrary stimulus-stimulus associations in an effort to make learning more dependent on the hippocampus and less dependent on sensorimotor regions of the striatum. Participants were randomly assigned to one of two conditions: static versus dynamic. In both conditions, there was a consistent association between a kaleidoscopic image and the color of the correct response button. In the static learning condition, the color remained in the same location across the experiment such that not only was there a constant association between a stimulus and its correct color, but there was also a constant association between the stimulus and its correct response location. In the dynamic learning condition, the colors randomly changed location on every trial. The trial timing and structure were very similar to the first experiment. Trials began with the presentation of a kaleidoscopic image and four superimposed square outlines (500 ms). During the delay period (700 ms) the four outlines filled with one of four colors (red, green, blue, and yellow). Following a cue (“Go!”) participants had 700 ms to select which color they believed was associated with the kaleidoscopic image presented at the beginning of the trial. Immediately following their response the selected color turned white (Figure 1B). Each trial was separated by the presentation of a fixation cross for 300 ms (inter-trial-interval). Participants were told that each kaleidoscopic image was associated with a color and their job was to learn through trial and error which color was associated with each image. Following their selection participants were provided feedback (800 ms) similar to the first experiment: a green “yes” if they were correct, a red “no” if they were incorrect, and a “?” if they did not respond in time.

Procedure: Baseline trials

The timing of baseline trials was the same as test and reference trials. However, rather than a kaleidoscopic image, participants were shown a fixed random visual static pattern. The four squares were filled with a transparent white. One of the squares was randomly assigned as the target on each baseline trial and set to a slightly greater opacity then the three remaining options. Participants were instructed to identify the brightest box and press the corresponding button. Following their response the selected box turned fully-opaque white, indicating their selection. Feedback was provided while the random visual static pattern as well as the selected and remaining boxes were presented until the trial ended. The opacity of the target varied, titrating the difficulty to each participant's performance (Figure 1C).

Procedure: Prescan training

Participants were trained on a set of four “reference” stimuli 24 to 48 hours before scanning. The relevant associations for each “reference” stimulus were consistent across each experiment. However, the relevant associations were learned according to their respective experimental conditions (e.g., reference stimuli in the stochastic feedback experiment were learned via stochastic feedback). Prescan training consisted of 360 trials (240 “reference” stimulus trials and 120 baseline trials). Participants also received training prior to the beginning of the experiment on the day of the scan. During the acquisition of the T1 weighted structural scan participants performed an additional 120 trials (80 “reference” trials and 40 baseline trials) to ensure the associations for “reference” stimuli were well-learned prior to beginning scanning.

Scanning sessions

Both experiments were divided into six runs. Each run consisted of 132 total trials (72 associative learning trials, 30 reference trials, and 30 baseline trials). Trial order was randomly determined. Each run lasted 6.5 minutes. To maximize the number of associative learning trials, new associative learning stimuli were removed and replaced during each run when real-time behavioral performance met the criterion of five out of the last six responses were correct. Additionally, to equate performance across conditions in both experiments participants in the 100%-valid feedback and static color location conditions learned the associations of 8 stimuli concurrently, while participants in the 80%-valid feedback and dynamic color location conditions learned 4 associations concurrently.

fMRI imaging

All scanning was performed on a Phillips 3.0 Tesla scanner (Best, the Netherlands), using a SENSE (Sensitivity Encoding) head coil at the Research Imaging Center at UC Irvine. During each scanning run, 198, T2*-weighted single-shot echoplanar volumes were acquired covering the majority of the MTL and basal ganglia (35 slices). Each slice was 1.8 mm thick separated by a 0.2 mm gap. Functional pulse sequences had a repetition time (TR) of 2000 ms, an echo time (TE) of 26 ms, a flip angle of 70°, an acquisition matrix size of 128 × 128 mm, a field of view (FOV) of 180 × 180 mm, and a SENSE factor of 2.5, resulting in an in-plane acquisition resolution of 1.5 × 1.5 mm. The first four functional volumes were discarded to accommodate for T1 equalization. T1-weighted whole-brain anatomical images were acquired using a sagittal magnetization-prepared rapid gradient echo (MP-RAGE) scan (TR, 11 ms; TE, 4.6 ms; flip angle, 18°; matrix size 320 × 264 mm; FOV, 240 × 150 mm; resolution 0.75 mm isotropic; 200 slices).

Behavioral data analysis

We used a logistic regression algorithm to calculate a trial-by-trial probability correct estimate as well as its 95% confidence interval (Wirth et al., 2003; Smith et al., 2004) for each kaleidoscopic image. The algorithm uses a state equation (Gaussian random-walk model) and an observation equation (Bernoulli model) to calculate trial-by-trial probability correct estimates for each stimulus based on the behavioral performance (1, correct; 0, incorrect). The stimulus specific learning curves provided a measure of trial-wise learning. Grouping the trial-specific probability correct estimates into five equivalently spaced bins created discrete memory strength indices (Str1 to Str5). We used the memory strength indices combined with Reference trials to assess learning related changes in BOLD fMRI activity. In other words, the memory strength analysis permits us to characterize changes in functional activations related to how well associations have been learned. Moreover, by combining trials of similar memory strengths across stimuli we increase our signal to noise when examining learning curves. This approach also helps to mitigate concerns over variations in learning rates across conditions as regardless of learning rate, trials in the same memory strength bin share the same approximate probability correct.

We evaluated the number of trials to reach the onset of learning for each kaleidoscopic image defined as the trial when the lower 95% confidence interval exceeded chance performance (25%). For each participant we assessed the total number of stimuli that they encountered throughout the course of the experiment. Lastly, we evaluated the change in reaction time across the discrete memory strength indices and reference trials.

fMRI data analysis

We used Analysis of Functional Neuroimages (AFNI; Cox, 1996) and Advanced Normalization Tools (ANTs; Avants et al., 2008) software to perform all imaging data analyses. Functional volumes were slice-time and motion corrected. Time points exceeding 3° rotation, 2 mm translation, or 2 standard deviations away from the mean within run global signal intensity, as well as the immediately adjacent TRs were not analyzed. Each run was concatenated into a single time series for each participant. Prior to concatenation functional data were spatially smoothed to a targeted isotropic 3 mm FWHM using AFNI's 3dBlurToFWHM.

Normalization of each participant's T1-weighted MP-RAGE used the ANTs toolkit (Avants et al., 2008) and began with a 12-parameter affine registration to a template based on the Talairach atlas (Talairach and Tournoux, 1988). This first step mitigates large differences between participants prior to the more fine-tuned alignment. The second step utilized SyN to create a diffeomorphic 3D vector field mapping each participant's brain to a template space. We used a template derived from previous work in our laboratory in young adults (Lacy et al., 2010). The resulting transformation parameters were applied to the functional data following first level analyses.

fMRI data analysis: Memory strength analysis

Behavioral design matrices included regressors for trials from the five memory strength indices, reference trials, as well as the first presentation of each stimulus. Nuisance regressors coding for mean offset, linear, and higher order drifts in the MR signal were also included. We used a deconvolution approach based on multiple linear regression to analyze each participant's data. The hemodynamic response for each event of interest is estimated using 9 time-shifted tent functions, estimating the BOLD activity from 0 to 16 s after trial onset. The resulting time-shifted beta coefficients represent activity versus baseline for each regressor of interest at a given time point in each voxel. We used the summed beta coefficients (summing over 4 to 12 s after trial onset) as the model's estimate for each regressor of interest. Our experimental design and analyses do not permit us to isolate activity related to distinct periods within each trial (e.g., action selection, feedback, etc...). Thus, the summed estimate for each regressor of interest cannot be used to interpret within trial processes but rather reflects the activity over the entire trial.

Anatomical region of interest analyses

We anatomically defined the bilateral hippocampus, head of the caudate, and putamen in template space. The bilateral hippocampus was segmented according to the boundaries and landmarks outlined in Duvernoy (2005), while the head of the caudate and putamen were based on landmarks described in the Atlas of the Human Brain (Mai et al., 1997). Specifically, the head of the caudate extended inferiorly to z = −2 and was bounded by the lateral ventricles on its medial surface and the internal capsule on its lateral surface. The most posterior or rostral extent of the head of the caudate extended to y = −1, or until the last coronal slice where the anterior commissure was visible. We selected these anatomical regions based on prior work in our laboratory using a similar task showing that activity in the hippocampus increases in a linear fashion across memory strength (Law et al., 2005; Kirwan et al., 2007), while activity in the anterior caudate and putamen increases during learning and subsequently decreases when associations are well-learned (Mattfeld and Stark, 2010). The summed beta values were averaged across all voxels within the masks for each event of interest – memory strength indices and reference trials.

For each experiment we had two hypotheses. In the first experiment we expected to observe a change in activity across memory strength and reference trials in the bilateral hippocampus for participants in the 100%-valid feedback condition but not in participants who received 80%-valid feedback. In contrast, we expected participants in the 80%-valid feedback condition to show a learning related change in activity in the bilateral anterior caudate. In the second experiment we hypothesized that both groups would exhibit similar learning related activity in the bilateral hippocampus, while participants in the static color location group would exhibit changes in activity across memory strength and reference trials in the putamen.

Anatomical region of interest analyses: statistical tests

Based on prior work utilizing this task (Law et al., 2005; Wirth et al., 2003) we hypothesized that changes in hippocampal activity that correlated with learning would exhibit a monotonic increase that could be well-fit by a linear trend. In contrast, we expect activity in the anterior caudate and putamen to increase as associative strength increases and subsequently decrease for well learned associations (Williams and Eskandar, 2006; Mattfeld and Stark, 2010). Activity throughout the striatum therefore should be best fit by a non-monotonic curve such as a quadratic trend. We do not mean to imply that these curves must be linear and quadratic respectively. Rather, we propose that they are monotonic and non-monotonic. Linear and quadratic are merely the instantiations of these that have the fewest assumptions. To test our hypotheses we performed group (Experiment 1: 100%-valid feedback versus 80%-valid feedback; Experiment 2: Static color location versus Dynamic color location) by linear (contrast weights: −5, −3, −1, 1, 3, 5) and quadratic (contrast weights: −5, 1, 4, 4, 1, −5) trend analysis of variance (ANOVA) on the data obtained from the hippocampal and striatal ROIs. To evaluate the difference in explained variance related to each analysis we report the their related partial eta squared values (η_p² = SS_effect/(SS_effect + SS_error)).

fMRI data analysis: Psychophysiological interaction (PPI) analysis

We performed a psychophysiological interaction (PPI) analysis (Friston et al., 1997) to assess the changes in functional coupling between the hippocampus and ventral striatum as a function of learning across the different experimental manipulations. PPI analyses utilize the interaction between activity in one brain region and a psychological context to account for activity in another region. Memory strength indices were used to define the psychological context term. For the PPI analysis we added two regressors to each participant's original design matrix: one regressor for the time series activity from the seed region and a second representing the interaction between our learning contexts and the time series from our seed region. To create the time series and interaction regressors we isolated the activity for all events of interest. First, we used the hand segmented ROIs in template space back projected to subject specific space to calculate mean time series for each seed region by averaging across all voxels within each ROI. We then deconvolved the resulting time series into its underlying neural function (Gitelman et al., 2003). Interaction terms were then created by combining the physiological event (deconvolved time series) with an orthogonal set of contrast weights coding for a linear, quadratic, and cubic change across learning. The resulting neural interaction terms were then convolved with a gamma basis function using AFNI's waver program.

We assessed the change in correlation across learning by determining whether or not the correlation between our seed regions and the rest of the brain changed as a linear, quadratic, or a cubic function of memory strength. Separate models were used to test whether regions showed an interaction in correlation following either a linear, quadratic, or a cubic change in memory strength. The correlation coefficients for the interaction terms from the separate models were Fisher's z-transformed and analyzed at the group level using repeated measures ANOVAs to identify voxels showing any change in correlation as a function of memory strength (e.g., linear, quadratic, or cubic). When correcting for multiple comparisons we were only interested in how the correlations change between regions. Therefore, when evaluating the change in correlation across memory strength with striatal seeds we limited our analysis to the bilateral hippocampus. Similarly, we limited our analysis to the bilateral striatum when using seeds from the hippocampus. The reduced volumes required a final spatial extent threshold of 78 mm³ at the same height threshold of p < 0.01 to result in an overall corrected alpha probability of p < 0.05 as determined by AFNI's “AlphaSim” program. A similar procedure was used to create PPI terms for each memory strength index separately; these data were used for display purposes only.

Results

Behavioral performance

Behavioral results (Experiment 1)

The estimated onset of learning was defined as the trial when the lower 95% confidence interval exceeded chance performance (25%). Participants in the 100%-valid feedback condition required fewer training trials (mean [SD]: 8.8 [2.4]; range, 7 – 14) than participants in the 80%-valid feedback condition (mean [SD]: 15.2 [7.9]; range, 8 – 42) to reach the onset of learning (t₄₆ = 3.18, p = 0.0004). Participants in the 100%-valid feedback condition learned more new stimuli to criterion (mean [SD]: 31.9 [10.4], range, 9 – 47) than participants in the 80%-valid feedback condition (mean [SD]: 16.3 (6.7); range, 3 – 31) (t₄₆ = 6.16, p < 0.0001). The groups (100%- and 80%-valid feedback) did not significantly differ from each other in reaction time (main effect of group; F_1,46 = 1.5, p = 0.22). However, we did observe a main effect of memory strength (F_5,230 = 6.1, p < 0.0001) and a group by memory strength interaction (F_5,230 = 2.6, p = 0.025), suggesting that the participants in the 100%-valid feedback condition had a greater decrease in their reaction time across memory strength than participants in the 80%-valid feedback condition.

Behavioral results (Experiment 2)

In the second experiment the onset of learning was defined similarly to Experiment 1. The groups (static and dynamic) did not significantly differ from one another on the average number of training trials required to reach the onset of learning (static mean [SD]: 8.2 [1.9], range, 6 – 14; dynamic mean [SD]: 9.7 [4.8], range, 4 – 24) (t₄₂ = 1.33, p = 0.18). Participants learned more new kaleidoscopic image-color associations in the static color location condition compared to participants in the dynamic color location condition (static mean [SD]: 30.8 [8.7]; mean dynamic [SD]: 22.5 [7.3]) (t₄₂ = 3.38, p = 0.001). Reaction time decreased across memory strength for both groups (main effect of memory strength; F_5,210 = 11.7, p < 0.0001), however the groups did not differ significantly from each other (group; F_1,42 = 2.2, p = 0.15) nor did we observe a significant interaction (F_5,210 = 1.8, p = 0.11).

fMRI analyses

Memory strength index – Hippocampus (Experiment 1)

By grouping trials according to memory strength (probability correct), we can compare activity at a similar level of performance regardless of variations in learning rate across our conditions. In the left hippocampus we observed a significant linear trend across memory strength in the 100%-valid feedback (F_1,115 = 92.3, p < 0.0001, η_p² = 0.44) and the 80%-valid feedback (F_1,115 = 5.15, p = 0.02, η_p² = 0.04) groups, as well as a significant linear trend by group interaction (F_1,230 = 26.08, p < 0.0001, η_p² = 0.10; Figure 2, Top, Left). We observed a similar significant linear trend across memory strength in the right hippocampus in the 100%-valid feedback group (F_1,115 = 39.05, p < 0.0001, η_p² = 0.25) but not the 80%-valid feedback group (F_1,115 = 0.15, p = 0.70, η_p² = 0.001), and a significant group by linear trend interaction (F_1,230 = 22.13, p < 0.0001, η_p² = 0.08; Figure 2, Top, Right).

Figure 2.

In the bilateral hippocampus we observed a significant linear trend across memory strength in the 100% valid feedback group and in the left hippocampus of the 80% valid feedback group. We also observed a significant group by linear interaction in the bilateral hippocampus (TOP). In the bilateral anterior caudate the 80% valid feedback group exhibited significant quadratic change in activity across memory strength which was only observed in the right anterior caudate of the 100% valid feedback group. Bilaterally in the anterior caudate we observed a significant group by quadratic trend interaction (BOTTOM). Error bars represent ± SEM. L = left; R = right.

Memory strength index – Anterior Caudate (Experiment 1)

The left anterior caudate did not exhibit a significant quadratic trend across memory strength in the 100%-valid feedback group (F_1,115 = 0.93, p = 0.33, η_p² = 0.009), while we observed a significant quadratic trend across memory strength in the 80%-valid feedback group (F_1,115 = 8.62, p = 0.004, η_p² = 0.06) and a significant group by quadratic trend interaction (F_1,230 = 8.37, p = 0.004, η_p² = 0.03; Figure 2, Bottom, Left). In the right anterior caudate we observed a significant quadratic trend in both the 100%-valid feedback (F_1,115 = 3.98, p = 0.04, η_p² = 0.03) and the 80%-valid feedback (F_1,115 = 12.44, p = 0.0006, η_p² = 0.09) groups as well as a significant group by quadratic trend interaction (F_1,230 = 6.86, p = 0.009, η_p² = 0.02; Figure 2, Bottom, Right) across memory strength.

Memory strength index – Hippocampus (Experiment 2)

We observed a significant linear trend across memory strength in both the static color location (F_1,105 = 36.45, p < 0.0001, η_p² = 0.25) and dynamic color location (F_1,105 = 33.51, p < 0.0001, η_p² = 0.24) groups and no significant group by linear trend interaction (F_1,210 = 0.06, p = 0.80, η_p² = 0.0002) in the left hippocampus (Figure 3, Top, Left). We identified a similar linear trend across memory strength in the right hippocampus in the static color location (F_1,105 = 4.87, p = 0.02, η_p² = 0.04) and dynamic color location (F_1,105 = 17.91, p < 0.0001, η_p² = 0.14) groups and no group by linear trend interaction (F_1,210 = 0.61, p = 0.43, η_p² = 0.001; Figure 3, Top, Right).

Figure 3.

In the bilateral hippocampus we observed a significant linear trend across memory strength in both the static and dynamic color location groups. We did not observe a significant group by linear interaction in the bilateral hippocampus (TOP). In the bilateral putamen the static color location group exhibited significant quadratic change in activity across memory strength, which was only observed in the right putamen of the dynamic color location group. In the left putamen we observed a significant group by quadratic trend interaction (BOTTOM). Error bars represent ± SEM. L = left; R = right.

Memory strength index – Putamen (Experiment 2)

In the left putamen the static color location group exhibited a quadratic trend across memory strength (F_1,105 = 16.57, p < 0.0001, η_p² = 0.13), which was not significant in the dynamic color location group (F_1,105 = 3.03, p = 0.08, η_p² = 0.02; Figure 2, Bottom, Left). In the same region we observed a significant group by quadratic trend interaction (F_1,210 = 4.42, p = 0.03, η_p² = 0.02). In the right putamen however, both the static (F_1,105 = 7.5, p = 0.007, η_p² = 0.06) and dynamic (F_1,105 = 5.02, p = 0.02, η_p² = 0.04) groups exhibited significant quadratic trends across memory strength and no significant group by quadratic trend interaction (F_1,210 = 0.73, p = 0.39, η_p² = 0.003; Figure 3, Bottom, Right).

Psychophysiological Interaction (PPI) Analyses (Experiments 1 & 2)

We investigated how the functional connectivity between the hippocampus and the striatum varied as a function of context (memory strength). We used a repeated measures ANOVA to identify voxels where Fisher's z-transformed correlation coefficients changed in either a linear, quadratic, or cubic fashion. Participants in both the 100%-valid feedback (Figure 4A) and static color location (Figure 4B) conditions exhibited context dependent interactions in the left hippocampus when using the left ventral striatum as the seed. Note that here, we present individual memory strength PPI terms for demonstration and visualization purposes only (Figure 4, right). No subsequent statistical analyses were performed on these terms. Rather, statistical analyses were performed on the linear through cubic interaction terms directly (see Methods).

Figure 4.

Psychophysiological interaction (PPI) analysis showing how the correlation between activity in the left nucleus accumbens and the hippocampus in Experiment 1 (A) and Experiment 2 (B) changed as a function of how well associations were learned (i.e., memory strength and reference trials). A. BOLD fMRI activity in the left nucleus accumbens increased its functional coupling with activity in the left hippocampus across learning (top; red bars: Str1-Str5) but this functional correlation dropped off for well learned associations (blue bar: Ref) only in participants who received deterministic (100% valid) feedback. B. Similarly, in participants who learned static color location associations during Experiment 2, activity in the left nucleus accumbens increased its functional correlations with the left hippocampus during learning (bottom; red bars: Str1-Str5) but the functional coupling between the regions decreased for well learned stimulus color associations. Bar graphs to the right are used for display purposes only no further statistics were performed. Insets coronal view. Error bars represent ± SEM.

In both groups regions within the left hippocampus exhibited a gradual increase in functional coupling during learning (Str1-5) that dropped sharply when the associations were well learned (Ref) when using the left ventral striatum as the seed. We did not observe significant context dependent correlations in the 80%-valid feedback or dynamic color location groups nor with any of the other seed regions (e.g., anterior caudate, hippocampus, or right hemisphere ventral striatum) following corrections for multiple comparisons.

Discussion

We found evidence for unique functional contributions from the hippocampus and subregions of the striatum during a multi-trial conditional visuomotor associative learning task. When the ability to rapidly learn new associations was degraded by stochastic feedback (Experiment 1) we observed group differences in the bilateral hippocampus and anterior dorsal caudate. When the consistency of stimulus-response mappings was removed (Experiment 2) we observed group differences in the left sensorimotor putamen. We also observed changes in functional coupling between the hippocampus and ventral striatum during learning but not when the associations were well learned in both experiments. Thus, the present findings suggest that the hippocampus and dorsal striatum contribute to arbitrary associative learning and memory through computationally unique processes while the hippocampus and ventral striatum interact over the course of learning.

Multiple learning and memory systems are thought to support learning in the face of computationally incompatible goals (Sherry and Schacter, 1987; McClelland, McNaughton, and O'Reilly, 1995; Norman and O'Reilly, 2003). Studies both in patients (Knowlton et al., 1996; Shohamy et al., 2004; for review see Squire, 2004) and using functional neuroimaging (Poldrack et al., 1999; 2001; for review see Poldrack and Rodriguez, 2004) have shown that probabilistic category learning versus rapid paired-associate learning differentially rely on the basal ganglia and medial temporal lobes respectively. Prior studies using separate tasks have posited that gradual interleaved versus rapid flexible learning are dependent on unique learning and memory systems. In the present study, variations of the same task taxed the compatibility of learning conditions by either introducing stochastic feedback (Experiment 1) or the consistency of response-mappings (Experiment 2) and observed learning related differences in the hippocampus and subregions of the striatum.

Lesion studies in non-human primates have shown that both the hippocampus (Murray and Wise, 1996) and basal ganglia circuit (Canavan et al., 1989; Nixon et al., 2004) are critical for the acquisition of new and successful recall of well-learned conditional visuomotor associations respectively. These studies typically used well-trained animals with deterministic reinforcement learning schedules similar to participants who received 100%-valid feedback in the present study. Our results suggest that the use of ratio and interval schedules in monkeys may make learning less dependent on the hippocampal learning and memory system and more dependent on anterior and posterior striatal regions respectively (Dickinson et al., 1983).

The functional dissociation observed between BOLD fMRI activity in the hippocampus and anterior caudate is consistent with prior neurophysiological investigations that have shown that activity in the hippocampus and adjacent cortices correlate with the acquisition and retention of arbitrary associations when provided consistent feedback (Wirth et al., 2003; Law et al., 2005; Yanike et al., 2009) but fail to correlate with behavior under conditions of inconsistent feedback such as during probabilistic categorization learning (Poldrack et al., 2001). In contrast, both neurophysiological (Poldrack et al., 2001; Seger and Cincotta, 2005) as well as neuropsychological (Knowlton et al., 1996; Shohamy et al., 2004; 2008) studies have shown that the dorsal striatum is a critical component during tasks that require interleaved learning. Together, these results support the claim that activity in the hippocampus and adjacent cortices correlate with the learning and memory of arbitrary information that is disrupted when stochastic feedback is introduced, while activity in the anterior dorsal caudate correlates with the acquisition of arbitrary associations learned through an interleaved process required when stochastic feedback is provided.

The findings observed in participants who received deterministic feedback – activity in the hippocampus increased in a roughly linear fashion across memory strength and reference trials in participants who received consistent feedback – replicate prior work in our laboratory using the same task (Law et al., 2005; Kirwan and Stark, 2007) and work in nonhuman primates (Wirth et al., 2003). In participants who received probabilistic feedback the anterior dorsal caudate exhibited increases in activity during learning that subsequently declined when associations were well learned. This pattern of activity is consistent with single unit activity in the monkey caudate that correlated with the learning rate (Williams and Eskandar, 2006) and reward prediction error mechanisms, wherein the difference between expected and received reward (i.e., error term) is greatest during learning and subsequently declines once associations are well learned (Pagnoni et al., 2002; McClure et al. 2003; O'Doherty et al., 2003; 2004). Thus we suggest that BOLD fMRI activity in the hippocampus and anterior dorsal striatum reflect computationally distinct learning and memory systems.

Activity in the putamen failed to correlate with behavior when participants learned arbitrary associations under conditions where motor response mappings randomly varied on every trial, and thus there were no learnable stimulus-response associations. Participants who learned associations under conditions with consistent motor responses, on the other hand, exhibited an increase in activity during learning that subsequently declined when associations were well learned. This pattern of activity is similar to that observed during the first experiment in the anterior caudate and in prior studies by our laboratory investigating the contribution of the striatum to conditional associative learning and memory (Mattfeld and Stark, 2010). Lesions studies in rats (Yin and Knowlton, 2007) and monkeys (Nixon et al., 2004) have previously attempted to assess the representational specificity of different regions of the striatum. These studies identified a medio-lateral gradient in the rat striatum and a rostro-caudal gradient in the monkey striatum. Lesions or reversible inactivations along these axes differentially affected the acquisition of new (medial or rostral striatum) versus well-learned associations (lateral or caudal striatum). Additionally, our observed results in the left putamen are consistent with the overall cortico-striato-thalamo-cortical anatomical connectivity of the putamen (Alexander et al., 1986) posited to be a sensorimotor region of the striatum important for the learning of stimulus-response associations. In contrast to the results in the putamen, during both experimental conditions – static and dynamic – activity in the hippocampus increased in a linear fashion across memory strength and reference trials. These results replicate and extend prior findings in our laboratory (Law et al., 2005; Kirwan et al., 2007) and are consistent with the hypothesized functional role of the hippocampus in the rapid flexible learning and memory of arbitrary (e.g., kaleidoscopic-color associations) information. It should be noted however, that hippocampal activation during the static learning condition was more step-like. While a linear relationship will fit this pattern better than a quadratic function, neither may be accurate in evaluating the relevant functional contribution suggesting a more complicated relationship between learning and activation patterns in the hippocampus during this particular task.

To assess the functional coupling across learning between the ventral striatum and hippocampus we utilized a PPI analysis. In the two task conditions that did not preferentially tax the separate learning and memory systems – deterministic feedback (Experiment 1) and static color location conditions (Experiment2) – we identified an increase in coupling between the ventral striatum and hippocampus during the height of learning, when associations are likely being formed, that subsequently reduced when associations were well learned. These findings are similar to our previous study that observed an increase in functional coupling between the ventral striatum and medial temporal lobe during learning, followed by a decrease in functional correlations for well learned associations (Mattfeld and Stark, 2010).

Our results are consistent with the hypothesis that these brain regions interact to dynamically gate information from the hippocampus to the ventral striatum and eventually the ventral pallidum inducing dopaminergic mediated plasticity in target regions (O'Donnell and Grace, 1995; Goto and Grace, 2008; Lisman and Grace, 2005) which is no longer required when associations are well-learned. For example, the timing of the observed increases and decreases in correlations between the ventral striatum and hippocampus are consistent with theoretical and empirical work showing that learning is related to reward prediction errors when outcomes violate our expectations (Steinberg et al., 2013). However, it is believed that learning subsides when an organism can accurately predict outcomes in their environment leading to a reduction in reward prediction errors (Tobler et al., 2006). These results are also validated by the fact that the nucleus accumbens and hippocampus share direct anatomical projections (Groenewegen et al., 1987). Moreover, lesion studies in rats have also shown that the nucleus accumbens and hippocampus are functionally homologous, where lesions of the nucleus accumbens induce spatial learning deficits similar to those observed following hippocampal lesions (Sutherland et al., 1989; Ploeger et al., 1994; Seamans et al., 1994; Ferretti et al., 2010). Thus, the findings that these regions change their functional coupling during the acquisition of associations, suggests that the hippocampus and ventral striatum form a functional circuit during the learning of new information.

Summary

Prior neuroimaging research of arbitrary associative learning and memory has focused on evaluating differential brain activity and functional interactions under single task conditions (Toni et al., 2001; Law et al., 2005; Haruno and Kawato, 2006; Brovelli et al., 2008). Here, we examined both the unique and overlapping brain activity and the functional coupling between regions under experimental conditions that manipulated arbitrary associative learning and memory and consistent stimulus-response learning within the same paradigm. Activity in both the hippocampus and striatum correlated with the learning of arbitrary associations. Specifically, the hippocampus and anterior dorsal striatum contributed uniquely towards the same goal. Activity in the putamen, on the other hand, preferentially represents stimulus-response associations during conditional visuomotor associative learning. These findings suggest that both the hippocampus and subregions of the striatum contribute to the learning of arbitrary associations via computationally distinct and informationally specific roles and their functional coupling is an important feature during the learning but not the maintenance of conditional visuomotor associations.