Striatal and Medial Temporal Lobe Functional Interactions during Visuomotor Associative Learning

Abstract

A network of regions including the medial temporal lobe (MTL) and the striatum are integral to visuomotor associative learning. Here, we evaluated the contributions of the structures of the striatum and the MTL, as well as their interactions during an arbitrary associative learning task. We hypothesized that activity in the striatum would correlate with the rate of learning, while activity in the MTL would track how well associations were learned. Further, we expected functional correlations to show both facilitative as well as competitive relationships depending on the regions involved. Results showed that activity throughout the striatum was modulated by the rate of learning, while the sensorimotor and ventral striatum were also modulated by probability correct. Across the MTL, activity correlated with the probability of being correct, while the perirhinal cortex and right parahippocampal cortex were modulated by the rate of learning. The activity in the ventral striatum robustly coupled with activity in the MTL during learning, while interactions between the associative striatum and the MTL showed the opposite pattern. These findings suggest dissociable computational roles for different subregions of the striatum and MTL. These subregions interact in distinct ways, perhaps forming functionally integrated networks during the learning of arbitrary associations.

Introduction

Conditional associative learning is a form of learning that requires participants to associate specific actions to arbitrary cues in the environment. Successful performance in conditional associative learning tasks requires not only stimulus discrimination but also response selection and the storage and monitoring of previously selected responses and their representative outcomes (Hadj-Bouziane et al. 2006). Thus, conditional associative learning provides an ideal task to assess the interactions among multiple learning and memory mechanisms. Here, we used functional magnetic resonance imaging (fMRI) to record neural activity while participants learned through trial-and-error conditional visuomotor associations.

Electrophysiological and neuropsychological studies in nonhuman primates have identified regions that support both the acquisition and the retention of conditional visuomotor associations, including the medial temporal lobe (MTL) (Murray and Wise 1996; Wise and Murray 1999, 2000; Brasted et al. 2003; Wirth et al. 2003; Yanike et al. 2009), premotor cortex (Halsband and Passingham 1985; Mitz et al. 1991; Brasted and Wise 2004; Buch et al. 2006), prefrontal cortex (Murray et al. 2000; Wang et al. 2000; Bussey et al. 2001; Pasupathy and Miller 2005; Histed et al. 2009), and the striatum (Canavan et al. 1989; Hadj-Bouziane and Boussaoud 2003; Hadj-Bouziane et al. 2003, 2006; Brasted and Wise 2004; Nixon et al. 2004; Pasupathy and Miller 2005; Buch et al. 2006; Williams and Eskandar 2006; Histed et al. 2009). fMRI studies utilizing arbitrary associative learning tasks have reported changes in cerebral blood flow across a similar network of regions (Toni et al. 2001; Eliassen et al. 2003; Boettiger and D'Esposito 2005; Law et al. 2005; Grol et al. 2006; Hanakawa et al. 2006; Haruno and Kawato 2006; Brovelli et al. 2008).

Many of the regions identified in these studies were posited to support learning via different computational goals. For example, during learning over multiple trials, regions across the MTL track how well an association is learned (Toni et al. 2001; Law et al. 2005), while activity in the striatum is largely modulated by prediction error (Haruno and Kawato 2006; Brovelli et al. 2008). However, learning-related changes in behavior are likely the product of an interaction among many regions rather than isolated processes at local neuronal populations (McIntosh 2000). Attempts to elucidate the underlying computational principles of individual brain regions with assessments of the functional interactions across regions may facilitate our understanding of how regionally specific computational principles influence broader network interactions.

Few neuroimaging studies have directly compared how the striatum and MTL interact and how their interactions change during associative learning. In one notable study of striatal interactions, Toni et al. (2002) examined corticocortical and corticostriatal effective connectivity using a conditional visuomotor associative learning task and structural equation modeling. However, their model did not include structures in the MTL, leaving open questions about the nature of interactions between these regions during associative learning.

Here, we evaluated MTL and striatal contributions to an associative learning task. We hypothesized that activity in MTL regions would predominately track how well associations were learned, while activity in anterior regions of the striatum would be largely modulated by the rate of learning—a putative reward prediction error measure. Further, we hypothesized a functional dissociation between anterior and posterior striatum where posterior regions of the striatum would correlate with how well associations are learned as the stimulus–response associations are acquired.

In our model-based analyses, we used the rate of learning (slope) rather than a reinforcement learning model–derived prediction error estimate. Our questions concern the relative contributions that distinct brain regions make to probability correct and prediction error processes. Prediction error estimates correlate with the probability of being correct (see Supplementary Table 1) and therefore cannot be modeled within the same general linear model (GLM) without an orthogonalization process (which would distort their shape). Running separate models provides a qualitative description (i.e., which brain regions correlate with probability correct versus which brain regions correlate with prediction error) but not a quantitative one (i.e., what are the relative contributions to probability correct and prediction error for each region). Therefore, it is important to have each estimate modeled as a separate regressor within the same model. Slope is calculated as the difference between the current probability of being correct and the subsequent trial's probability of being correct: S(t) = Pa(t + 1) − Pa(t), providing an error term between the current and subsequent probabilities of being correct. This error term is similar to the delta rule λ − Qa(t) in the Rescorla–Wagner model: Qa(t + 1) = Qa(t) + η[λ − Qa(t)] (Rescorla and Wagner 1972). However, rather than using an arbitrarily set asymptotic value (λ), we substitute the subsequent trial's probability correct estimate into the equation similar to an ideal observer model. Supplementary Figure 1 shows a comparison between 2 stimuli and their slope and reinforcement learning model–derived prediction error estimates. Slope appears to be a smoothed version of prediction error for these representative stimuli; therefore, we believe that learning rate is capturing a similar computational principle as prediction error here. Notably, within trial estimates of prediction errors—which are captured by temporal-difference (TD) learning algorithms (Sutton and Barto 1998)—are a more accurate estimation of dopaminergic neuronal activity (Schultz et al. 1997). However, given the temporal structure of our trials, the prediction error generated by a Rescorla–Wagner rule would not differ from that generated by a TD algorithm; therefore, in our model-based analyses, we used slope as a surrogate for a Rescorla–Wagner rule trial-specific prediction error estimate.

To determine the extent of interactions between the MTL and striatum during the conditional visuomotor task, we performed a psychophysiological interaction (PPI) correlational analysis (Friston et al. 1997). Based on anatomical studies (Groenewegen et al. 1987) and findings supporting a functional coupling (i.e., increase in interaction) between the hippocampus and the nucleus accumbens (NAcc) (Goto and Grace 2008), we hypothesized that a functional correlation between the NAcc and the MTL would be evident.

The following results are from a novel and independent analysis of previous data collected across 2 experiments conducted in our laboratory utilizing the same task. Importantly, the previous studies were motivated by questions concerning the role of the MTL in the learning of arbitrary associations and did not report patterns of activity in the basal ganglia. Here, utilizing a similar analysis, we report activity specific to the striatum (the main input nucleus to the basal ganglia). Further, we introduce model-based analyses and functional connectivity analyses that were not performed in the previous studies to assess the computational mechanisms inherent to each region and how these putatively distinct learning and memory systems interact during the course of learning. The obtained results from the reanalysis constitute a significant and novel advance over the original studies.

Materials and Methods

Participants

Participants from 2 separate studies utilizing the same conditional visuomotor associative learning task were pooled for a total of 31 (18 females) right-handed participants (Law et al. 2005; Kirwan et al. 2007). All participants gave written informed consent before participating. Mean age was 26.7 (range 18–33). Participants were recruited from the Johns Hopkins community and were paid for their participation.

Materials and Experimental Design

The details of the behavioral task were previously published in Law et al. (2005) and Kirwan et al. (2007). Briefly, the stimuli were random computer-generated kaleidoscopic images (Miyashita et al. 1991). A fixation cross was presented for 300 ms at the beginning of each trial. Following the fixation cross, a kaleidoscopic image with 4 superimposed boxes in a horizontal row was presented for 500 ms. A brief delay (700 ms) followed the kaleidoscope presentation, during which the stimulus was removed and only the fixation cross and boxes remained. After the delay, during a 700-ms response window, participants were instructed to respond following a cue (Go!). Participants were informed that each kaleidoscope image was paired with 1 of 4 response buttons, corresponding to 1 of the 4 boxes on the screen. Directly following the participants' response, the selected box was filled with white, indicating which response had been recorded. Lastly, feedback was presented for 800 ms: “yes” (shown in green) if their response was the correct button associated with that stimulus, “no” (shown in red) if they were incorrect, and “?” if they failed to respond during the response window. Participants were instructed to use the feedback to learn through trial-and-error the correct associations for each kaleidoscope image. Each trial lasted 3000 ms. Figure 1A shows a representative trial.

Figure 1.

Sample stimulus and schematic diagram of trial structure for both learning/reference trials (A) and baseline trials (B).

To establish a reference for the fMRI signal and assist in the estimation of the hemodynamic response for all conditions of interest by inducing jitter between trial types, baseline trials were randomly presented during the experiment (Dale and Buckner 1997; Burock et al. 1998). The temporal structure of baseline trials was identical to learning trials. However, in place of the kaleidoscope image, a random visual static pattern was presented. In the study of Law et al. (2005), 1 of the 4 boxes was randomly filled with white at 50% opacity, indicating the target that the participant should select during the response window. In the study of Kirwan et al. (2007), the same basic task was used, but the target on each trial was set to be between 11% and 17% greater opacity than the other 3 non-target boxes, making the task more perceptually challenging. The stimuli were presented until a response was recorded or the trial ended. Baseline trials were intended to minimize mnemonic demand and to provide a stable level of activity across trials (Stark and Squire 2001). In contrasts between trial types of interest, all baseline activity is factored out, however. Figure 1B shows a representative baseline trial.

Prescan Training

Participants were trained on the task using a set of 4 “reference” stimuli, 24–48 h prior to scanning. The associations for each reference stimulus were maintained throughout the experiment. Prescan training consisted of 2 sessions during which 102 trials were presented (72 stimulus presentation trials—18 trials for each of the 4 reference stimuli—and 30 baseline trials). Reference stimuli were used to compare activity for well-learned associations versus associations that were in the process of being learned.

Scanning Session

Scanning runs consisted of 72 associative learning trials, 30 reference trials, and 30 baseline trials (132 total trials per run). Participants completed different numbers of runs ranging between 3 and 6 runs. Two methods were utilized to maximize the number of trials during which participants were actively learning new associations. First, the number of concurrently learned stimuli was tailored to each participant based on prescan training performance. Participants in the study of Law et al. (2005) learned 4, 8, or 12 stimuli concurrently, while participants in the study of Kirwan et al. (2007) learned 4 or 8 stimuli at the same time. Second, each participant's behavioral performance was monitored in real time to determine when a preset criterion was met (6 consecutive correct responses). For example, in the study of Law et al. (2005), if at least one-half of the stimuli being learned met the preset criterion, then a new stimulus set was introduced on the subsequent run. In the study of Kirwan et al. (2007), stimuli were automatically replaced during a run as performance on them improved.

fMRI Imaging Parameters

Imaging data were collected on a Phillips 3.0-T scanner equipped with a sensitivity encoding (SENSE) head coil at the F. M. Kirby Research Center for Functional Brain Imaging at the Kennedy Krieger Institute (Baltimore, MD). SENSE imaging capitalizes on the sensitivity profiles of multiple surface coils, allowing for an under-sampling of k-space with fewer phase-encoding steps while maintaining full field-of-view images that are free of aliasing—thereby, significantly reducing acquisition time and distortion attributable to magnetic susceptibility (Pruessmann et al. 1999). A high-speed echoplanar single-shot pulse sequence with an acquisition matrix size of 80 × 80, an echo time of 30 ms, a flip angle of 70°, a SENSE factor of 2, and an in-plane acquisition resolution of 3 × 3 mm, was used to collect functional echoplanar images. A total of 264 (132 trials per run, 3 s each trial, 2 acquisitions per trial) whole-brain 3D volumes were acquired with a repetition time (TR) of 1.5 s during each run. Functional volumes consisted of 30 triple-oblique axial slices aligned to the principle axis of the hippocampus. To allow for magnetic resonance (MR) signal stabilization, data acquisition began after the fourth image. To facilitate anatomical localization and cross-participant alignment, a standard whole-brain, 3D magnetization-prepared rapid gradient echo (MP-RAGE) scan was acquired (150 oblique axial slices, echoplanar with the fMRI data, 1 × 1 × 1 mm voxels).

Calculation of the Learning Curve and Memory Strength Index

Similar to Wirth et al. (2003), Law et al. (2005), and Kirwan et al. (2007), we utilized a logistic regression algorithm developed by Brown and colleagues (Smith and Brown 2003; Smith et al. 2004) to calculate a learning curve for each stimulus, which was then used to derive an estimate of memory strength. Behavioral performance for each stimulus was converted into a binary response across the multiple trials (1 if the response was correct and 0 if the response was incorrect). The algorithm used the binary performance for each stimulus to calculate the probability of a correct response based on the number of trials using a Gaussian random walk model as the state equation and a Bernoulli model as the observation equation (Wirth et al. 2003; Smith et al. 2004). Figure 2 shows 3 representative learning curves.

Figure 2.

Representative learning curves for 3 stimuli learned by a single participant. Gray dots represent incorrect trials, while black dots represent correct trials. Plots show the trial-specific probability of being correct (black line) bounded by its 95% confidence interval (dashed lines).

The learning curve for each stimulus provided a quantitative measure of how well that stimulus had been learned on a trial-by-trial basis (probability of being correct: range 0–1). In order to determine the correlation between learning and memory and fMRI activity, we used the learning curve for each kaleidoscope image to derive a discrete memory strength index. Memory strength values from 1 to 5 corresponded to estimated probability correct scores of 0–1 in 5 increments of 0.2 each (Str1 to Str5).

Calculation of the Slope of the Learning Curve (i.e., Ideal Observer Prediction Error Estimate)

For each learning stimulus, we calculated the first derivative (slope) of the learning curve. Additionally, as a control analysis, we modeled each participant's performance with a Q-learning algorithm (Watkins 1989), which used a Rescorla–Wagner learning rule (Rescorla and Wagner 1972) to update Q values (see Supplementary materials).

Cross-Participant Alignment

We used a region of interest alignment (ROI-AL) approach developed by our laboratory (e.g., Stark and Okado 2003) to align both the structural and the functional data. To begin, all structural and functional scans are aligned to the Talairach atlas (Talairach and Tournoux 1988) with the functional scans resampled to 2.5 mm isotropic in the process (and blurred by a mild 4 mm full-width at half-maximum [FWHM] Gaussian kernel to reduce any resampling artifacts). This first pass helps remove large spatial shifts between subjects, providing an initial common registration prior to subsequent fine-tuning. The Talairach transformed MP-RAGE (1 mm³) structural images were then used to segment anatomical ROIs for each participant. A total of 14 regions in the MTL and striatum were defined (MTL: bilateral hippocampus, temporopolar, perirhinal, entorhinal, and parahippocampal cortices; striatum: bilateral caudate and putamen). Regions in the parahippocampal cortex were segmented according to the boundaries outlined by Insausti et al. (1998), while striatal regions were based on the landmarks described in the Atlas of the Human Brain (Mai et al. 1997).

A model for the fine-tuned transformation calculations was then constructed by choosing a single participant (number 29) to serve as the initial model for the transformation calculation for all the other participants. The ROI-AL approach uses high dimensionality diffeomorphic techniques (ROI-Demons) (Stark and Okado 2003; Yassa and Stark 2009) to map the transformation between an individual's ROI segmentations and the model's segmentation. ROI-Demons generates a smooth 3D vector field that is used to transform images between coordinate systems. This or related techniques have been used successfully to align across participants the structures of the MTL and the substructures of the hippocampus (Stark and Okado 2003; Law et al. 2005; Miller et al. 2005; Kirwan and Stark 2007; Kirwan et al. 2007; Bakker et al. 2008) and have been extended here to the striatum. After each participant's structural image was aligned to the model, the resulting transformation matrices were applied to align the functional images.

fMRI Data Analysis

We performed 2 univariate analyses to assess how activity changed during learning. First, we performed a trial-based analysis, binning together trials that showed a similar probability of being correct. Our binned analysis examined how blood oxygen level–dependent (BOLD) fMRI changed as a function of how well the associations were learned. This analysis was constrained to the striatum. This is the same type of analysis as our previous work (Law et al. 2005; Kirwan et al. 2007) (see Supplementary Fig. 5 for MTL results). The second analysis was a more direct interrogation of learning-related changes in activity. We utilized parametric regression to identify regions that correlated with trial-specific probability correct (how well the associations are learned) and learning rate (slope) estimates. The 2 styles of analyses yielded comparable changes in activity over the course of associative learning.

Analysis of Functional Neuroimages (AFNI) software (Cox 1996) was used to perform all data analyses. Functional data were coregistered in 3 dimensions and through time to reduce any effects of head motion. Any time periods with more than 3 degrees of rotation or 2 mm of translation in any direction were eliminated (along with the immediately adjacent TRs) from further analysis. Each participant's data were concatenated across all runs for subsequent analysis. We functionally defined ROIs by setting a voxel-wise threshold of P < 0.04 with a connectivity radius of 2.6 mm and a spatial extent threshold of 609 mm³ for the memory strength and parametric analyses and 531 mm³ for the PPI analyses resulting in an overall alpha probability of P < 0.05 as determined by Monte Carlo simulations (AFNI's “AlphaSim” program). In the PPI analysis, our initial hypotheses were concerned with the interaction between memory systems; therefore, we corrected for multiple comparisons using the volume of the non-seed memory system resulting in a smaller spatial extent correction than the memory strength and parametric analyses. Smoothness was established for the AlphaSim correction using AFNI's 3dFWHMx estimating the intrinsic smoothness of the residuals combined with the moderate spatial blurring kernel utilized in the functional ROI analysis (4 mm FWHM).

Binned Memory Strength Analysis

Behavioral design matrices were created for each subject containing regressors for the 5 memory strength indices, reference trials, and the first presentation of each stimulus. Additionally, nuisance regressors coding for drift in the MR signal were included in the design matrix. Baseline trials were not entered into the design matrix, and activity associated with these trial types served as a reference point for all other trials. The design matrices were used to analyze the fMRI data from each participant using a deconvolution approach based on multiple linear regression (3dDeconvolve; http://afni.nimh.nih.gov/pub/dist/doc/manual/Deconvolvem.pdf). The deconvolution technique estimates the hemodynamic response for each vector of interest by creating 11 time-shifted versions of the design matrix, coding for BOLD activity from 0 to 16.5 s after trial onset. The resulting fit coefficients (β coefficients) represent activity versus baseline for each condition of interest at a given time point in each voxel. The sum of the β coefficients over the expected hemodynamic response (∼3 to 12 s after trial onset) was taken as the estimate of the model of the response to each regressor of interest. The summed β values for each participant were used in group-level analyses, utilizing repeated measures analysis of variance (ANOVA) to contrast all memory strengths, first presentation, and reference conditions.

Continuous Auxiliary Behavioral Information (ABI) Analysis

In a separate analysis, we directly modeled learning (probability correct estimates) and learning rate (slope) to evaluate how activity changed over the course of learning. Here, we used the continuous measures of probability correct obtained from the stimulus-specific learning curves and the first derivative of the learning curves (slope), as ABI, to parametrically evaluate changes in BOLD activity with learning. Each participant's trial-by-trial probability correct and slope estimates were convolved with a canonical hemodynamic response function and correlated with each participant's functional data using multiple linear regression. Design matrices for each participant included regressors for the event occurring, the probability of being correct, and the slope estimate, as well as regressors of no interest coding for drift in the MR signal. The resulting β coefficients for the parametrically modulated regressors (probability correct and slope) were used for subsequent repeated measures ANOVA and t-tests at the group level.

Finally, separate control analyses were performed to account for the possibility that the resulting BOLD modulation was in part due to the outcome of each trial (i.e., correct vs. incorrect) or a decrease in reaction time as associations became well learned. A significant modulation due to outcome or reaction time may confound regions that potentially show a modulation to the slope. Therefore, we split trials according to whether or not participants responded correctly or incorrectly and performed a similar parametric regression analysis as outlined above. In the control analysis, 2 additional parametric regressors were added to the model for the correct and incorrect reaction times (see Supplementary materials).

Functional Connectivity Analysis

In its most basic form, functional connectivity analyses attempt to correlate changes in activity over time in one region with changes in activity in a second region (Friston 1994). Such “seed style” analyses can provide evidence for some form of connection between regions (see Supplementary materials for such an analysis on these data). Direction, causality, or modulations in this connectivity cannot be inferred. This last aspect (modulation) can be inferred from a more complex form of functional connectivity analysis known as PPI functional connectivity analysis (Friston et al. 1997).

PPI analyses attempt to account for activity in one region of the brain in terms of an interaction between activity in another region and a psychological context. We defined context as the different stages of learning outlined by our memory strength indices. To perform the PPI analysis, we added 3 regressors to the model from our memory strength analysis (binned style analysis), one regressor representing global activity across the task, another regressor for the time series activity from our seed region, and a third regressor representing the interaction between our learning contexts and the time series from our seed region. Ordinarily, an additional regressor is added to the model to account for the context events (1 if the context is true and 0 otherwise); however, since our original model already contained this regressor (each memory strength index), we made no such addition, so as to avoid the complication of multicollinearity. One participant was dropped from this analysis due to signal drop out during a portion of a run that would specifically harm this style of analysis, leaving a total of 30 participants for this analysis.

In the PPI analysis, we assessed the change in correlation across learning by determining whether or not the correlation between our seed regions and the rest of the brain changed as a linear, a quadratic, or a cubic function of memory strength. To construct our interaction regressor, we isolated a single time series of activity for all events of interest including first presentation through reference trials. We then used the hand-segmented ROIs (used for cross-participant alignment) to calculate a mean time series for each seed region by averaging across all voxels within each ROI. We then deconvolved the seed time series into its underlying neural function prior to calculating the interaction term (Gitelman et al. 2003). This deconvolution step was utilized to accommodate for the temporal lag and other aspects of the hemodynamic response. We then created the interaction term by combining the physiological event (deconvolved seed time series) with an orthogonal set of contrast weights coding for a linear, quadratic, and cubic change in learning. The resulting neural interaction terms were then convolved with a gamma basis function using AFNI's “Waver” program.

We then ran 3 separate models to test whether regions showed an interaction in correlation following a linear, a quadratic, or a cubic change in memory strength. The correlation coefficients for the interaction terms from the separate models were Fisher's z transformed and analyzed at the group level using repeated measures ANOVAs. Group analyses were utilized to identify whether correlations with the seed region changed according to our prescribed functions (linear, quadratic, or cubic).

As a control, the data were analyzed in a more agnostic manner using a sliding “tent” function to define contexts (e.g., the current and adjacent memory strengths vs. all remaining memory strength indices). While this method was less directed, it provides a more readily interpretable visualization of how the correlations between regions changed as a function of learning. This analysis was performed to corroborate the direct analysis. Lastly, we performed a standard seed style functional connectivity analysis to establish the underlying functional interactions between our ROIs. Reliable seed style functional correlations between the ROIs will help with interpretation of the PPI analysis results. See Supplementary materials for methods and results of both the tent style PPI and the standard seed style functional connectivity analyses.

Results

Behavioral Results

Similar to previous analyses (Wirth et al. 2003; Law et al. 2005; Kirwan et al. 2007), we considered an estimate of the onset of learning to be the trial when the lower 95% confidence interval exceeded chance performance (i.e., probability of being correct exceeded 25%). For all participants, the mean number of training trials required to reach the onset of learning was 4.67 (standard deviation [SD], 3.7; range 2–28). Across both experiments, the mean number of new stimuli learned per run was 7.5 (SD, 1.7; range 3–12).

Reaction times for correct trials separated according to each memory strength can be found in Table 1. The reaction time data correspond to the time to make a response following the response window cue. We found a small but reliable effect of memory strength on reaction time (F = 3.59, P = 0.002). A Bonferroni post hoc pairwise test showed that the only pairwise difference that was reliable was the modest 38 ms difference in reaction times between strengths 3 and 5 (t = 3.68, P < 0.05). Given the response window, reaction times need not be clearly related to processing time, however.

Table 1.

Reaction times for correct trials versus trial type

Memory strength	Mean reaction time (SD)
First	264.12 (62.82)
Strength 1 (0 ≤ P[c] ≤ 0.2)	287.12 (66.69)
Strength 2 (0.2 ≤ P[c] ≤ 0.4)	283.21 (48.53)
Strength 3 (0.4 ≤ P[c] ≤ 0.6)	293.67 (42.74)
Strength 4 (0.6 ≤ P[c] ≤ 0.8)	265.07 (54.13)
Strength 5 (0.8 ≤ P[c] ≤ 1.0)	255.06 (51.45)
Reference	271.60 (57.58)

Table 2.

PPI analysis

Seed ROI	ROI	MNI (x, y, z)	Volume (mm3)
Pattern 1
Left NAcc	Left parahippocampal cortex	−22, −34, −17	562
Left NAcc	Right hippocampusa	24, −10, −23	343
Right NAcc	Left entorhinal cortexa	−25, −5, −37	343
Right hippocampus	Right entorhinal cortexa	16, −10, −29	375
Pattern 2
Right caudate	Right hippocampus/perirhinal cortexa	26, −23, −23	328
Left hippocampus	Left caudatea	−6, 8, −1	359
Right hippocampus	Left caudatea	−16, 1, 23	359

^aIdentified using a reduced spatial extent correction.

Imaging Results

Change in Activity with Memory Strength Index (Binned Analysis)

To identify regions in which activity varied over the course of associative learning, we compared activity across all memory strengths, first presentation, and reference conditions using a repeated measures ANOVA. We masked all the reported results by the MTL/striatal model used for alignment as the approach utilized to functionally define our ROIs does not control for Type I errors in larger volumes. The functional ROI analysis revealed several subregions within the striatum. Specifically, within the putamen, there appeared to be 2 distinct ROIs, one anterior to the anterior commissure and another posterior to the anterior commissure. In order to maintain the fidelity of these ROIs when masked by the MTL/striatal model, we augmented the model to differentiate between anterior and posterior putamen, defining the border as 2 mm posterior to the last slice in which the anterior commissure was visible. A similar anterior/posterior distinction was used earlier in nonhuman primates to assess novel learning (anterior) versus the recall of well-learned (posterior) sequences in a sequential hand movement task (Miyachi et al. 1997, 2002). Additionally, previous studies (e.g., O'Doherty et al. 2004) suggest a functional distinction between the dorsal and the ventral striatum; therefore, the model was augmented to define the NAcc via template matching with the Atlas of the Human Brain (Mai et al. 1997).

The pattern of activity observed in the anterior dorsal and ventral striatum was consistent with what we would expect for an area modulated by reward prediction error. Early in training, activity was low and increased until strength 4, after which activity decreased progressively for associations that were well known. Subsequent post hoc trend analyses constrained to the 5 memory strength indices identified significant quadratic and cubic trends in all anterior striatal ROIs (all F > 5, P < 0.05). No significant linear trends were found in the caudate ROIs (all F < 1, P > 0.4). However, both the anterior putamen and the NAcc bilaterally showed a significant linear component (all F > 18, P < 0.05).

Activity found in the left posterior putamen rose monotonically, tracking how well associations were learned in a pattern similar to what was observed in the MTL ROIs (see Supplementary Fig. 5). This activity showed a linear increase in activity from memory strength index 1 to 5 (linear trend analysis: all F > 23, P < 0.0001), with only a trend but no reliable indications of higher order components (all F < 3.14, P > 0.07) (Fig. 3).

Figure 3.

Regions of striatum (A, coronal; B, axial) where activity varied over the course of learning. Activity in the anterior striatum tracked the rate of learning (a putative prediction error signal): left caudate (red), right caudate (dark blue), left anterior striatum (yellow), right anterior striatum (green), left NAcc (purple), and right NAcc (dark brown). Activity in the posterior striatum tracked the amount of information learned similar to MTL ROIs: left posterior putamen (orange). Regions in the MTL (pale yellow) also correlated with learning; these results are reported in the supplementary materials (Supplementary Fig. 5). Parameter estimates for the striatal ROIs are plotted above. Error bars represent the standard error of the mean. Colors represent ROI labels derived from group-level analysis. L, left; R, right; Caud, Caudate; Ant Put, anterior putamen; Post Put, posterior putamen.

Change in Activity with Probability Correct and Slope Estimates

While the binned analysis is an unbiased method of examining activity changes over the course of learning over multiple trials, it does not directly test the hypothesis that activity in a voxel is correlated with behavioral measures of learning. To test this directly, we used ABI measures derived from each participant's performance to parametrically regress fMRI activity with both the probability of being correct and the first derivative of the learning curve (slope/learning rate). We utilized slope rather than the prediction error estimates derived from our Q-learning algorithm to include both the probability correct and the learning rate estimates within the same GLM (see Supplementary materials). Slope is orthogonal to the probability correct estimates. Therefore, a regressor accounting for slope could be included in the same model with the probability correct estimate without affecting the estimate of probability correct (prediction error and probability correct are correlated, see Supplementary Table 1). At the same time, slope of learning captures many of the same features as prediction error (Supplementary Fig. 1). Additionally, in our control analyses, we observed greater activity for correct versus incorrect trials across the striatum. Further splitting the parametrically modulated regressors by correct and incorrect trials showed no reliable difference for the probability correct and slope estimates between correct and incorrect trials; therefore, the following reported results collapsed across outcome (see Supplementary materials).

Probability correct and slope parameter estimates were taken to a group-level analysis. We used a repeated measures ANOVA to identify voxels where BOLD fMRI activity was modulated by the event occurring, probability of being correct, or slope of the learning curve. Similar to the binned analysis, we excluded voxels that did not fall within our alignment mask. In post hoc analyses using Bonferroni corrections for multiple comparisons, we found that bilateral hippocampal, entorhinal, and left parahippocampal ROIs were reliably modulated by the probability of being correct (all t > 3.31, P < 0.003), while the bilateral perirhinal and right parahippocampal cortices showed significant BOLD activity for slope (t = 3.46, P < 0.003) (Fig. 4A). Activity in the right perirhinal and parahippocampal cortices were modulated by the probability of being correct, but this result did not survive corrections for multiple comparisons. Conversely, the slope modulated activity in ROIs within the striatum (all t > 3.87, P < 0.003). However, in addition to a reliable modulation by the slope, the bilateral NAcc, posterior putamen, and the left anterior putamen showed significant modulation by the probability correct regressor (all t > 3.7, P < 0.003) (Fig. 4B). This analysis was not performed in our prior work.

Figure 4.

Regions of the MTL (A) and the striatum (B) where activity varied as a function of how well the associations were known (P [probability of being correct]) versus slope. ROIs within the MTL were largely modulated by the probability of being correct (dark gray bar). However, the bilateral PRC and right PHC were significantly modulated by slope (white bar). ROIs throughout the striatum were mostly modulated by slope. However, bilateral NAcc and posterior putamen showed significant modulation by the probability of being correct. Bar graphs represent the mean parameter estimate for each ROI. Error bars represent the standard error of the mean. A Bonferroni correction was used for all post hoc analyses. *P < 0.05, **P < 0.003; L, left; R, right; PRC, perirhinal cortex; ERC, entorhinal cortex; HPC, hippocampus; PHC, parahippocampal cortex; Caud, caudate; Ant Put, anterior putamen; Post Put, posterior putamen.

Here, we observed robust modulation across the MTL by the probability of being correct (a direct measure of associative learning). Notably, bilateral perirhinal and left parahippocampal cortices were largely modulated by slope rather than the probability of being correct. Dorsal anterior regions of the striatum were consistently modulated by slope, while bilateral NAcc, posterior putamen, and left anterior putamen were significantly modulated by both slope and probability correct estimates.

Functional Connectivity Results

We used a PPI analysis to evaluate how correlations between the MTL and striatum changed as a function of learning. We used a repeated measures ANOVA to identify voxels where Fisher's z-transformed correlation coefficients changed in a linear, quadratic, or cubic fashion. The cubic change in correlation with learning was not reliably significant in any of the regions identified.

We observed an increase in correlation between the NAcc and MTL during the height of learning, suggesting that the NAcc and the MTL form a functionally integrated network that is recruited during the learning of arbitrary associations. Specifically, the left NAcc seed showed a reliable context-dependent interaction with the left parahippocampal cortex. The correlation between the left NAcc and parahippocampal cortex showed a significant positive quadratic and negative linear change with learning. It is important to note that in a PPI analysis, the sign of the PPI value represents an increase (positive) or decrease (negative) in functional coupling. Therefore, a negative PPI value does not represent an anticorrelation. The change in correlation between the NAcc and the MTL ROIs demonstrated little to no coupling early in learning, increased functional coupling during the height of learning (near Str3), and a subsequent decrease in coupling as associations were learned (Str5 and reference trials). Figure 5A shows the left NAcc seed and parahippocampal cortex ROI for the first pattern of context-dependent change in correlation. See Supplementary materials for tent style analysis.

Figure 5.

PPI analysis showing how the correlation between seed regions and the rest of the brain changed as a function of memory strength. ROIs were defined by assessing change in correlation as a linear, quadratic, or cubic function of memory strength. Two patterns of correlations were observed. The first pattern of functional coupling was characterized by a robust quadratic interaction wherein the identified regions increased in correlation during the height of learning followed by a decrease in coupling as associations became well learned. Representative ROI shown L PHC with L NAcc (seed) (A). The second pattern of activity was identified at a reduced spatial extent correction showed a significant negative quadratic interaction characterized by a decrease in functional coupling during the height of learning followed by a subsequent increase as associations were learned. Representative ROI shown L caudate (Caud) with L hippocampus (Hipp) seed (seed) (B); Bar graphs represent the mean PPI interaction term Fisher's z-transformed correlation coefficients. Error bars represent the standard error of the mean. A Bonferroni correction was used for all post hoc analyses. *P < 0.016. L, left; R, right; PHC, parahippocampal cortex; ERC, entorhinal cortex.

Many of the functionally defined ROIs appeared punctate and did not survive the spatial extent corrections required for controlling Type I error. To assess activity in smaller ROIs, we used the same voxel-wise peak threshold as our previous analyses (P < 0.04) but reduced the spatial extent correction from 531 (34 voxels) to 328 mm³ (21 voxels). Interpretation of these results should be measured given that these ROIs did not sufficiently correct for familywise error.

Using the reduced spatial extent correction, we observed regions showing a negative linear and positive quadratic functional coupling with learning (the left NAcc seed and the right hippocampus and the right hippocampal seed and the right entorhinal cortex). This pattern was similar to the pattern of correlation that was observed between the left NAcc and left parahippocampal cortex. Activity in the right NAcc seed showed a negative linear trend with the left entorhinal cortex—activity in the right NAcc was positively correlated early in learning and over the course of learning the functional coupling between these regions decreased.

A second pattern of functional connectivity emerged between the associative striatum and MTL using the less stringent spatial extent correction. The left hippocampal seed showed a change in its correlation with the left caudate as a function of learning. Early in learning, these regions showed little to no coupling; however, during memory strength 3, these regions significantly decreased their functional coupling and then increased their interaction with subsequent learning. A similar relationship was identified between a region that spanned the right perirhinal and hippocampal cortices and the right caudate when the right caudate was the seed. The right hippocampal seed showed a significant decoupling with the left caudate early in learning and subsequently increased functional coupling as associations became well learned. Figure 5B shows a representative ROI (left caudate and left hippocampal seed) for the second pattern of context-dependent correlation identified. See Table 2 for a summary of the PPI analysis results at both the stringent and relaxed statistical thresholds.

Discussion

We observed modulations in the BOLD fMRI signal across the MTL and the striatum during the acquisition of arbitrary associations. Activity in the bilateral caudate, anterior putamen, and NAcc showed a peak in activity when associations were being acquired (Str3 and Str4), which subsequently declined as associations became well learned (Str5 and Ref). The left posterior putamen, in contrast, showed a monotonic increase in activity with learning similar to MTL ROIs.

The parametric analysis revealed that the bilateral caudate and anterior putamen were largely modulated by the slope of the learning curve while the bilateral NAcc and posterior putamen showed a significant modulation by both slope and the probability of being correct. In the MTL, the hippocampus, entorhinal cortex, and left parahippocampal cortex showed a robust modulation to the probability of being correct, while the slope reliably modulated activity in the perirhinal and right parahippocampal cortices. These data add to our previous analyses of the MTL (Law et al. 2005) showing a monotonic increase in activity with learning across the hippocampus and entorhinal cortex while the perirhinal and parahippocampal cortices showed evidence for a more curvilinear change in activity with learning—which may be accounted for by the significant modulation by the slope here.

Our results in the MTL based on the parametric analysis correspond with previous work in nonhuman primates and human fMRI studies that have identified the engagement of structures in the MTL during the learning of arbitrary associations (nonhuman primate studies: Murray and Wise 1996; Wise and Murray 1999, 2000; Brasted et al. 2003; Wirth et al. 2003; Yanike et al. 2009; human study: Toni et al. 2001). The MTL appears to track how well the associations were learned (i.e., reliable modulation by the probability of being correct). Importantly, this analysis also revealed several regions within the MTL (bilateral perirhinal and right parahippocampal cortices) that showed significant modulation by the rate of learning. These regions, while increasing their activity in a roughly monotonic fashion, showed a much steeper rise in activity with a plateau in activity as associations were well learned. The reliable modulation by slope here provides a potential computational mechanism for the distinct curvilinear changes in learning-related activity observed in the perirhinal and parahippocampal cortices (see Supplementary Fig. 5). While the differential modulation within the MTL by probability correct and slope suggests that different parts of the MTL are contributing to learning and memory in unique ways, we believe that this strong conclusion requires more direct investigation. The seeming divergence between the binned memory strength analysis and the model-based analysis may be the result of nonlinearities in the BOLD effect itself. Future electrophysiological studies investigating changes in spike rate and or local field potential with measures such as learning rate will be integral to this end. However, the data presented here suggest that the original “memory systems” framework (e.g., Squire et al. 2004) may be due for some degree of refinement.

While this framework has been influential, accumulating evidence suggests that regions once thought to be dedicated to specific types of learning and memory (e.g., the basal ganglia and habit learning) may in fact be participating in multiple forms of learning and memory (e.g., Yin and Knowlton 2004). Perhaps, given the results reported here (i.e., bilateral perirhinal and right parahippocampal cortices), similar qualifications should be extended to the MTL. Lesion studies provide information regarding the necessity of a region for a particular task but cannot address whether or not a region may be engaged under similar conditions. Likewise, while such studies have demonstrated that declarative and nondeclarative memory can be independent, it does not follow that they always must be mutually exclusive. Nonetheless, fMRI studies are beginning to probe the computational processes that may be supporting different forms of learning and memory (Bakker et al. 2008; O'Doherty et al. 2003). Studies of the processes underlying different forms of learning and memory offer another way of thinking about the basis for multiple memory systems (e.g., McClelland et al. 1995), consistent with the results reported here.

In the anterior regions of the striatum, the activity identified likely corresponds to learning-related changes in activity induced by reward prediction error signals. This interpretation is in line with previous fMRI studies that have identified correlations between reward prediction error and fMRI activity in the dorsal and ventral striatum (Pagnoni et al. 2002; McClure et al. 2003; O'Doherty et al. 2003, 2004; Haruno and Kawato 2006; Rodriguez et al. 2006; Brovelli et al. 2008; Schonberg et al. 2010). The reduced activity across the anterior striatum during Str1 and Str2 may appear inconsistent with a reward prediction error hypothesis—unexpected reward received early in learning should generate the largest errors in prediction. Such an account neglects the fact that Str1 and Str2 contain the largest proportion of errors. These errors lead to the inclusion of a large number of negative prediction errors, which would lead to an overall decrease in BOLD fMRI activity during these memory strength indices. Importantly, our control analyses (see Supplementary materials) suggest that activity across the striatum is not preferentially modulated by error trials with slope equally modulating both correct and incorrect trials. Therefore, we are confident that this pattern is consistent with a prediction error hypothesis. Furthermore, these results are consistent with previous studies that have reported a reduction in BOLD signal for trials that predominately code for negative prediction errors (Tobler et al. 2006).

Activity in the posterior putamen, however, was similar to that observed in MTL structures, wherein this region tracked the amount of information learned. While similarities in the pattern of activity between the striatum and the MTL exist, we believe the nature of what is learned is different (Knowlton et al. 1996; Poldrack and Rodriguez 2004). For example, this region may be participating in the formation of the motoric component of relevant stimulus–response associations. In our own data, the functional connectivity analyses support the notion that the similarities are more superficial. The posterior putamen showed little connectivity with the MTL, unlike other striatal regions (Supplementary Fig. 3). Similar changes in activity in nonhuman primate studies have been identified that also support this conclusion. As monkeys learned novel associations during an arbitrary visuomotor associative learning task, cells in the putamen increased their activity in an approximately linear fashion–tracking behavioral performance (Hadj-Bouziane et al. 2003; Brasted and Wise 2004; Williams and Eskandar 2006). Our results in the posterior putamen coincide with the findings of Haruno and Kawato (2006) who identified regions near the anterior commissure in the putamen that were more correlated with the formation of arbitrary visuomotor associations; however, our ROIs extend more posterior.

Similarities aside, one of the limitations of the current study is that fMRI only provides correlational and not causal support for the striatum and MTL playing both similar and differential roles in learning and memory. Neuropsychological studies have provided compelling evidence, suggesting that the striatum and MTL support distinct representations (McDonald and White 1993; Knowlton et al. 1996; Packard and McGaugh 1996). However, accumulating evidence in animals suggests that these dissociations need to be further refined. For example, lesion studies in rats have identified a medial versus lateral division of labor in the dorsal striatum. The dorsomedial striatum appears to support forms of learning and memory once thought to be the providence of the hippocampus (i.e., “place” learning) while the dorsolateral striatum is integral for procedural forms of learning and memory (i.e., “response” learning) (Devan and White 1999; Yin and Knowlton 2004). The medial/lateral division in the dorsal striatum of the rat roughly maps onto an anterior/posterior divide of the striatum in primates. Miyachi et al. (1997, 2002) have shown that reversible inactivation of the anterior putamen (associative striatum) disrupted the learning of novel sequential hand movements, while inactivation of the posterior putamen (sensorimotor striatum) disrupted well-learned sequences.

These results support the notion that the striatum is participating in multiple forms of learning and memory depending on the subregion involved (Divac et al. 1967; Murray and Wise 2004; for review, see Yin and Knowlton 2006). Anterior regions of the striatum may represent visuospatial information early in learning via reward learning mechanisms (Hikosaka et al. 1999) while posterior regions in the putamen in conjunction with the premotor/motor cortices participate in the gradual development of stimulus–response associations.

In addition to the learning-related changes in activity, we also identified correlations between MTL and striatal regions and found that the correlations between these regions changed during the course of learning. Our PPI analysis identified an increase in coupling with MTL ROIs during the height of learning when the seed was located in the ventral striatum. At a reduced statistical threshold, the associative striatum and hippocampal ROIs showed a decrease in coupling with each other at a similar time point.

Toni et al. (2002) used structural equation modeling to determine the correlation between striatocortical and corticocortical regions in a similar task. Given that their model did not incorporate a link between the striatum and any region in the MTL, they were unable to interpret how the 2 regions interacted. The positive correlation between the NAcc and the MTL regions identified in our study is supported by the fact that there is a strong anatomical link between the MTL and the ventral striatum (Groenewegen et al. 1987). Additionally, lesion and reversible knockout studies in rats have suggested a functional link between these 2 regions (Schacter et al. 1989; Sutherland and Rodriguez 1989; Ploeger et al. 1994; Seamans and Phillips 1994).

The finding that the correlation between these 2 regions increases during the height of learning is consistent with a hippocampal gating hypothesis posited by Goto and Grace (2008). According to their hypothesis, the bistable membrane potential of the medium spiny neurons in the NAcc are driven to an “UP” state by hippocampal afferent activity gating concomitant prefrontal cortical activity through the NAcc on to the ventral pallidum (O′Donnell and Grace 1995; Grace 2000; Lisman and Grace 2005; Goto and Grace 2008). This has been shown to be especially prominent during phasic dopamine bursts facilitating input from the hippocampus through D1 receptor activation (Goto and Grace 2005). We hypothesize that a similar phasic increase in dopaminergic activity coincides with the peak in BOLD activity that we see in the NAcc during Str3—supported by the significant modulation due to the slope estimate. Moreover, it would be adaptive to gate relevant information during the height of learning (between Str2 and Str4) and to decrease this process as associations become well learned.

There is one notable discrepancy between our findings and the gating hypothesis proposed by Goto and Grace (2008): while they posit a functional relationship between the subicular projection from the hippocampus and the ventral striatum, the ROIs that we identified were not limited to the hippocampus but distributed throughout the MTL. However, given the robust projections from the adjacent cortices to the striatum including the NAcc (Witter and Groenewegen 1986), we do not believe that our findings are inconsistent with their model. The functional decoupling observed between the caudate and the MTL ROIs should be interpreted cautiously due to the relaxed spatial extent threshold required to identify voxels; however, the results are consistent with the functional dissociation identified between these regions during probabilistic categorization tasks (Poldrack et al. 2001) extending previous results to a conditional associative learning task.

One potential caveat to keep in mind when interpreting the functional connectivity analyses is the possibility that a third unmodeled region, in this case the midbrain dopaminergic neurons—specifically the ventral tegmental area (VTA), is driving the observed correlation between the ventral striatum and the MTL. Projections from the VTA target both the ventral striatum and the hippocampus and entorhinal cortex among other regions outside of our investigation (Simon et al. 1979; Swanson 1982; Oades and Halliday 1987). Recent fMRI studies suggest that interactions between midbrain dopamine neurons and the hippocampus facilitate scene encoding (Adock et al. 2006), encoding of novel events (Wittman et al. 2007), and the encoding and integration of associations (Shohamy and Wagner 2008).

While the current study cannot completely discount the possibility that midbrain dopaminergic neurons are driving the observed correlations, we believe that there are 2 observations that make this unlikely and support our original conclusion. First, PPI functional connectivity analyses (Friston et al. 1997) investigate the interaction between a seed time series and a psychological construct. This analysis is very underpowered given the fact that the seed time series for each ROIs is used as a regressor of no interest in the model. If the VTAs (the common input to the ventral striatum and hippocampus) were modulating the time series of the Nacc, the variance associated with this modulation would be accounted for by the regressor of no interest mitigating the influence of the VTA on this particular analysis. Second, the most reliable region to survive thresholding in this analysis was the posterior parahippocampal cortex. This region of the MTL lacks any substantial dopaminergic innervation from the VTA (Swanson 1982); therefore, it is unlikely that the VTA could serve as a common input to drive the correlation between the posterior parahippocampal cortex and NAcc; however, the same conclusion cannot be drawn for the entorhinal and hippocampal ROIs. Further, while a functional coupling was observed between the associative striatum (i.e., anterior caudate) and the MTL, these regions receive distinct dopaminergic innervation from the substantia nigra pars compacta and VTA, respectively (Beckstead et al. 1979).

Conclusions

These data show that the MTL and the striatum are participating in the learning of arbitrary associations by tracking the amount of information learned and reward prediction error mechanisms. Intriguingly, the results reported here suggest a more complicated picture than the traditional MTL/striatal dichotomy. While the distinct computational mechanisms largely adhere to the expected notions of MTL and striatal function, distinct subregions within these 2 structures appear to deviate from the traditional functional roles. For example, the posterior putamen and NAcc appear to track how well an association is learned, albeit through computationally distinct principles, while the bilateral perirhinal cortex reliably correlates with slope. This divergence from the traditional dichotomy is consistent with recent work in animals; however, further human fMRI studies will be necessary to delineate the functional role of the posterior putamen signal from the signals observed in the MTL and the perirhinal cortex from the signals observed in the associative striatum (for an elegant study positing a role for the posterior putamen in human habit learning, see Tricomi et al. 2009). Importantly, MTL and striatal ROIs demonstrate a functional coupling and decoupling during the height of learning—consistent with theories positing a functional relationship between the MTL and the NAcc and a functional dissociation between the MTL and the caudate.

Supplementary Material

Supplementary material can be found at http://www.cercor.oxfordjournals.org/.

Funding

National Science Foundation (BCS-0236431, BCS-0544959 to C.E.L.S.).