Feedback signals in the caudate reflect goal achievement on a declarative memory task

Abstract

The striatum has been shown to be a key region in the processing of reward-related information. The head of the caudate nucleus has been implicated in processing performance feedback, or in other words, information about the outcomes of one’s actions. However, feedback provides multiple types of information, and it is not clear which of these types of information drive a caudate response. We sought to determine whether the signal in the caudate differed when feedback was informative but only arbitrarily related to performance versus when it provided information about goal achievement. To do this, we used functional magnetic resonance imaging (fMRI) to examine caudate activation during a feedback-based paired associate word learning task. During an initial round of 60 distinct trials, participants chose one of two responses on each trial and received feedback about whether their responses were correct. On the subsequent two rounds, the 60 trials were repeated and participants chose their responses based on their memory of the correct answer. The caudate nuclei were strongly engaged only during the second two rounds, when feedback reflected the accuracy of memory. These results support the idea that feedback-based caudate activation is context dependent: the caudate can be engaged in feedback-based declarative memory tasks, but it is more strongly engaged when feedback is “earned” by performance than when it is informative but not tied to goal achievement.

The head of the caudate nucleus, which lies in the dorsomedial portion of the striatum, has been implicated in processing reward-related information (Delgado et al., 2000; Elliott et al., 2000b; Knutson et al., 2000), including the processing of performance-related feedback (e.g., (Elliott et al., 1997; Poldrack et al., 2001; Tricomi et al., 2006). However, little is known about the specific role that the caudate plays in feedback-based learning. The goal of this experiment was to examine the parameters that govern feedback-related caudate activation.

Neuropsychological work has implicated the striatum in habit learning; both Parkinson’s disease and Huntington’s disease reflect disruptions in striatal function, and in both diseases performance is impaired on gradually learned tasks in which there is little awareness of what marks the right response (Cools, 2006; Frank et al., 2004; Knowlton et al., 1996; Monchi et al., 2004; Packard and Knowlton, 2002; Seger, 1994; Shohamy et al., 2004). However, these tasks tend to rely on learning from feedback, and recent evidence indicates that the deficits observed may be specifically related to the presence of feedback, rather than to the gradual or nondeclarative nature of habit learning (Shohamy et al., 2004; Smith and McDowall, 2006). The idea that feedback-based learning can be dissociated from habit learning is supported by animal work, which suggests that the dorsomedial striatum is involved in learning response-outcome associations whereas the dorsolateral portion of the striatum supports the acquisition of stimulusresponse habits (Yin and Knowlton, 2006). Therefore, it is possible that the striatum would be involved in feedback-based learning even in tasks that are not typical of habit learning tasks (e.g., feedback-based declarative learning tasks).

The striatum is not usually implicated in declarative learning; instead, other brain regions, including the hippocampus and adjacent cortex within the medial temporal lobe (MTL), and the prefrontal cortex (PFC), have been the focus of research on declarative learning and memory. Although in some circumstances the striatal and hippocampal systems can interact competitively (Poldrack et al., 2001), other work suggests that the two systems can learn in parallel, with the more efficient system generally governing behavior unless it is inactivated or damaged (Bayley et al., 2005; Packard and McGaugh, 1996; Squire, 2004). Further work indicates that the brain’s dopamine-mediated reward system, which is often associated with the striatum’s role in learning, facilitates hippocampus-dependent memory formation (Adcock et al., 2006; Lisman and Grace, 2005; Wittmann et al., 2005).

If the caudate is critically involved in feedback processing, rather than playing a role that is specific to habit learning, then it should be activated during feedback-based declarative learning. We tested this idea by using a feedback-based paired associate word-learning task. Like other declarative memory tasks, this task involved the conscious formation of associations between arbitrary stimuli (Zola and Squire, 2000). Participants encountered the same set of sixty distinct trials three times. The task was set up in a “multiple choice” format, with a target word and two choices of possible words for the second half of the pair (Figure 1). The participants began with no prior knowledge of the correct pairings, which were arbitrary. For each of the sixty trials on the first round, they were required to pick an answer and then feedback was delivered about the accuracy of their guess. The second and third times they encountered the same trial, they were instructed to try to remember the correct answer and respond accordingly, and feedback indicated whether their answer was correct.

Figure 1. Experimental Design.

Each trial, a target word was presented, along with options for possible word matches, labeled as in a multiple-choice test. After a 4 s response period, the display was replaced with a 1 s feedback display of 3 green √s, indicating a correct response, 3 red Xs, indicating an incorrect response, or 3 white hyphens, indicating that no response was made. After a 13 s delay, with a screen showing a fixation cross, the next trial began.

Although the structure of the three rounds of trials was identical, we hypothesized that feedback was serving two fundamentally different roles on the first round of trials compared with the second two rounds. During the first round of trials, feedback provides information about what the correct answer is; importantly, with only two response options, the informational value of the feedback is held constant, because the correct answer for each pair can be determined from either negative or positive feedback. During this round, however, feedback does not provide an indication of task success. Since the sixty word pairs were arbitrary, performance was also arbitrary. Participants were instructed that there was no possible way to know in advance which of the two word pair options went with the target word, but that the pairs would be consistent across the three rounds, so they should learn the correct pairings based on the feedback they would receive. This instruction differentiates this study from other studies involving guessing tasks with monetary rewards and punishments, in which a “gambler’s fallacy” may cause participants to believe that there is some way to perform better than chance when, unbeknownst to the participants, the outcomes are randomly determined (e.g., Delgado et al., 2000; Elliott et al., 2000b; Tricomi et al., 2004). In contrast to the first round of trials, on the second and third rounds of trials in our experiment performance was no longer arbitrary; that is, feedback now indicated the accuracy of the participants’ responses in a deterministic fashion.

Functional magnetic resonance imaging (fMRI) studies have shown that activation in the caudate nucleus is modulated by tasks involving the anticipation and receipt of monetary rewards and punishments (Delgado et al., 2000; Elliott et al., 2000b; Knutson et al., 2000) as well as performance-related feedback (Poldrack et al., 2001; Seger and Cincotta, 2005; Tricomi et al., 2006). There are two temporal components to this modulation. First, when feedback (or some other type of reward or punishment) is expected, there is an initial rise in activation that begins prior to the expected feedback (Delgado et al., 2000; Tricomi et al., 2006). When no feedback is expected, this rise is minimal or absent (Tricomi et al., 2006). Second, once the outcome is revealed, the signal differentiates between positive and negative feedback (or other types of rewards and punishments), by showing more sustained activation following rewards or positive feedback and a sharper decrease in activation (often below baseline levels) following punishments or negative feedback (Delgado et al., 2000; Tricomi et al., 2006).

The magnitude of the caudate signal during anticipation of feedback and the degree to which the signal differentiates between positive and negative outcomes is modulated by a variety of contextual factors (Delgado et al., 2003; Delgado et al., 2004; Nieuwenhuis et al., 2005; Tricomi et al., 2004; Zink et al., 2004). In general, trials that elicit the greatest initial rise in signal from the trial onset also show the greatest differentiation in signal between positive and negative outcomes once they are revealed (Delgado et al., 2003; Delgado et al., 2004; Tricomi et al., 2004). Specifically, recruitment of the head of the caudate during anticipation and processing of feedback may be modulated by the following factors: 1) sense of agency in determining the outcome, 2) the subjective value of the feedback to the individual and 3) outcome uncertainty.

First, evidence that a sense of agency in determining the outcome affects the caudate signal comes from a study involving a guessing task with monetary outcomes, which showed that the caudate was recruited only when participants believed that there was a contingency between their action and whether they subsequently received a reward or punishment (Tricomi et al., 2004). If a learner has no prior knowledge of the task, and therefore no basis on which to choose a response, the caudate might be less strongly activated than when the feedback provides a true assessment of performance. Second, feedback can have different effects on caudate activation, depending on the participants’ goal, indicating that the value of feedback to the individual affects how it is processed in the caudate. In a guessing task, the caudate displayed a larger signal rise at the trial onset and a larger difference between positive and negative outcomes when feedback indicated monetary gain or loss than when there were no monetary incentives (Delgado et al., 2004). However, in a learning study in which feedback indicated performance accuracy but had no monetary value, a robust signal in the caudate was observed in anticipation of the feedback, and the signal also differentiated between trials with positive and negative feedback (Tricomi et al., 2006). In this case, the goal of participants was to improve task performance, rather than to earn money, so positive feedback may have been valued highly because it provided information about achievement of this goal. We hypothesized that in the current task, the caudate would be more strongly engaged during anticipation of feedback and would display greater differentiation between positive and negative feedback the second, as compared to the first, time each participant encountered the item. We reasoned that during Round 1, participants might feel only a weak sense of agency in determining the outcome, since they knew that feedback during this round was not a reflection of good or poor performance; we reasoned that during Rounds 2 and 3, the sense of agency might be stronger, since feedback was then a direct indication of memory accuracy. Additionally, we thought that the perceived value of the feedback in the current experiment might be affected by whether it provided meaningful information about task mastery, which did not occur until the second time the trials were presented.

Finally, once a task is well-learned, feedback becomes completely expected, and therefore ceases to provide information. At this point, positive feedback may become less rewarding. Several studies have found that reward-related activation in the striatum and midbrain is greatest when the reward is unpredictable or uncertain (Aron et al., 2004; Berns et al., 2001; McClure et al., 2003). Further studies have indicated that activation in the caudate nuclei decreases over the course of learning (Delgado et al., 2005; Haruno et al., 2004; Jueptner et al., 1997; Law et al., 2005; Pasupathy and Miller, 2005; Williams and Eskandar, 2006). These findings led to a second hypothesis that caudate activation might be attenuated on the third round of trials, as positive feedback becomes more expected.

MATERIALS AND METHODS

Participants

Twenty healthy, right-handed adults were recruited through posted advertisements and were paid $57 for their participation in the experiment. One was excluded due to a technical problem. The main analysis did not include data from participants who had fewer than 8 trials in any condition, due to movement (4 subjects) or ceiling performance (4 subjects). Data from the remaining eleven subjects were analyzed (5 women, 6 men; mean age ± SD, 21.9 ± 2.3). All participants gave written informed consent according to the Institutional Review Board at the University of Pittsburgh.

Materials

A 3 Tesla Siemens head-only scanner and standard radio frequency coil was used for all the MR scanning sessions. Stimulus presentation and behavioral data acquisition was controlled using “E-prime” software (Schneider et al., 2002) and the integrated function imaging system (IFIS) [Pittsburgh, PA].

Procedure

Scan session

Structural images were collected using a standard T1-weighted pulse sequence, in thirty-eight contiguous slices (3.125 × 3.125 × 3.0 mm voxels) parallel to the AC-PC line. Thirty-eight functional images were collected in the same locations as the structural slices, which in most subjects provided coverage of the entire cerebrum and partial coverage of the cerebellum. Images were acquired using a one-shot echo-planar imaging (EPI) pulse sequence [TR=2000 ms, TE=25 ms, FOV=20 cm, flip angle = 79°].

Behavioral paradigm

This experiment involved a paired associate word learning task, and participants were scanned as they performed three rounds of trials on this task: the initial encoding round, and two subsequent rounds. Participants received performance-dependent feedback following each trial throughout the scan session.

The scan session consisted of nine six-minute runs, divided conceptually into three rounds of 60 trials. Each trial was 18 s long. On each trial of Round 1 (the first three runs), participants saw a target word and two choices of possible word matches, labeled “a)” and “b)” (Figure 1). The words contained 4–8 letters and 1–2 syllables, had Kucera-Francis frequencies of 20–650 words per million, and had high imagibility ratings (score of over 400 according to the MRC database; Coltheart, 1981). The words were matched for word length and frequency at the trial level. Additionally, words presented on the same trial were not semantically related, with a score of less than 0.2 on the Latent Semantic Analysis similarity matrix (Landauer et al., 1998) and they did not rhyme or begin with the same letter. Participants were asked to guess which response word went with the target word by pressing one of two buttons on a response glove. Since the words were unrelated, guesses were arbitrary. Participants had 4 s to respond, after which the display was replaced by a feedback display, which was shown for 1 s (Figure 1). On 50% of trials, participants received positive feedback following their response (three green √s), indicating they guessed correctly, and on the other 50% of trials, participants received negative feedback (three red Xs), indicating they guessed incorrectly. If the subject made no response, three white hyphens were shown and the trial was excluded from analysis. Participants were asked to use the feedback to try to remember the correct word pairs (the correct pairings remained consistent throughout each scanning session, although which item was the correct one was chosen randomly for each participant). Each trial ended with a 13 s delay period in which participants fixated on a white cross in the center of the screen. Participants were not given any constraints or advice on how to remember the word pairs.

On Round 2, the same 60 trials were repeated, in random order, with the position of the response options (i.e., which was choice “a” and which was choice “b”) chosen randomly. Although the procedure was the same, it should be noted that participants’ guesses were no longer arbitrary; instead, participants were asked to pick the correct response based on the feedback they received during Round 1. Feedback was again presented after each trial, indicating whether the participant answered correctly or incorrectly. Finally, on Round 3, the procedure from Round 2 was repeated, with the same sixty trials presented in random order.

Following the scan, participants took a computerized post-test, using the E-prime program, in which the 60 trials were repeated one more time, with no feedback. Following each trial, participants were asked to make a confidence judgment by choosing a number from 1–7 (1 = complete guess, 7 = completely sure). The post-test was self-paced, unlike the trials during the scan. Approximately one week later (7–9 days following the scan session), all but one participant returned to the lab and completed the post-test a second time.

Data Analysis

Behavioral Data

Analyses were performed on the behavioral data of the participants who were used in the main fMRI data analyses. One-sample t-tests were used on the accuracy data from each round and from the post-tests to determine which conditions differed from chance. A repeated measures ANOVA was performed with subject as a random factor and round as a within-subjects factor to determine if accuracy changed over the course of the experiment. A similar ANOVA was performed to assess whether reaction time changed over the course of the scanning session. Two-tailed t-tests were used to determine whether reaction time and/or confidence differed between correct and incorrect trials. Finally, a linear regression of reaction time on confidence was fit for the post-test data to determine whether reaction time differed as a function of confidence.

FMRI Data

The NeuroImaging Software package (NIS 3.5), developed at the University of Pittsburgh and Princeton University, was used to analyze the fMRI data. Images were reconstructed and then were corrected for head motion using a 6-parameter rigid-body automated registration algorithm (AIR 3.08; Woods et al., 1992). Data from runs in which head motion exceeded 4 mm or degrees in any direction were not used for analysis; this excluded four participants from the main analysis. A voxel-wise detrending of the functional images was performed to adjust for linear scanner drift. The skull was stripped from the structural images of each participant, which were then co-registered to a common reference brain (chosen from among the participants; Woods et al., 1993). Functional images were transformed into the same common space, globally mean-normalized by a mean scaling of each image, and smoothed using a three-dimensional Gaussian filter (8 mm FWHM) to account for anatomical variability among participants.

A voxel-wise analysis was performed on the fMRI data, through a repeated-measures three-way ANOVA with subject as a random factor and accuracy (correct vs. incorrect), round (1–3), and time period (2 s time periods T1–T9) as within-subject factors. A voxel-wise significance threshold of p < 0.0001 was used to identify regions of interest (ROIs). In addition, a contiguity threshold was applied as a precaution against type-1 errors (Forman et al., 1995). The AFNI AlphaSim program was used on the F-maps for each statistical contrast to set the contiguity thresholds such that the map-wise probability of a false detection remained lower than 0.05 (Cox, 1996). All ROIs were transformed into Talairach space (Talairach and Tournoux, 1988) using the AFNI program (Cox, 1996).

Further analysis focused on whether levels of activation during Round 1 predicted accuracy on Round 2. To do this, trials were coded according to the accuracy for the identical trial on the following round. A repeated measures ANOVA was then performed with subject as a random factor and current round accuracy (correct vs. incorrect), future accuracy (correct vs. incorrect), and time (2 s time periods T1–T9) as within-subject factors. To increase the power of this analysis, data from participants whose performance was at ceiling on Round 3 were included, as was data from the three participants who were excluded from the main analysis due to excessive movement, since the movement occurred after Round 1; thus, n = 19 for this analysis. A similar analysis was not performed on the data from Rounds 2 and 3, since if performance is accurate on any given trial on Round 2, it is more likely to be accurate subsequently as well, and so effects due to performance and to learning cannot be dissociated on these rounds.

RESULTS

Behavioral Results

The mean accuracy and reaction times for each round in the scanner and the two post-tests are listed in Supplemental Table 1. With the exception of Round 1, which was programmed so that each participant would receive positive feedback 50% of the time, performance was significantly better than chance (p < 0.05), indicating that learning occurred. A repeated measures ANOVA indicated that accuracy changed over the course of the experiment (F(4,39.3) = 56.7, p < 0.05). Reaction time did not differ significantly over the three rounds in the scanner (F(2,20) = 1.57, p > 0.05). Reaction times during the scan session were not compared to post-test reaction times, since the post-tests were self-paced.

Two-tailed t-tests showed no significant differences between reaction times for correct and incorrect trials on Rounds 1 and 2 (t(10) = −0.74 and 0.30, respectively, p >0.05), but did show significant differences on Round 3 (t(10) = −5.06, p < 0.05), the immediate post-test (t(10) = −3.32, p < 0.05) and the 1-week post-test (t(9) = −3.21, p < 0.05; data was absent from one participant who failed to return for the post-test). Confidence was significantly greater for correct trials than incorrect trials on both the immediate post-test (t(10) = 7.6, p < 0.05) and the 1-week post-test (t(9) = 4.8, p <0.05). Finally, increases in confidence judgments were associated with decreases in reaction time; mixed model linear regression analyses of reaction time on confidence, with subject as a random factor, showed this effect to be significant for both post-tests (F(6, 641) = 13.8 and F(6, 581) = 5.5, for post-tests 1 and 2, respectively, p < 0.05).

fMRI Results

A voxel-wise ANOVA with subject as a random factor and round (1–3), accuracy (correct vs. incorrect), and time period (2 s time periods T1–T9) as within-subject factors was performed. The resulting activation clusters showing a Round by Time effect, at a threshold of p < 0.0001 and a contiguity threshold of 12 voxels, are listed in Table 1 and shown in Figure 2. Although there were subtle differences in activation between Rounds 2 and 3, the most salient differences were between Round 1 activation and the activation during the other two rounds.

Table 1.

Regions Displaying a Round X Time Interaction (p<0.0001)

Region of Activation	BA	Size (# voxels)	Peak Talairach Coordinates (x, y, z)	Maximum F Value
Round 1 > Rounds 2 & 3
Medial frontal gyrus (L)	6	61	−11, 1, 58	6.2
Superior frontal gyrus (L) (decreasing activation)	8, 9	44	−21, 48, 32	5.3
Middle frontal gyrus (R)	8, 9	15	32, 24, 35	4.7
Inferior frontal gyrus (L)	44, 45	48	−47, 24, 9	5.2
Inferior parietal lobule (R) (decreasing activation)	40	13	57, −39, 26	3.8
Rounds 2 & 3 > Round 1
Medial frontal gyrus (L)	6	32	−1, 18, 44	4.8
Dorsolateral prefrontal cortex (L)	8, 9	66	−44 18, 32	5.8
Dorsolateral prefrontal cortex (R)	9	26	48, 21, 29	5.9
Middle frontal gyrus (L)	10	13	−38, 46, 15	4.2
Inferior parietal lobule, angular gyrus (L)	7, 39	157	−40, −58, 38	7.2
Cuneus, precuneus, posterior cingulate cortex (L)	18, 19, 31	279	−11, −71, 23	9.9
Insula (R)	13	24	26, 24, 3	5.5
Putamen (L)		33	−27, −6, 9	4.6
Putamen (R)		58	26, −5, 15	5.6
Caudate nuclei (bilateral)		32	8, 8, 6	3.8

Figure 2. The Caudate Displays a Round by Time Interaction.

The voxel clusters shown display a Round by Time interaction (p < 0.0001; contiguity threshold of 12 voxels). The green crosshair marks the caudate voxel with the peak F-value. Images are left-right reversed.

Some regions showed more activation for Round 1 than Rounds 2 and 3, while other regions showed greater activation for Rounds 2 and 3 than Round 1 (Table 1). The caudate nuclei showed the latter pattern in the left and right hemispheres—that is, a relatively flat response during Round 1 and an increase in the response during Rounds 2 and 3 (Figure 3; Supplemental Figure 1). The response is greatest for Round 2, with a slight attenuation for Round 3. Due to the sluggishness of the hemodynamic response, one would expect that activation related to anticipation of feedback would peak approximately 4–6 s after the trial start, which corresponds to time point T3 in this experiment. Pairwise two-tailed t-tests performed on time point T3 revealed significant differences between Rounds 1 and 2 (t(10) = −4.0, p < 0.05) and between Rounds 1 and 3 (t(10) = −2.5, p < 0.05), with a trend towards significance between Rounds 2 and 3 (t(10) = 1.9, p < 0.1). Because the response profile tends to diverge following feedback presentation, depending on whether the outcome is positive or negative, the response when collapsed across positive and negative feedback trials would not necessarily be expected to show an effect of round after the feedback has been revealed.

Figure 3. Time Course of Activation in the Caudate Region Displaying a Round by Time Effect.

The caudate nuclei displayed a relatively flat pattern of activation during Round 1 trials, while there is greater activation during Rounds 2 and 3. The activation is somewhat attenuated on Round 3 relative to Round 2. The time course is shown for the bilateral activation cluster identified as showing a Round by Time interaction. Each time period (T1, T2, etc.) represents a 2 s image acquisition. The dotted line indicates the onset of the feedback display.

The activation clusters showing an Accuracy by Time effect at p < 0.0001 with a contiguity threshold of 14 voxels are listed in Table 2 and shown in Figure 4. Bilateral activation was found in the ventral head of the caudate. The significance threshold was raised until the caudate region separated from contiguous regions, and the time courses of activation for each round are depicted in Figure 5 (Supplemental Figure 1 depicts the time courses with the data from the subjects with ceiling performance included). Although the time course of activation during Round 1 is relatively flat, there is a slight differentiation between trials with positive and negative feedback, with the activation dipping slightly below baseline following negative feedback. For Rounds 2 and 3, after an initial rise, the signal following negative feedback shows a pronounced dip below baseline, whereas the signal following positive feedback does not. In previous work, the BOLD signal has shown significant differentiation between correct and incorrect trials 6–9 s after the feedback display (Tricomi et al., 2006). This corresponds to T6 and T7 in the current experiment. Two-tailed t-tests were performed on these time points for each round. Significant differences were found at T6 on Round 1 (t(10) = 2.3, p < 0.05); at T6 and T7 on Round 2 (t(10) = 6.0 and 3.8, respectively, p < 0.05), and at T6 and T7 on Round 3 (t(10) = 7.9 and 4.4, respectively, p < 0.05). To test the hypothesis that the differentiation between correct and incorrect trials would vary as a function of round, an ANOVA was performed on the data from T6 and T7, which showed a significant Round by Accuracy interaction (F(2, 20) = 8.9 and 9.2 for T6 and T7, respectively, p < 0.05). Two-tailed t-tests on the data from these time points show that the signal difference between correct and incorrect trials was significantly different on Round 1 versus Round 2 (t(10) = −3.2 for T6, t(10) = −2.3 for T7, p <0.05) and on Round 1 versus Round 3 (t(10) = −3.6 for T6, t(10) = −4.7 for T7, p < 0.05), but not for Round 2 versus Round 3 (t(10) = −1.1 for T6, t(10) = −1.8 for T7, p > 0.05).

Table 2.

Regions Displaying an Accuracy X Time Interaction (p<0.0001)

Region of Activation	BA	Size (# voxels)	Peak Talairach Coordinates (x, y, z)	Maximum F Value
correct > incorrect
Parahippocampal gyrus, posterior cingulate (L)	19	91	−21, −52, 12	7.19
Superior temporal gyrus (R) (decreasing activation)	22	39	51, −5, −3	6.56
Cuneus, middle occipital gyrus, lingual gyrus, parahippocampal gyrus, fusiform gyrus (R)	19	356	23, −59, −6	10.94
Cuneus, middle occipital gyrus, inferior occipital gyrus (L)	19	77	−31, −75, −6	7.06
* Caudate head (bilateral)		See below	−8, 8, 0	15.45
incorrect > correct
Medial frontal gyrus, superior frontal gyrus, dorsal anterior cingulate (midline)	6, 8, 32	275	−2, 0, 53	9.67
Superior frontal gyrus (L)	9	19	−24, 39, 29	6.21
Middle frontal gyrus, inferior frontal gyrus (L)	9, 45, 46	109	9, 45, 46	7.83
Postcentral gyrus (L)	1, 2, 3	70	−43, −22, 53	7.52
Superior parietal lobule, inferior parietal lobule, angular gyrus (L)	7, 39, 40	130	−37, −56, 38	9.59
Inferior parietal lobule (R)	40	16	45, −58, 47	7.78
Middle temporal gyrus, inferior temporal gyrus, fusiform gyrus, anterior cerebellum (L)	21, 37	168	−40, −53, −24	9.3
Medial inferior cerebellum (bilateral)		104	14, −75, −24	8.99
* Posterior cingulate, caudate body, thalamus, putamen (bilateral)		See below	−14, −13, 18	12.5

^*A large activation cluster of 998 voxels spanned over the caudate head, posterior cingulate, caudate body, thalamus, and putamen and contained multiple activation peaks. The threshold was raised until this cluster split into clusters of fewer than 100 voxels, and the activation patterns in these smaller clusters were examined. The caudate head showed greater activation on correct than incorrect trials, while the other subclusters showed greater activation on incorrect than correct trials.

Figure 4. Brain Regions Showing an Accuracy by Time Effect.

The voxel clusters shown display an Accuracy by Time interaction (p < 0.0001; contiguity threshold of 14 voxels). The green crosshair marks the caudate voxel with the peak F-value. Images are left-right reversed.

Figure 5. Time Courses of Activation in the Caudate Region Displaying an Accuracy by Time Effect.

There is a slight differentiation between trials with positive and negative feedback during Round 1, whereas for Rounds 2 and 3, the signal following negative feedback shows a pronounced dip below baseline, unlike the signal following positive feedback. Each time period (T1, T2, etc.) represents a 2 s image acquisition. The dotted line indicates the onset of the feedback display.

Since the caudate cluster displaying an Accuracy by Time effect extended more dorsally than the caudate cluster displaying a Round by Time effect, we explored whether this was a functional dissociation by investigating the data, sorted by round, in the Accuracy by Time cluster, and the data, sorted by accuracy, in the Round by Time cluster. In both cases, the results showed effects at our time points of interest that were consistent with the effects from the main analyses. Specifically, pairwise two-tailed t-tests performed on time point T3 in the Accuracy by Time cluster showed significant differences between Rounds 1 and 2 (t(10) = −3.1, p < 0.05) and between Rounds 2 and 3 (t(10) = 2.5, p < 0.05). Two-tailed t-tests were performed on time points T6 and T7 in the Round by Time cluster revealed significant differences between correct and incorrect trials on Round 2 (t(10) = 5.2 for T6 and 2.6 for T7, p < 0.05) and Round 3 (t(10) = 7.9 for T6 and 4.0 for T7, p < 0.05) and no significant differences for Round 1. An ANOVA performed on the data from T6 and T7 showed a significant Round by Accuracy interaction at T7 (F(2, 20) = 8.2, p < 0.05) and a trend toward significance at T6 (F(2,20) = 3.5, p = 0.05). These findings suggest that although the more dorsal portion of the head of the caudate may be somewhat more sensitive to contextual differences related to the anticipation of feedback and the more ventral portion of the head of the caudate may be somewhat more sensitive to outcome valence, there is not a true functional dissociation between these subregions.

The activation clusters showing a three-way Round by Accuracy by Time effect at p < 0.0001, with a contiguity threshold of 11 voxels, are listed in Table 3. In general, the difference between correct and incorrect trials was greatest on Round 3, with one cluster exhibiting more activation for correct trials than incorrect trials and the rest showing the reverse pattern. The only clusters of activation in the caudate that showed a significant three-way interaction were in the caudate body, whereas the clusters showing Round by Time and Accuracy by Time effects were observed in the head of the caudate. The activation pattern in the caudate body clusters shows a distinct pattern of activation (Figure 7). Activation is elicited on all three rounds, with an earlier response on correct trials than on incorrect trials during Round 3, but not Rounds 1 and 2.

Table 3.

Regions Displaying a Round X Accuracy X Time Interaction (p<0.0001)

Region of Activation	BA	Size (# voxels)	Peak Talairach Coordinates (x, y, z)	Maximum F Value
correct > incorrect difference greatest on Round 3
medial frontal gyrus (R)	6	24	11, −5, 53	4.3
incorrect > correct difference greatest on Round 3
superior frontal gyrus (L)	8	42	−5, 36, 50	4.42
medial frontal gyrus (midline) (decreasing activation)	9	17	2, 42, 20	4.13
dorsolateral prefrontal cortex, inferior frontal gyrus/ventrolateral prefrontal cortex, insula (L)	9, 13, 46, 47	201	−37, 17, 6	5.03
anterior cingulate (midline)	32	25	−2, 24, 38	4.33
posterior cingulate (midline)	31	44	−2, −39, 29	5.66
precuneus (R)	7	22	−5, −55, 67	4.12
superior parietal lobule, inferior parietal lobule, angular gyrus, middle temporal gyrus (L)	7, 39, 40	178	−47, −61, 44	4.65
middle temporal gyrus, inferior temporal gyrus, fusiform gyrus (L)	20, 21, 37	53	−56, −24, −6	4.21
caudate body (R)		35	17, −8, 23	4.55
caudate body (L)		11	−18, −16, 23	3.78
Medial inferior cerebellum (L)		38	−14, −87, −24	4.62

Figure 7. Time Courses of Activation in the Caudate Body Region Showing a Round by Accuracy by Time Interaction.

Activation is elicited on all three rounds in the caudate body, with a response that shows a Round by Accuracy by Time Interaction. Each time period (T1, T2, etc.) respresents a 2 s image acquisition. The dotted line indicates the onset of the feedback display.

By far the largest cluster showing the three-way interaction was in the left dorsolateral prefrontal cortex (DLPFC) and inferior frontal gyrus (IFG). This area shows a robust response which begins prior to the feedback display in a similar way across conditions. However, the magnitude of the response to positive feedback, but not negative feedback, decreases with increasing round, so that the signal most strongly differentiates between incorrect and correct trials on Round 3 (Figure 8).

Figure 8. Time Courses of Activation in the Left Prefrontal Cortex Region Showing a Round by Accuracy by Time Interaction.

The signal is greater following incorrect than correct trials, and this difference is greatest on Round 3. Each time period (T1, T2, etc.) respresents a 2 s image acquisition. The dotted line indicates the onset of the feedback display.

Although the hippocampus/parahippocampus did not show any significant interactions, it did show a very strong main effect of time, bilaterally (peak Talairach coordinates: −27, −16, −12 and 23, −12, −15). The threshold was increased to isolate the hippocampal activation clusters from contiguous activated regions. These clusters are shown in Figure 6, along with the time courses of activation. The signal decreases from trial onset, and recovers following the feedback display.

Figure 6. Hippocampal Regions Showing a Main Effect of Time and their Time Courses of Activation.

Hippocampal activation clusters displaying a main effect of time are shown, along with the time courses of activation in these regions. The signal decreases from trial onset, and recovers following the feedback display. Each time period (T1, T2, etc.) represents a 2 s image acquisition. The dotted line indicates the onset of the feedback display. The brain image is left-right reversed.

Previous research has indicated that the parahippocampal/fusiform gyrus and the prefrontal cortex are more active when successful encoding is taking place, as measured by performance on a subsequent memory test (Brewer et al., 1998; Reber et al., 2002; Wagner et al., 1998). To see if a similar effect was present in the current task, the data on Round 1 were coded by whether the participant would answer correctly on the corresponding trial on Round 2, and an ANOVA was performed to identify voxels showing a Future Accuracy X Time effect. No regions were identified at the significance threshold of p < 0.0001, with a cluster threshold of 10 voxels. An exploratory analysis at a threshold of p < 0.001, however, did reveal regions similar to those found in previous work that displayed greater activation on trials associated with subsequent correct performance than on trials associated with subsequent incorrect performance. Specifically, such regions included the left fusiform gyrus (peak Talairach coordinates: −50, −43, −9), left posterior lateral inferior frontal gyrus (peak Talairach coordinates: −40, −1, 29), and the left middle frontal gyrus (peak Talairach coordinates: −47, 21, 26), with a location superior to the anterior lateral inferior frontal gyrus area found to show a subsequent memory effect in previous work (cf. Wagner et al., 1998). No regions were identified in the striatum or the hippocampus.

The striatal learning system is thought to support learning at a much slower rate than the declarative memory system (Bayley et al., 2005; Packard and Knowlton, 2002). Therefore, activation on one trial might not be expected to increase accuracy on the very next trial. Nevertheless, if the caudate is acting to automate retrieval of the correct response, activation in this region might be expected to result in decreased reaction time on the subsequent trial. To test this idea, the Round 2 trials for which performance was accurate on both Rounds 2 and 3 were coded by the difference in reaction time between the two rounds (note that trials with a large decrease in reaction time are not necessarily faster overall than trials with a smaller decrease or an increase in reaction time). A median split was used to divide the trials into those with a large reaction time decrease versus a small reaction time decrease (or an increase in some cases). No clusters showed a Reaction Time Difference by Time effect in the caudate nuclei at a threshold of p < 0.0001 and a cluster threshold of 9 voxels; however, an exploratory analysis with a threshold of p < 0.001 did identify a cluster in the right ventral caudate nucleus (peak Talairach coordinates: −8, 6, 0). The activation peak was higher for trials with a subsequently large reaction time decrease than for a small reaction time decrease. At this threshold, other regions also showed a Reaction Time Difference by Time effect, including the midbrain, right cuneus, right postcentral gyrus, and right superior cerebellum.

DISCUSSION

Feedback processing in a declarative memory task

The engagement of the caudate nuclei in this experiment provides support for the idea that the caudate processes performance feedback irrespective of whether the feedback is provided in the context of a task involving declarative or nondeclarative memory. Although nondeclarative learning may have been taking place in our experimental task, the task differed in many respects from tasks designed to primarily engage nondeclarative memory and was instead similar to tasks designed to engage declarative memory. As defined by Zola and Squire (2000), “Declarative memory refers to the capacity for conscious recollection of facts and events. It is specialized for rapid, even one-trial learning, and for forming conjunctions between arbitrarily different stimuli. It is typically assessed in humans by tests of recall, recognition, or cued recall…” (p. 486). In our experiment, learning occurred quickly, with performance significantly above chance after only one presentation of the 60 trials. Additionally, participants were consciously trying to learn the word pairs, which were arbitrarily related, and their memory was assessed through a recognition test. Nevertheless, the caudate nuclei were engaged in much the same way as in previous work examining feedback processing during nondeclarative learning (Tricomi et al., 2006). If the brain structures associated with declarative and nondeclarative learning always acted in an antagonistic manner, one would predict that the hippocampus would show a Round by Time effect in the opposite direction to that observed in the caudate. Instead, there was no Round by Time effect found for the hippocampal signal. Rather, hippocampal activation was modulated by our task during all three rounds of the experimental paradigm. Therefore, hippocampal activation may be modulated by the memory demands of a task, whether the task involves feedback processing or not. Indeed, a recent study found similar activation in the hippocampus for observational learning and feedback-based learning versions of a categorization task (Cincotta and Seger, 2007), and other work has found that the caudate and hippocampus are cooperatively involved in route recognition (Voermans et al., 2004).

Neuropsychological work demonstrates the crucial importance of the MTL in forming arbitrary associations, like those that were learned in the experiments presented here (Squire, 2004; Zola and Squire, 2000). Yet, this region showed a decreasing response profile in our experiment. Previous work has also found that MTL activation tends to be below baseline when that baseline is unconstrained (Stark and Squire, 2001). These decreases can be avoided if a baseline task is utilized that constrains cognitive activity, such as responding as to whether digits are odd or even (Stark and Squire, 2001). However, this would seem to change the nature of the task, eliminating not only task-unrelated thought that might be associated with activation in the MTL, but also reducing the amount of time when the MTL can be recruited to aid in memory formation. Although the MTL seems to display a high level of tonic activation, which then tends to decrease when attention is focused on a learning a simple arbitrary association, this relatively low level of activation may still support memory formation. It should be noted that arbitrary word pairs are much less rich in detail than the more complex associations that we make every day, and indeed, the hippocampus has been noted to be especially important for memory of rich configural associations, such as those required in spatial navigation (Maguire et al., 1997). The lack of detail of the word pairs may cause them to activate the MTL at a relatively low level, but this low level of activation could still be critical to memory formation and cause amnesics with MTL damage to have great difficulty in making arbitrary word pair associations.

The caudate’s role in feedback processing

Importantly, the results of this experiment indicate that the head of the caudate is more robustly activated when feedback will indicate task success, rather than when feedback is purely informational. Although the same sixty trials were repeated three times, the resulting activation in the caudate, as well as in other brain regions, was quite different. In this experiment, the time between the trial onset and the feedback display was kept constant across trials; this eliminates confounds that may arise from delaying feedback (Maddox et al., 2003) but also means that the BOLD signal observed on each trial may reflect a combination of activation occurring prior to and after the feedback presentation. Previous work indicates that the presence of feedback increases the signal in the caudate, even prior to the feedback presentation, compared to trials with no feedback (Tricomi et al., 2006). The experiment presented here, however, indicates that the presentation of feedback is not sufficient to drive a robust response in the caudate.

On Round 1, the information provided by the feedback about which response is correct is maximal, yet the caudate signal during feedback anticipation was significantly lower, and the differentiation in signal following positive versus negative feedback was significantly smaller, on this round compared to the subsequent rounds. This fits with the idea that one’s sense of agency in determining the outcome may be an important factor in determining caudate activation. Simply performing an action prior to the receipt of a reward or punishment is not enough to elicit caudate activation; rather, one must believe that the outcome is contingently linked to the response (Tricomi et al., 2004). In Round 1 of the experiment, the perception of response-outcome contingency may have been minimal, since it was stressed to the participants that the correct word pairs were arbitrary, and there was little reason for participants to engage in the gambler’s fallacy since money was not at stake.

A second factor which may determine the degree to which the caudate is recruited during feedback-based learning is the subjective value of the feedback to the individual. During Round 1, both positive and negative feedback provide equal amounts of information about which answer is correct, and so both are equally valuable in helping learners achieve their goal of learning the correct answers. Because of this, learners may not be invested in which type of feedback they receive. It is possible that if different types of feedback were differentially effective in facilitating goal achievement, or if trials with and without feedback were interspersed, the caudate would be more responsive to the feedback. There was a significant increase in activation on Round 2, once feedback began to reflect task performance. At this point, positive and negative feedback indicate whether learners are achieving the goal of learning the correct word pairs. The two types of feedback begin to differ in their reward value; positive feedback indicates task success, while negative feedback indicates an error in performance. Because of this, participants may have been more invested in the outcome valence. The crucial role that participants’ goals and expectations play in determining activation in the head of caudate underscores the high degree of influence of top-down control in this region. Whereas the putamen receives much of its input from sensorimotor cortex and the ventral striatum receives input from limbic cortex, the caudate receives inputs from a wide range of association cortices, which may make it ideally situated for cognitive influence (Haber and Fudge, 1997).

Feedback-related activation of the midbrain and striatum has been found in studies involving probabilistic learning (e.g., Aron et al., 2004; Poldrack et al., 1999). In these tasks, category labels are probabilistically associated with feature combinations and over many trials knowledge of these associations are slowly built up, although participants often have the sense that they are guessing. In these tasks, the individual trials are not completely independent of one another; knowledge of the relationship between each feature and the category it predicts is relevant for all trials involving that feature. Therefore, feedback provides information about task success after only a few trials, so the striatum would be expected to become active quite early in training. In contrast, for our study, the sixty trials within each round were completely distinct, so the feedback for each trial was only useful in determining the correct answer for the corresponding trial on the next round and not for other trials within the current round.

One possible interpretation of our data is that the caudate activation on Round 2 reflects expectation of a predicted reward followed by a signal reflecting prediction error, that is, whether the outcome was better or worse than expected (e.g., Fiorillo et al., 2003). The ventral and dorsal striatum have been shown previously to reflect such prediction error signals (e.g., (O'Doherty et al., 2004). This interpretation must be reconciled with the lack of a robust caudate signal during either expectation or receipt of feedback during Round 1. It is possible that during the first round of trials, participants did not form specific predictions about the upcoming feedback, so signals reflecting prediction of reward and prediction error were absent.

An additional finding was that although the caudate response was still significantly greater on Round 3 than on Round 1, the magnitude of the rise was somewhat attenuated from the signal rise on Round 2. This fits within the context of other experiments that have shown diminished activation in the caudate as appropriate responses become well-learned (Delgado et al., 2005; Haruno et al., 2004; Jueptner et al., 1997; Law et al., 2005; Pasupathy and Miller, 2005; Williams and Eskandar, 2006). This finding can be explained in terms of outcome uncertainty and the decreased value of the feedback to the learner once the outcomes become known. In the present experiment, correct performance was already at 75% for Round 3. As correct responses become well-learned, positive feedback becomes more expected. Therefore, it carries less information about performance. In other words, if a learner is already highly confident that a particular response is correct, positive feedback does not tell the learner anything new. If, on the other hand, the learner has only a “hunch” that a given response is correct, positive feedback confirms that hunch, and in doing so provides performance-related information that the learner wouldn’t have otherwise known.

Support for this idea comes from the fact that reaction time begins to significantly differ between correct and incorrect responses on Round 3. On the post-tests, confidence and reaction time were inversely correlated; participants tended to respond more quickly on trials for which they were most confident. We can assume that this correlation was in effect during the scanning session as well, so the finding that reaction times were faster for correct trials than incorrect trials during Round 3 suggests that during this round, participants were more confident of their correct responses. In contrast, on Round 2, no such reaction time effect exists, indicating that although performance was above chance, participants may not yet have been more confident of their correct responses than their incorrect responses. Therefore, feedback would provide useful information about task success, even on correct trials.

If activation in the caudate nucleus is behaviorally significant, then one might expect that confidence would increase most when the caudate signal is greatest. Indeed, there was some evidence that this was the case. Since reaction time tracks inversely with confidence, large decreases in reaction time should signify increases in confidence, relative to small decreases or increases in reaction time. The right caudate nucleus did show greater peak activation on correct trials during Round 2 that were associated with a large decrease in reaction time on Round 3. Although this effect was found at an exploratory significance threshold, it raises the intriguing possibility that caudate activation may act to solidify knowledge of correct responses, even when those responses are conceptual in nature, rather than depending on a particular motor action (since the required keypress for the correct word match was randomly determined each round). Over repeated trials, this activation could potentially serve to automate responding.

Subsequent memory effects in declarative memory regions

Although our main hypotheses were in regard to activation in the caudate nuclei, clearly other brain regions play an important role in our task. Brain regions that have been found to show subsequent memory effects in other declarative memory tasks, such as the left prefrontal cortex and left fusiform gyrus (Brewer et al., 1998; Wagner et al., 1998), also showed a similar effect during Round 1 of our task, whereas the caudate did not. Indeed there was relatively little activation in the caudate during Round 1, and yet performance was above chance on Round 2, indicating that learning took place. This suggests that although the caudate may not be responsible for utilizing the information provided by the feedback to guide memory encoding, other brain regions do process this information, allowing learning to proceed.

The effects of memory strength on brain activation

The effects of memory strength on activation of the MTL and PFC, as well as the striatum, have been studied in the context of a task which was also an arbitrary multiple choice task with feedback, although the feedback was not the focus of the study (Law et al 2005). Participants learned to associate kaleidoscope images with one of four spatial locations, training on small sets of associations until they were well learned. MTL activation was found to increase with memory strength, while activation in other regions, including the DLPFC and right caudate nucleus, were found to decrease once an association was well-learned. Consistent with the latter finding, we found activation in the caudate and prefrontal cortex during early learning, and found early indications of a decrease in caudate activation as associations became well learned. Additionally, in the left prefrontal cortex, the signal for correct trials, but not incorrect trials, decreased with increasing round. This is consistent with the idea that activation in this region decreases when positive feedback is no longer informative, whereas the signal does not decrease for trials with negative feedback, which remains informative. We did not observe increases in MTL activation over the three rounds of trials, however, indicating that more than three repetitions may be necessary for an effect of memory strength in this region to become apparent.

Other work has demonstrated that the MTL and striatum are sensitive to stimulus novelty, with novel stimuli eliciting greater activation than familiar stimuli (Berns et al., 2001; Dolan and Fletcher, 1997; Tulving et al., 1996). In our experiment, the words would have been most novel on Round 1, yet activation of the MTL and striatum was not greater on this round compared to other rounds. Perhaps since the word displays all looked similar and the participants were unlikely to be attending to the novelty of the words, the effects of novelty on activation in these regions was diminished. That the caudate response was actually higher in Round 2 than Round 1 indicates that the effects in the caudate cannot be explained in terms of novelty.

Contributions of striatal subregions

Although the results from Round 1 of our experiment indicate that the PFC and MTL can act independently from the striatum, these regions are nevertheless part of a broader integrative network. The striatum, as the input unit of the basal ganglia, receives a wide range of converging input from the cortex (Bar-Gad et al., 2003; Haber et al., 2006). Several regions in the frontal cortex, including the dorsolateral prefrontal cortex, the ventromedial prefrontal cortex, the orbitofrontal cortex, and the anterior cingulate have been linked in various ways to reward processing and reward-related learning (Elliott et al., 2000a; Elliott et al., 2000b; Fellows and Farah, 2005; Holroyd and Coles, 2002; O'Doherty et al., 2001). These cortical regions all project to the striatum (Haber et al., 2006), and they also receive input from the basal ganglia through basal ganglia-thalamocortical pathways (Middleton and Strick, 2002). Thus, they are in a good position to interact with the striatum during feedback-based learning. Similarly, neural circuitry supports the idea that the hippocampus and striatum may interact. Midbrain dopaminergic neurons project to both the hippocampus and striatum, and the hippocampus and midbrain form a functional loop via projections through the ventral striatum (Lisman and Grace, 2005). Striatonigrostriatal pathways may allow for further integration of information from the dorsal and ventral striatum (Haber et al., 2000).

As our understanding of the role of the striatum in reward-related processes increases, the contributions of different subregions of the striatum are becoming more well-known. Although in this study the voxels with the peak F-values lie in the dorsal striatum, the Round by Time and Accuracy by Time ROIs extend into the ventral striatum (to z = −6). The contributions of the dorsal and ventral striatum have been dissociated in some studies (Ito et al., 2000; Ito et al., 2002; O'Doherty et al., 2004; Robbins and Everitt, 1992; Zink et al., 2003). Although we did not find a dissociation between response profiles in the dorsal versus ventral striatum in this study, the experiment was not designed to dissociate activation in these two striatal subregions, and our results are consistent with other work which has found that activation in both the dorsal and ventral striatum is modulated during instrumental learning (O'Doherty et al., 2004).

Previous work has also suggested that the head and body of the caudate have dissociable roles (Cincotta and Seger, 2007; Nomura et al., 2007; Seger and Cincotta, 2005). We too found that the body of the caudate showed a different pattern of activation from that observed in the head of the caudate. Specifically, the caudate body showed a signal that increased from baseline on all rounds, and this signal emerged later in time course than for the caudate head. Differences in the signal for correct and incorrect trials began to emerge as Round increased. Late in the time course, the signal was greater for incorrect trials compared to correct trials, and on Round 3, the signal rose more quickly for correct than incorrect trials. Seger and Cincotta (2005) found that activation in the caudate body changes over the course of training. They detected an increased response over many trials of training on a classification task with feedback, and greater activation on correct than incorrect trials late in training. It is possible that the results of the present study show the beginnings of performance-related differences in the signal produced by the caudate body, which might continue to evolve with more training. However, further research will be required to pinpoint the functional role of this region relative to the caudate head.

Conclusion

In this experiment, we manipulated the meaning of feedback relative to the goal of the learner. Nothing about feedback displays (e.g., green checkmarks) is intrinsically rewarding. It is the meaning in relation to the task that endows feedback with value. Notably, however, the goals of obtaining positive feedback and improving performance are not the same (Kluger and DeNisi, 1996). Each round of trials in our experiment was superficially identical, but the feedback was only indicative of goal achievement on the second two rounds, and during these rounds, the head of the caudate showed an increase in activation. The context-driven nature of the response in the caudate reveals one way in which learning may be influenced by incentives and objectives. Even when the process of mastering a task is not in itself rewarding, the knowledge of increasing mastery may be, and the widespread use of feedback as an instruction tool may capitalize on this to increase learners’ motivation and learning efficacy.