Selective role for striatal and prefrontal regions in processing first trial feedback during single-trial associative learning

Abstract

Discrete jumps in knowledge, as exemplified by single-trial learning, are critical to survival. Despite its importance, however, one-trial learning remains understudied. We sought to better understand the brain activity adaptations that track punctuated changes in associative knowledge by studying visual-motor associative learning with functional magnetic resonance imaging. Human and primate neurophysiological studies of feedback-based learning indicate that performance feedback elicits high activity at first that diminishes rapidly with repeated success. Based on these findings we hypothesized a network of brain regions would track the importance of feedback, which is large early in learning and diminishes thereafter. Specifically, based on neurophysiological findings, we predicted that frontal and striatal regions would show a large activation to first trial feedback and a subsequent reduction selective to performance feedback but not stimulus cue presentation. We observed that the striatum and frontal cortex as well as several other cortical and subcortical sites exhibited this pattern. These findings match our prediction for activity in frontal and striatal regions. Furthermore, these observations support the more general hypothesis that a large network of regions participates in the associative process once the behavioral goal is definitively identified by first trial performance feedback. Activity in this network declines upon further rehearsal but only for feedback presentation. We suggest that, based on the timing of this process, these regions participate in binding together stimulus cue, motor response, and performance feedback information into an association that is used to accurately perform the task on after the first trial.

1. Introduction

Associative learning plays a key role in survival by providing a mechanism for behavioral adaptation. This reinforcement learning mechanisms allow organisms to learn through experience how to respond to environmental contexts that promote or threaten survival. When an organism’s response is highly consequential to survival learning may occur in one trial. Despite its importance, however, the neural correlates of one-trial learning remain understudied.

Identifying single-trial learning effects on the brain depends upon the precise experimental control of learning. Of the two basic approaches to the study of instrumental associative learning, deterministic and probabilistic, deterministic paradigms are better suited to identify single-trial learning effects. Deterministic paradigms use fixed feedback rules such that positive feedback is always given for the correct response and negative feedback is always given for the incorrect response. Probabilistic paradigms use stochastic feedback rules that can randomly vary from trial to trial according to a predetermined proportion but that on average indicate that one response is better than another. Although probabilistic paradigms contribute to our understanding of striatal, prefrontal and medial temporal lobe involvement in associative learning (Aron et al., 2004; Cools et al., 2002; Poldrack et al., 2001), probabilistic approaches cannot address one-trial learning because associative knowledge accumulates gradually over many trials. Complex deterministic category learning paradigms also yield gradual learning effects (Boettiger and D’Esposito 2005), but simple deterministic tasks are well suited to examine discrete changes in brain activation. This is because associations can be conclusively learned in a few trials and remain fixed thereafter. In spite of this, most deterministic paradigms, our own included, provide less than the ideal control of the learning process that is needed to examine one-trial learning (Bedard and Sanes 2009; Brovelli et al., 2008; Eliassen et al., 2003). In particular, successful learning can be defined by the occurrence of the first correct response to a stimulus cue. This first correct trial, when followed by only correct responses, indicates a definitive knowledge of the cue-response-outcome association. Deterministic paradigms often use multiple response alternatives or rules, however, which allows learning to occur on error trials preceding the first correct response through a process of elimination. These preliminary errors provide information to guide future behavior by identifying the remaining possible response alternatives. Often, the correct response can be learned before being selected, as in the Wisconsin Card-Sorting Test and similar paradigms (Barcelo et al., 2000; Barcelo et al., 2002; Monchi et al., 2001; Zanolie et al., 2008). This allows definitive associative knowledge to be obtained before the first correct trial. In that case, analyses that aggregate across preliminary errors or that examine only the first correct response provide an incomplete description of the changes associated with single-trial learning.

Instrumental associations can only be learned on the basis of feedback. As associations become learned by trial and error, feedback begins to convey less critical information about how to conduct future behavior. Feedback becomes completely predictable, especially if the associations are uncomplicated and deterministic. Electrophysiological correlates of this change in feedback predictability have been observed in both primate and human studies. These changes provide the basis for our hypothesis that brain activation in single-trial learning will track the importance, or utility, of feedback to future task performance. In human event-related potential (ERP) EEG studies, researchers have identified activity changes associated with unexpected feedback following a rule switch. Specifically, Barcelo and colleagues observe an increase in the frontally located P300A waveform in response to feedback presentation after a rule-switch compared to before (Barcelo et al., 2002). This activity diminishes quickly with repeated successes following a switch. Primate single-unit studies of the reward system show that activity in dopaminergic ventral tegmental area neurons also tracks changes in feedback utility. The so-called reward response of this system initially reacts to the presentation of unpredicted positive performance feedback. As the positive feedback becomes predictable with learning this reaction diminishes (Schultz et al., 1995; Schultz et al., 1997). The frontal targets of the reward system also exhibit activity correlated with feedback utility in a probabilistic paradigm (Aron et al., 2004). However, deterministic human learning studies to date provide only indirect evidence of a single-trial feedback-specific change in striatal activity with learning. Brovelli et al. (Brovelli et al., 2008) show that the striatum is more active early in learning. The relationship of this activity to trial-by-trial changes in knowledge remains unclear, though, because they use a paradigm in which learning is defined across several trials through a modeling procedure that defines a prediction error signal. Using a trial-based analysis strategy, Bedard and Sanes (Bedard and Sanes 2009) observe that Parkinson’s Disease patients differ from controls more early in learning than later, indirectly suggesting a more prominent striatal role early in learning in healthy individuals. Altogether, this research suggests that performance feedback activation should decrease following single-trial learning in striatum and prefrontal cortex. Such activity changes would correlate with the declining utility of feedback to guiding future performance.

In the current study we investigated this question using a one-trial visual-motor associative learning paradigm. The paradigm required participants to learn associations between pictures and responses. The task used easily nameable color pictures and a two-alternative choice response. Trials were classified by the pattern of performance, and the analysis included only associations where no errors occurred after the first trial. We predicted that stimulus cue representations and performance feedback representations would be differentially altered by associative learning on the first trial. Specifically, in the transition from learning on the first trial to rehearsal on the second trial there would be reductions in feedback activation in frontal and striatal regions but not similar changes in stimulus activation.

2. RESULTS

2.1 Behavior

The analysis of reaction times indicated the presence of learning related reductions. There was a significant effect of experience (F(5,95)=11.46, P<0.001), whereby reaction time differed significantly across the six categories from 1^st Trials through Control trials. Post-hoc Tukey comparisons indicated significant pair-wise differences in reaction time among all learning trials and control (all P<0.005, denoted by the $ in Fig. 2). In addition there were significant reductions in reaction time between 1^st Trials and Correct 4^th Trials (P<0.01, denoted by the * in Fig. 2) as well as significant differences between 1^st and 5-8^th (P<0.001), 2^nd and 5-8^th (P<0.05), and 3^rd and 5-8^th (P<0.05), which are denoted by the # Fig. 2. There was no difference between any consecutive pair of learning categories. In summary, reaction time progressively decreased with successive presentations of each picture (Fig. 2), and reaction time differed significantly between the learning and control tasks.

Figure 2.

Reaction time as a function of experience (1^st Trials, Correct 2^nd, etc). Learning is evident as RT decreases from early to later learning trials. The asterisk (*) indicates RTs significantly different from Correct 4^th Trials. The pound symbol (#) indicates RTs significantly different from Correct 5-8^th Trials. The dollar sign ($) indicates RTs significantly different from Control Trials.

We collected error information from each participant. We did not report standard accuracy measures because we selected only correct trials for the analysis of learning, thus there was no error rate over time to use as a proxy for learning (cf. (Eliassen et al., 2003)). Errors after 1^st Trials indicated a failure to learn and series with such errors before the fifth trial were excluded from 1^st- Correct 4^th Trial analyses. After 1^st Trials, there were an average of 5±4 errors during the learning task (mean±SD. dev.; range 0-17) and 1±1 errors during the control task (range 0-6). We had a fairly even number of learning trials across subjects. There were 21±2 trials in each of the categories 1^st Trials, Correct 2^nd Trials, Correct 3^rd Trials, and Correct 4^th Trials (range 14-24). All but one subject had 19 or more 1^st through Correct 4^th Trials. The numbers are identical for each category because we excluded from analysis any series of trials with an error after the first and before the fifth trial (see methods). We also had similar numbers of late learning and control trials across subjects. There were 91±5 Correct 5-8^th Trials (range 80-96) and 95±1 Control trials (range 91-96). On the whole the numbers of trials were consistent from subject to subject.

2.2 Brain Imaging

Learning led to changes in brain activation across a wide set of regions. We observed widespread reductions in activation for both stimulus and performance feedback presentation from 1^st Trials to Correct 2^nd Trials (Fig. 3A, B). In support of our prediction, we observed reductions in activation from 1^st Trials to Correct 2^nd Trials that were specific to feedback in frontal, parietal and striatal regions (Fig. 3C). Several regions exhibited a similar pattern indicating that activation in response to feedback presentation on 1^st Trials diminished dramatically on Correct 2^nd and later trials (Table 2; Fig. 3C and D-I). These feedback selective regions included medial parietal cortex, bilateral superior and inferior parietal lobule, medial frontal cortex (supplementary motor area), bilateral middle frontal gyrus, left precentral gyrus (premotor area), and bilateral striatum. Several additional areas exhibited this same pattern, although we did not predict such changes. These regions included bilateral thalamus, bilateral mid-insula, bilateral occipital cortex and left middle temporal gyrus. Six of these regions are graphed in Fig. 3 (panels D-I) to illustrate the decrease in activity that is unique to feedback on the 1^st Trial vs. Correct 2^nd Trial and the subsequent trials for comparison. In contrast, the activity in response to stimulus cue presentation remained elevated, increased slightly, or decreased slightly from 1^st Trials to Correct 2^nd Trials. Visual inspection of activity in all the subsidiary peaks listed in Table 2 indicates this same pattern (data not shown). Following our secondary deconvolution analysis we extracted the BOLD time courses from these same six regions. These time courses are plotted in Fig. 4, which shows the more pronounced reduction between the first and second for the feedback event compared to the stimulus event. Thus, the learning effects identified by the fixed shaped canonical hemodynamic response function can be clearly observed in the time varying plots of BOLD activity from these same regions. The regions shown in Fig. 3A and B indicate a number of sites where activity decreased (blue regions) or increased (orange regions) for stimulus and feedback in the transition from 1^st to Correct 2^nd Trials. Regions with decreasing activity with learning included bilateral insula/inferior frontal operculum, anterior cingulate cortex and ventral striatum. Regions with increasing activity during learning included the bilateral posterior insula. These regions exhibited non-selective learning changes affecting both stimulus and feedback events.

Figure 3.

Brain regions exhibiting a significant effect of learning. (A) Regions that show a significant change in activation for stimulus cue presentation from 1^st to Correct 2^nd Trials. (B) Regions that show a significant change in activation for performance feedback presentation from 1^st to Correct 2^nd Trials. (C) Regions that show a significant interaction between event type (stimulus vs. feedback) and experience (1^st Trials vs. Correct 2^nd Trials). Orange indicates that the decrease in activation seen in panels A and B is larger for the feedback event than for stimulus presentation. (D-F) Graphs of activity in selected regions of interest (black circles) from Table 2. Graphs depict the selective decrease from 1^st to Correct 2^nd Trials for feedback presentation but also show for comparison activity in each region for Correct 3^rd, 4^th and 5^th-8^th Trials and Control Trials. Activation magnitude is displayed in arbitrary MR units (color bar). For the main effects in A and B, orange regions show an increase in activation from 1^st to Correct 2^nd Trials and blue regions show a decrease. For the interaction effect in C, orange indicates a greater reduction in activation from 1^st Trials to Correct 2^nd Trials for feedback than for stimulus.

Table 2.

Regions with unique learning effects for stimulus and feedback

Cluster #	Cluster Size		Center of Mass				Region
	(# Voxels)		X	Y	Z
			(Local Peak Activation)
1	2214		7	60	42
1.1		(	5	74	39	)	L precuneus
1.2		(	40	59	51	)	L SPL, BA 7	{H}
1.3		(	43	31	44	)	L IPL, BA 40
1.4		(	-34	52	38	)	R IPL, BA 40
1.5		(	-25	74	44	)	R SPL, BA 7
2	1326		-28	-13	42
2.1		(	-32	2	63	)	R MFG, BA 6
2.2		(	-29	-20	54	)	R MFG, BA 6	{I}
2.3		(	-32	-41	38	)	R MFG, BA 8
2.4		(	-49	-5	44	)	R MFG, BA 6
2.5		(	1	1	51	)	medial SFG, BA 6
2.6		(	-38	-8	5	)	R insula, BA 13
2.7		(	-22	-8	14	)	R putamen	{G}
3	961		28	49	-20
3.1		(	-5	32	-28	)	pons
3.2		(	19	58	-22	)	L cerebellum (VI)	{D}
3.3		(	10	70	-28	)	L cerebellum (Crus 2)
3.4		(	28	59	-31	)	L cerebellum (Crus 1)
3.5		(	13	44	-46	)	L cerebellum (IX)
3.6		(	44	55	-21	)	L FFG, BA 37
3.7		(	59	52	8	)	L MTG, BA 22
3.8		(	50	37	2	)	L MTG, BA 22
4	337		33	-31	36
4.1		(	35	-38	39	)	L MFG, BA 8
5	335		31	6	57
5.1		(	35	11	63	)	L PCG, BA 6	{F}
5.2		(	35	-8	53	)	L MFG, BA 6
6	279		-1	78	3
6.1		(	2	95	3	)	L cuneus, BA 18
6.2		(	-11	65	20	)	R precuneus, BA 31
6.3		(	13	73	2	)	L lingual gyrus, BA 18
6.4		(	-2	70	-4	)	R lingual gyrus, BA 18
7	272		26	-3	8
7.1		(	11	17	12	)	L thalamus
7.2		(	28	10	5	)	L putamen
7.3		(	34	-8	5	)	L insula, BA 13
7.4		(	19	-20	11	)	L caudate
8	235		-21	64	-34
8.1		(	-8	74	-28	)	R cerebellum (Crus 2)
8.2		(	-34	58	-30	)	R cerebellum (Crus 1)
8.3		(	-14	70	-49	)	R cerebellum (VII/VIII)
9	141		-17	11	8
9.1		(	-14	11	12	)	R thalamus
10	87		12	24	8
10.1		(	-2	23	12	)	R thalamus	{E}
10.2		(	10	22	8	)	L thalamus

BA, Brodmann’s Area; L, center; R, Right; SPL, superior parietal lobule; IPL, inferior parietal lobule; MFG, middle frontal gyrus; SFG, superior frontal gyrus; FFG, fusiform gyrus; MTG, middle temporal gyrus; PCG, precentral gyrus; Letters in curly braces refer to activity graph panels in Fig. 3.

Figure 4.

BOLD Time Courses from Regions of Interest. Temporal plots of brain activation corroborate the observed effects in Fig. 3, panels D-F, where we showed a greater reduction in brain activation in response to feedback between the 1^st Trial and Correct 2^nd Trial than for stimulus. (A) Left Cerebellum. (B) Right Thalamus. (C) Left Precentral Gyrus. (D) Right Putamen. (E) Left Superior Parietal Lobule. (F) Right Middle Frontal Gyrus. Plots are of the fit coefficients extracted from the regions of interest in Fig. 3 and Table 2. Plotted with standard error bars. Time in seconds is plotted on the x-axis, and MR signal intensity in raw units is plotted on the y-axis. The key is as follows 1^st Stm=1^st Stimulus, Cor. 2^nd Stm=Correct 2^nd Stimulus, 1^st Fbk=1^st Feedback, Cor. 2^nd Fbk=Correct 2^nd Feedback. Stimulus events are plotted in solid lines with filled markers. Feedback events are plotted in dotted lines with open markers. Diamond markers denote 1^st Trials, and square markers denote Correct 2^nd Trials.

3. Discussion

This study was designed to test the hypothesis that striatal and frontal activity would respond in relation to the importance of feedback during learning. Feedback at the beginning of learning is of greater consequence to planning future behavior than after learning. After an association has been learned on the basis of feedback, feedback becomes predictable. Human and primate studies report greater brain activity in the dopamine system (Schultz et al., 1995; Schultz et al., 1997) and its afferent targets (Aron et al., 2004) early in learning when feedback is uncertain. ERP studies of the P300A also indicate that frontal cortex responds more prominently to uncertain feedback following a rule switch in the WCST (Barcelo et al., 2002). On the basis of these studies we predicted higher activity in striatum and PFC during as compared to after learning. We observed greater activation in response to performance feedback on 1^st Trials than on Correct 2^nd Trials in a number of regions including striatum and bilateral middle frontal gyrus. This effect was specific to feedback presentation and not stimulus cue presentations. Stimulus cue activation decreases little or none after 1^st Trials in the regions that exhibit the feedback effect. Because activation is strongest on 1^st Trials when feedback identifies the response rule, we interpret this to indicate that striatum and PFC participate in performance evaluation. This evaluative process would include associating, or binding together, information about the stimulus cue, the motor response and the outcome. This process can only and must occur following feedback presentation on the 1^st trial in order to perform the second trial correctly. This matches the idea that the brain representation of an associative rule has to be activated, or in this case created, before the behavior it governs (Barcelo et al., 2002; Miller and Cohen 2001). Also, other similar paradigms suggest the importance of the striatum early in learning (Bedard and Sanes 2009; Brovelli et al., 2008). Moreover, Brovelli and colleagues have shown, using both model-based and trial-by-trial analyses, that ventro- and dorso-lateral PFC and striatum respond selectively to feedback during early learning trials including preliminary errors and the first correct trial (Brovelli et al., 2008). Our new data align our own and other prior findings that PFC and striatum participate critically in the earliest associative learning trials (Brovelli et al., 2011; Eliassen et al., 2003). This study extends those findings to show that prefrontal and striatal activity is coupled specifically to the feedback outcome event. Altogether, activity in striatum and PFC is consistent with a role in the association process that, at least in this paradigm, must be initiated at the end of the first trial.

Despite the fact that these effects occur on 1^st Trials, explanations based on novelty and uncertainty do not fully capture the observed changes. A novelty response would be expected to yield the greatest activity on 1^st Trials and less activity on later trials. This was observed for feedback responses, but not for stimulus cue responses, which are arguably more novel than feedback. The feedback symbols were used throughout the experiment in different combinations whereas pictures in each run were unique, especially on the first trial. Activation to stimulus cues did not diminish as much as for feedback in any region despite the greater novelty of pictures. Uncertainty also does not explain the differences. A strict correlation between brain activation and uncertainty predicts that Correct 2^nd Trial feedback activity should drop to zero, and not all regions show such a pattern. A less rigid relationship between uncertainty and brain activation would predict a gradual decrease over time, which is not observed universally either in the response to stimulus or feedback. Neither novelty nor uncertainty provides a satisfying explanation of our learning effects, similar to conclusions reached regarding ERP learning effects in the WCST (Barcelo et al., 2000).

In addition to striatum and PFC, we observed that portions of parietal lobe, occipital lobe, temporal lobe, thalamus, and cerebellum responded uniquely to 1^st Trial feedback (Table 2). We also observed learning effects that were shared by stimulus and feedback events (Fig. 3A,B). The learning changes common to stimulus and feedback were, for the most part, reductions in activation as we have seen previously (Eliassen et al., 2003). Two regions in particular stand out. The bilateral frontal operculum/inferior frontal gyrus/insula region (Fig. 3A,B slice 9S) has been identified in several related studies. Activity in this region decreases with learning or reduced uncertainty (Cools et al., 2002; Konishi et al., 2002; Monchi et al., 2001; O Doherty et al., 2001; O Doherty et al., 2003). The central operculum, the orange regions in Fig. 3A,B, shows an increase with learning for both stimulus and feedback, although more prominently and bilaterally for stimulus presentation. This region has been shown in a human classical conditioning paradigm to increase activation with learning as expectations change (Ramnani et al., 2000). Although our observed changes occurred between the first and second trials, our results remain consistent with existing learning studies. The unique effects for feedback include a substantial number of regions outside PFC and striatum. These additional regions could be participating in the process of association since they were selectively active on the first trial like PFC and striatum. Alternatively, activity in these regions, especially parietal and occipital sites, might derive from the visual properties of stimulus and feedback presentation and the visual-motor transformations required to execute responses. Changing the psychophysical properties in future experiments might help clarify this issue. For example using auditory stimulus and feedback presentation and requiring verbal responses would be predicted to alter activation in brain regions that are concerned with stimulus and response modality as opposed to learning. Another possibility is that association processes arising in the striatum and frontal cortex exert top-down influences on these other regions through cognitive control mechanisms. In any case, the unpredicted activity in these other regions during the critical 1^st Trial feedback encourages further characterization of the contributions of these other areas to associative learning.

One aspect of this study that may affect our interpretations is that on 1^st Trials we included both positive and negative feedback outcomes. We adopted this approach because 1^st trial feedback, regardless of valence, identifies the correct response for the second trial. Also, preliminary analyses indicated no significant differences at our thresholds (corrected P<0.01, voxel p<0.001, cluster n≥20). Relaxing the threshold, however, showed some differences between positive and negative feedback that overlapped current observations in thalamus and striatum. The current results were substantially stronger and more widespread than feedback valence differences in our preliminary analyses. Studies have identified positive and negative feedback differences in frontal and striatal regions (Bischoff-Grethe et al., 2009; Liu et al., 2007; van Duijvenvoorde et al., 2008), so it remains a possibility that positive and negative feedback contribute differentially to 1^st Trial feedback responses.

Across a network of cortical and subcortical regions we observed a selective reduction in activation in response to feedback during single-trial learning as predicted by animal and human learning studies but not yet demonstrated. These changes occurred in correlation with a discrete jump in associative knowledge at the conclusion of the first trial. This pattern implies that the bilateral middle frontal gyrus and striatum, and possibly other regions, participate in performance evaluation and that activity in this network serves to bind stimulus cue, motor response, and outcome information into an associative response rule that can be accessed during the decision-making process on future trials.

4. METHODS

4.1 Participants

Twenty-one healthy volunteers were recruited from the greater Cincinnati area and received monetary compensation for their participation. All participants gave written informed consent and were screened for MR safety according to standard, established Center for Imaging Research (CIR) protocols and University of Cincinnati Institutional Review Board guidelines. One participant was excluded from analysis due to accuracy of less than 50% in two scanner runs resulting from a miscommunication in explaining the task. Among the remaining 20 subjects, the mean age was 26±3 years, and included 14 women and 6 men. Nineteen participants were right handed with an average score of 16 on a 10-question modified Edinburgh handedness survey (Eliassen et al., 2001; Oldfield 1971) and one was left-handed with a score of -12.

4.2 Behavioral Paradigm

The behavioral experiment was programmed in E-Prime (www.pstnet.com). All participants wore MR-compatible VGA goggles and headphones equipped with a microphone (Resonance Technologies, Northridge, CA; www.mrivideo.com). The MRI system triggered the start of the E-prime paradigm in order to ensure precise timing of the task with respect to image acquisition. Participants responded by pressing with their right thumb one of two buttons on a custom-built MR-compatible button box. E-prime recorded task and behavioral data to a file, including the timing of stimulus and feedback presentations, responses, and outcomes of each trial.

The behavioral paradigm consisted of a control task and a learning task (Fig. 1A). For the learning task, participants viewed easily nameable color pictures, e.g., butterfly, fork (Rossion and Pourtois 2004), and had to learn by trial-and-error to associate each picture with button “1” or “2” on the button box. For the control task participants viewed the digits 1 or 2 and pressed the corresponding button. All trials began with the appearance of a fixation X in the center of the screen. After a variable delay of 0.5 to 3.5 sec a picture or control digit appeared on the screen, and subjects had to make a response. After 2.125 sec the screen went blank for a variable delay of 0.5 to 3.5 sec following which performance feedback was provided for a fixed duration. If a subject failed to respond before the screen went blank then an uninformative “?” was presented as feedback. Just before entering the MRI scanner, each participant was given oral and written instructions and brief training with pictures different from those used during scanning.

Figure 1.

Behavioral paradigm. (A) Time lapse schematic of associative learning and control tasks. At the beginning of a trial a fixation x appears for a variable interval. The fixation screen is followed by the presentation of a stimulus cue (picture or digit). In the associative task participants view an easily nameable color stimulus and must respond while the stimulus remains on screen by pressing one of two buttons. Presentation of the stimulus cue is followed by a blank screen for a variable delay, after which, participants receive performance feedback. The actual feedback symbols that are presented depend upon the participant’s performance and which Award Schedule is in effect (Table 1). The two feedback symbols used in this schematic figure are the positive “+” and negative “0” feedback symbols from Award Schedule 3 in Table 1. Display times are below each slide. (B) During a single scanner run, 16 control trials were followed by 32 associative learning trials. Total elapsed time approximately 6 min 18 sec.

We administered six scanner runs to each participant. During each scanner run participants performed both the control (16 trials) and learning task (32 trials; Fig. 1B). In each run participants had to learn the responses to four pictures. A different set of pictures was used for each run. A given picture was presented eight times in a run. The order of picture presentation was pseudorandom according to Latin Squares. Six scanner runs were administered in order to address two issues. First, we were interested in having as many rule acquisition trials as possible (up to 24) within a reasonable scan time (one hour) in order to maximize statistical power while still allowing for a sufficient number of rehearsal trials (up to 7) for each stimulus cue in order to allow novelty effects to dissipate. Second, we were concerned about learning effects being explained by a fixed outcome value so we provided a different monetary amount as performance feedback in each run. Monetary awards and penalties were used as performance feedback although not strictly associated with correct and incorrect performance, respectively. The monetary outcome values ranged from the -$0.75 to +$0.75 in $0.25 increments. The outcome for a correct response was always $0.25 better than the outcome for an incorrect one. The six different feedback schedules that we used ranged from the best schedule, awarding $0.75 for a correct response or $0.50 for incorrect response to the worst, penalizing $0.50 for a correct response or $0.75 for an incorrect response (Table 1). The order of feedback schedules was randomized across subjects according to Latin Squares. We used a combination of +, -, and 0 as feedback symbols in the task. Each plus or minus sign indicated positive or negative $0.25. For example ++ provided as feedback indicated a +$0.50 monetary award, and --- indicated a penalty of -$0.75. Participants were informed that monetary outcomes were in effect only during the learning trials and not the control trials. The feedback symbols, however, were the same within a run for correct and incorrect responses regardless of whether it was a learning or control trial. Participants were told that they would be paid $20. They were also told that they could win up to $24 additional depending upon performance. All participants, however, were paid $50 plus parking costs.

Table 1.

Award schedules used to control for outcome value effects.

	Correct response		Incorrect response
Award Schedule	Onscreen Symbol	Monetary Outcome	Onscreen Symbol.	Monetary Outcome
1	+++	+$0.75	++	+$0.50
2	++	+$0.50	+	+$0.25
3	+	+$0.25	0	$0
4	0	$0	−	−$0.25
5	−	−$0.25	− −	−$0.50
6	− −	−$0.50	− − −	−$0.75

Onscreeen Symbols appeared on the computer screen at the end of each trial to indicate performance. Monetary Outcome was the amount awarded or penalized for a specific response. No monetary information appeared on screen during the feedback presentation interval.

4.3 Behavioral Data Analysis

In order to identify changes in brain activation from rule acquisition on the first trial to rule rehearsal on the second and later trials, we grouped learning trials into five categories. These categories were used for the analyses of behavioral performance and brain activation. Each picture was presented eight times in a scanner run, so perfect performance yielded one learning trial and seven rehearsal trials. “1st Trials” included all trials where a picture was seen for the first time and followed consecutively by at least three correct trials. Following 1st Trials, the next three consecutively correct trials were assigned to the categories “Correct 2^nd Trials,” “Correct 3^rd Trials,” and “Correct 4^th Trials.” Subsequent to these categories all learning trials that were correct five or more times were grouped into a single category called “Correct 5-8^th Trials.” Control trials were grouped separately. It is important to note that although 1^st Trial outcomes could be correct or incorrect by chance, the same associative information was learned either way so trials that resulted in both positive and negative feedback were included in “1^st Trials.” Also important is the fact that if an error occurred after the first trial and before the fifth, none of the first four trials was included in the 1^st to Correct 4^th categories. Behavioral analysis of reaction times and analysis of fMRI data used these six categories. For the fMRI analysis, stimulus and feedback events were modeled separately for each of the category. This gave us a total of twelve events of interest, stimulus and feedback events for 1st Trials, Correct 2^nd Trials, Correct 3^rd Trials, Correct 4^th Trials, Correct 5-8^th Trials and Control Trials. By separately modeling each of the first four correct trials we precisely equated the learning experience across subjects, because, for instance, all Correct 4^th trials were preceded by a learning trial and two consecutively correct rehearsal trials. No indirect, or latent, measures of learning need to be calculated in order to establish how much learning has taken place or at what rate in different individuals. No separate regressors were included for the behavioral responses in any condition. Behavioral responses are tightly time-locked to stimulus presentation, and stimulus events therefore contain information about the combined cognitive processes supporting decision-making and response execution.

Control trials were presented at the beginning of each run rather than interleaved within the task. In prior work on this topic (Eliassen et al., 2003) we observed during preliminary analyses that control trials at the beginning and end of the paradigm had different magnitudes of the BOLD response, We remained concerned in this study that novelty effects might occur at the beginning of each scanner run. BOLD responses associated with learning are also largest at the beginning of learning. In order to reduce the potential for novelty effects to confound learning effects, learning trials commenced in the middle of each scanner run rather than at the beginning. Although not within the scope of this study of single-trial learning, in future work we may compare Control and Correct 5-8^th trials to understand the differences between association rehearsal and control trials. For this reason we attempted to match the number of trials within these two categories during the analysis phase (see results for the average number of trials within the different categories).

4.4 Image Acquisition

Brain images were acquired on a whole-body Varian Inova 4 Tesla MR system at the University of Cincinnati College of Medicine’s Center for Imaging Research (CIR). A high-resolution T1-weighted anatomical volume was acquired using a Modified Driven Equilibrium Fourier Transform acquisition (MDEFT; (Lee et al., 1995)): T_MD=1.1 s, TR=13 ms, TE=6 ms, FOV=25.6 × 19.2 × 19.2 cm, matrix 256 × 192 × 96 pixels, voxel resolution 1×1×2mm, flip angle 20 degrees, zero-filled to 1×1×1mm during reconstruction. For fMRI measurements we acquired six runs of T2*-weighted gradient-echo echoplanar images (EPI) consisting of 35 contiguous 5 mm coronal slices covering the entire brain (TR/TE=3000/29 msec, FOV=208 × 208 mm, matrix 64 × 64, slice-thickness 5 mm, flip angle 75 degrees, resolution 3.25×3.25×5mm) utilizing the BOLD contrast mechanism (Kwong et al., 1992; Ogawa et al., 1992) and a multi-echo reference scan for correction of geometric distortion and Nyquist ghost (Schmithorst et al., 2001). Two acquisitions were acquired prior to a trigger at the beginning of the behavioral paradigm, and these images were discarded to account for magnetization equlibrium effects. A total of 888 volumes were acquired from each subject for analysis.

4.5 MRI Analysis

Raw MRI data were reconstructed with 2D Hamming filtering in the XY plane using in-house software developed in IDL (www.ittvis.com). Images were subsequently processed, analyzed, and visualized with AFNI (Analysis of Functional Neuroimages, afni.nimh.nih.gov; (Cox 1996; Cox and Hyde 1997)). The six fMRI runs were motion corrected to the third image of the first run using a six-parameter rigid-body transformation with Fourier interpolation (Cox and Jesmanowicz 1999). The MDEFT image was normalized to Talairach space using tools in AFNI to match each subject’s image to the International Consortium for Brain Mapping’s ICBM452 template from UCLA’s Laboratory of Neuroimaging (www.loni.ucla.edu). The EPI datasets were spatially smoothed with a 6 mm full-width, half-maximum Gaussian filter, then normalized by adopting the MDEFT transform, and resampled to 3×3×3mm. The application of Hamming filtering at reconstruction plus smoothing after normalization resulted in effective smoothing of 9mm in the xy imaging plane and 6mm in the z-slice plane. The Monte Carlo simulations used to estimate statistical significance with p-value and cluster size thresholds took this smoothing into account.

For each participant, the list of times for each event type was convolved with a canonical hemodynamic response function (HRF) in AFNI with user specified parameters including a delay time (2sec), rise time (4 sec), and fall time (4 sec) based on the observed timing of hemodynamic responses in our previous work (Eliassen et al., 2003). In addition to the behavioral events, polynomials up to third order and the motion corrections parameters were fit to each run to account for signal drift. The fit coefficients for the events were determined from the fMRI data using linear regression with AFNI’s 3dREMLfit, which estimates brain activation and accounts for serial autocorrelations in the fMRI time series. Fit coefficients for the behavioral events were entered as dependent measures in statistical analyses to identify brain activation effects of interest. In addition to this fixed shape HRF, we also modeled the behavioral events in the MRI data with a time-varying HRF. For this purpose we used a set of three sinusoidal basis functions for each event type, which allowed us to examine the time course of the BOLD response to each event type. Time courses from this secondary deconvolution analysis were extracted from the regions of interest in Fig. 3 and plotted in Fig. 4 in order to demonstrate that the temporal signal shows the same effects of learning suggested by the results obtained with the canonical HRF.

4.6 Statistical Analyses

We conducted statistical analyses of reaction times (RTs) and fMRI data. For the RT data, we evaluated the change in response times across all levels of experience (1st, Correct 2^nd, Correct 3^rd trials, etc.), specifically including a contrast between 1^st Trials to Correct 2^nd Trials. Behavioral analyses were conducted using a mixed effects analysis approach in Statistical Analysis Software (SAS, SAS Institute, Cary, NC). For the fMRI data we used AFNI’s GroupAna Matlab routines to conduct repeated measures Analysis of Variance (ANOVA). The fit coefficients from the deconvolution procedure were used as dependent measures in this analysis. We included subject as a random factor and event type (2 levels: stimulus or feedback) as a fixed factor. Experience (six levels: 1st, Correct 2^nd, Correct 3^rd, etc.) was included as a repeated measure. For the fMRI data, contrasts were included to examine a number of learning related changes separately for stimulus or feedback events. Most importantly, we included an interaction term that addressed our main hypothesis. This interaction term appraised whether the change in activity between 1^st Trials and Correct 2^nd Trials was the same or different for stimulus and feedback events. Because the utility of the feedback in guiding future behavior changes after learning on the first trial, we hypothesized that there would be more pronounced reductions in activity for feedback than for stimulus events in the frontal, parietal and striatal regions.

For all brain imaging comparisons we considered as significant a corrected p-value of 0.01, using a voxel-level p-value ≤ 0.001 and a cluster threshold of 30 voxels. These thresholds were determined according to Monte Carlo simulations using tools in AFNI (Forman et al., 1995; Friston et al., 1994; Xiong et al., 1995). In order to identify the locations of activation clusters we used the Talairach daemon (Lancaster et al., 2000) and the automated anatomical labels of the single-subject high-resolution T1 volume from the Montreal Neurological Institute (Tzourio-Mazoyer et al., 2002) as implemented in AFNI.

Highlights.

• We studied the brain activity during single trial associative learning using fMRI.

• We predicted frontal and striatal changes in feedback processing after the first trial.

• We observed reductions in brain activity between first and second trial feedback.

• Activity pattern suggests these structures bind stimulus, response, and outcome.