Positive and negative modulation of word learning by reward anticipation

Abstract

Recent evidence from neuroscience indicates that the anticipation of external rewards may enhance declarative memory consolidation by increasing dopaminergic‐modulated plasticity in the hippocampus. A number of studies in psychology, however, have shown that external rewards may have null, or even negative, effects on learning. To shed light on this issue, we developed a novel task, in which native Japanese speakers were rewarded to learn unknown English words inside a functional MRI scanner. Rewards had no effect on recall performance unless we used a rating of reward‐induced anxiety as a covariate. In this case, for highly rewarded words, we found a negative correlation between recall performance and anxiety ratings. For those words, high recall performance and low anxiety ratings were associated with enhanced activity in the midbrain dopaminergic centers, the hippocampus, and the amygdala. On the other hand, low recall performance and high anxiety ratings were associated with enhanced activity in the anterior cingulate and middle frontal gyrus, brain regions that have been shown to be involved with anxiety and divided attention, respectively. A connectivity analysis indicated positive functional connectivity between the midbrain dopaminergic centers and both the hippocampus and the amygdala, as well as negative connectivity between the anterior cingulate and the amygdala. Thus, both our behavioral and imaging results suggest that the anticipation of rewards can, depending on the individual level of reward‐induced anxiety, have either a beneficial effect or a negative effect on word learning. Hum Brain Mapp 2008. © 2007 Wiley‐Liss, Inc.

INTRODUCTION

It is uncertain whether the anticipation of external rewards has a negative or positive effect on learning, as two seemingly opposing views have been proposed. On one hand, a number of studies in neuroscience have shown that reward and memory systems are strongly interconnected in the brain, indicating a positive effect of reward anticipation in modulating encoding. Specifically, dopamine neurons in the substantia nigra pars compacta and ventral tegmental area (SN/VTA), which are known to be critically involved in reward prediction [Kirsch et al., 2003; Schultz, 1998], send substantial projections to the CA1 part of the hippocampus [Thierry et al., 2000]. Dopamine controls a major source of sensory inputs to the CA1, and enhances long‐term potentiation and prevents depotentiation [Benardo and Prince, 1982; Frey et al., 1989; 1990; 1991; Gribkoff and Ashe, 1984; Li et al., 2003; Otmakhova and Lisman, 1999]. In rats, selective lesion of the dopaminergic neurons that innervate the hippocampus seriously impairs spatial memory [Gasbarri et al., 1997]. In humans, dopaminergic midbrain structures and the hippocampus show greater activation for rewarded versus neutral items that were later recognized [Adcock et al., 2006; Wittmann et al., 2005]. Thus, there is good evidence at the neural level that reward anticipation increases dopamine release in the hippocampus, which in turn enhances plasticity and encoding.

On the other hand, research in psychology on the benefit of extrinsic rewards on learning has been inconclusive. Unrewarded rats learn to get out of a maze as well as do rewarded rats [Tolman and Honzik, 1930]. Performance dependent rewards can enhance subjects attention during encoding or rehearsal for reward‐associated items when compared with neutral items [Loftus, 1972], but when these item‐specific attention biases are minimized, the effects of rewards are small [Nilsson, 1987]. Furthermore, although this is a debated subject [Cameron and Pierce, 1994; Sansone and Harackiewicz, 2000], a large number of human studies have found a general null, or even detrimental, effect of external rewards on performance and learning—see [Deci et al., 1999; Sansone and Harackiewicz, 2000] for review. A possible explanation to this effect has been proposed as the “competing response model” [Reiss and Sushinsky, 1975], according to which the anticipation of performance‐dependent rewards leads to divided attention, which in turn reduces encoding. Furthermore, the possibility of a lack of reward acquisition contingent to failure can be perceived as a threat that can generate anxiety [Harackiewicz and Sansone, 2000], which further divide attention [Mogg and Bradley, 1998], and reduces performance. Studies have reported that decreased recall performance is correlated with increased anxiety [Andreoletti et al., 2006; the findings were for middle‐aged and older adults] and conversely that increased working memory performance is correlated with reduced anxiety [Hudetz et al., 2004].

Thus, from a behavioral standpoint, we hypothesized that reward anticipation has a beneficial modulatory effect on encoding, and that this beneficial effect can be canceled by reward‐induced anxiety. From a neural standpoint, we hypothesized that reward anticipation enhances midbrain (SN/VTA) dopaminergic neurons activity via modulation from amygdala activity, resulting in enhanced hippocampal encoding. We further hypothesized that, in individuals who experience high levels of reward‐induced anxiety, an antagonistic mechanism is simultaneously at work that reduces hippocampal encoding.

In support of our hypothesis, memory formation has been linked to increased dopamine release in the amygdala [Fried et al., 2001], increased SN/VTA activity [Heckers et al., 2002; Schott et al., 2004], and increased hippocampal activity [Meltzer and Constable, 2005; Schott et al., 2004; Wittmann et al., 2005]. Amygdala activity has been found to be related to encoding [Kensinger and Schacter, 2006; Phelps, 2004] and to predict working memory performance [Schaefer et al., 2006]. Furthermore, the amygdala is in part involved with mediating the function of positive rewards in eliciting goal‐directed behavior [Elliott et al., 2004; Gottfried et al., 2003]. Conversely, the amygdala also shows a decrease in activity to negative emotions [Ernst et al., 2005; Etkin et al., 2006]. Projections from the Amygdala to the SN/VTA [Fudge and Haber, 2000] may modulate the dopaminergic projections between SN/VTA and the hippocampus. We put forward here that in cases of low anxiety and reward anticipation, the amygdala facilitates dopaminergic release in SN/VTA that modulates (enhances) encoding in the hippocampus. Conversely, in the case of high anxiety there is a decrease of activity in the amygdala inhibiting dopaminergic release in SN/VTA that diminishes the modulatory effect it has on encoding in the hippocampus. The modulatory effect of the amygdala may be mediated by activity from the anterior cingulate cortex (ACC). The ACC has been associated with anticipatory anxiety [Chua et al., 1999], regulating cognitive and emotional processing [Bush et al., 2000], as well as relating actions to their consequences [Rushworth et al., 2004] and has an inhibitory influence on amygdala activity [Etkin et al., 2006].

To test these hypotheses, we developed a novel task, in which native Japanese speakers were differentially rewarded (300, 100, and 0 yen) to learn English words inside an functional MRI (fMRI) scanner (see Material and Methods for details). The experiment consisted of a pretest taken outside the fMRI scanner, followed by an encoding phase inside the scanner, then a distracter task inside the scanner, and a delayed recall posttest outside the scanner. Then, to quantify the degree of anxiety associated with each reward condition (reward‐induced anxiety), qualitative ratings of reward‐induced anxiety were obtained separately for each of the conditions. Subjects were then awarded a monetary sum corresponding to their performance on the post‐test.

MATERIALS AND METHODS

Subjects

Fifteen 23‐ to 34‐year‐old (mean, 26 years, Std. 3.9 years) right‐handed female native Japanese speakers with some English ability (at least 6 years of classes in junior and senior high school) from a predominantly all female school participated in this study. Only female subjects were available for this study therefore the results may relate to females only. Further research needs to be conducted to determine whether the results reported here generalize to males and/or whether there are gender differences. Only subjects who had a qualifying score on the pretest were included in the study. In addition to a variable reward amount based on learning performance (see later), subjects received a fixed amount for their participation, and gave written informed consent for experimental procedures, approved by the ATR Human Subject Review Committee. The subjects were instructed at the beginning of the experiment that they could earn a maximum of 9,840 yen and a minimum of 5,000 yen for their participation.

Procedure and Stimuli

The experiment consisted of a vocabulary pretest taken outside the fMRI scanner, an encoding phase inside the scanner, a distracter task inside the scanner, and a posttest outside the scanner. The pretest involved sequential presentation (using Matlab 6.0) of 82 English words (Appendix). For each word, the subject needed to type the corresponding Japanese word and a familiarity rating ranging from 0 to 5 (0 meaning no familiarity and 5 meaning highly familiar). Subjects were told to respond quickly and were informed that they could earn 10 yen for each correct answer on the pretest. A native Japanese speaker proficient in English immediately corrected the pretest. Only subjects who had at least 11 correct responses on the pretest with familiarity ratings of 5 and at least 33 incorrect responses with familiarity ratings of 1 or 0 were included in the study. Responses on which the subject took longer than 30 s to answer were discarded. Based on the results of the pretest, four groups of 10 words were created for the four conditions: 300 yen reward unknown words, 100 yen reward unknown words, 0 yen reward unknown word, and unrewarded known words.

An attempt was made to balance various aspects of the stimuli in experimental conditions. A native Japanese speaker fluent in English chose and classified the 82 words, in 30 very easy words, and 52 difficult words. All words used in the experiment were concrete nouns. The words selected for each of the experimental conditions for each subject were based on their own pretest results. The English words ranged from 4 to 7 letters (mean = 5.95; Std. = 0.88) and were 1 to 3 syllables long (mean = 2.01; Std. = 0.61). The corresponding Japanese translation words ranged from 2 to 3 Kanji characters (otherwise known as Chinese characters), or hiragana (one of the two Japanese phonetics system) when appropriate, (mean = 2.25; Std. = 0.44) and 2 to 5 mora sounds (mean = 3.81; Std. = 0.80). The number of English letters and syllables as well as the number of Kanji and hiragana in the corresponding Japanese translation were balanced for easy and difficult words (as well as across the different reward conditions) during composition of the word lists for each condition and subject. Post hoc analysis reveals that the complexity of the Kanji characters (defined by the number of strokes needed to write the character) did not significantly differ between easy and difficult words (as well as across the different reward conditions) for all subjects except one (P < 0.05). There was no significant correlation between behavioral performance and complexity of the kanji characters composing the words. The English words were presented in lower case in all phases of the experiment. Japanese words were presented only during the encoding phase and were mostly in Kanji with some hiragana where appropriate.

Following the pretest, subjects were given the instructions for the encoding phase of the experiment. Subjects were informed of the reward associated with getting a correct answer on the post‐test for each of the four conditions (unknown words 300, 100, 0 yen; known words = 0 yen). The different potential reward values for learning an English word and its Japanese translation was represented by a different colored surrounding square frame (gold = 300 yen; silver = 100 yen; green = 0 yen). To increase subjects' motivation and ensure that subjects pay attention to the nonrewarded words, incorrect answers were penalized by 100 yen.

The encoding phase consisted of three sessions of about 5 and a half minutes each. In each session, the 40 English words and their corresponding Japanese translation were presented once. To avoid excess eye movement subjects were instructed to fixate their eyes on a small fixation cross in the center of the screen. Each English word and its corresponding translation was presented within the reward‐coded colored surrounding square frame for 6 s during which subjects were instructed to subvocally rehearse the association between the English word and Japanese translation. The English word was presented just above the fixation point and the Japanese translation was presented just below the fixation cross. The onset of presentation was synchronized with fMRI scanning, using Neurobehavioral System's Presentation software. Following the six‐second stimulus presentation, only the fixation cross remained on the screen for 1 s, after which the next stimulus was presented (interstimulus interval = 7 s).

The fMRI experiment was an event related design, in which the various items were presented pseudo‐randomly once per session, such that subjects were not able to predict the condition of the next stimulus. A different event order was determined for each of the three sessions for each subject.

After the encoding phase, a distracter task, which took place inside of the fMRI scanner, was given to ensure that subjects would not rehearse words from the experiment until the time of the post‐test. The distracter task involved verbally producing visually presented English syllables beginning with a /r/, /l/, or a vowel alone. Because English /r/ and /l/ are very difficult for native Japanese speakers to produce and perceive, and because stimuli were presented at a rapid pace (every 2.25 s), this task left no time to rehearse words from the encoding task.

In the 10‐min delayed post‐test, conducted outside of the fMRI scanner, the 40 English words were presented (using Matlab 6.0) sequentially. Subjects were required to type the corresponding Japanese word and then indicate the condition to which the stimuli belonged (300, 100, 0, known). A native Japanese speaker proficient in English immediately corrected the post‐test. After the posttest, subjects were asked to give a qualitative rating (0–5) of their degree of anxiety associated with each condition (0 = low anxiety; 5 = high anxiety). Specifically they were asked the following (translated from the Japanese): “Did you experience any anxiety while learning the English words? Please indicate the level of anxiety by a number (0‐low anxiety, 5‐high anxiety) for each Gold (300 yen worth), Silver (100 yen worth), Green (0 yen worth) and known words.” The experiment was then over, and subjects were given the monetary reward corresponding to their performance.

fMRI Data Collection, Preprocessing, and Analysis

For functional brain imaging, a Shimadzu–Marconi's Magnex Eclipse 1.5T PD250 scanner, located at the ATR Brain Activity Imaging Center, was used. Functional T2* weighted images were acquired using a gradient echo‐planar imaging sequence (echo time 48 ms; repetition time 2,000 ms; flip angle 90°). A total of 20 contiguous axial slices were acquired with a 3.75 × 3.75 × 5 mm³ voxel resolution covering the cerebrum and the top part of the cerebellum (TR = 2,000 ms). A total of 162 scans were taken for a single session. The first 6 scans were discarded. Images were preprocessed using programs within SPM2 (Wellcome Department of Cognitive Neurology, University College, London). Differences in acquisition time between slices were accounted for, images were realigned‐unwarped and spatially normalized to a standard space (default) using a template EPI image (2 × 2 × 2 mm³ voxels), and were smoothed using a 7.5 × 7.5 × 10 mm³ FWHM Gaussian kernel.

The data was assessed (SPM2) using a general linear model employing a boxcar function convolved with a haemodynamic response function (with time and dispersion derivatives). High pass filtering (cutoff period equal to twice the maximum difference in seconds between two occurrences of the same condition) was carried out to reduce the effects of extraneous variables (scanner drift, low frequency noise, etc.). Auto‐regression was used to correct for serial correlations.

Fixed effect analyses were conducted for each subject separately for the contrasts of interest (300 yen reward versus the 0 yen reward condition). A random effect one‐sample t‐test was conducted using as data the contrast estimate (300 yen versus 0 yen condition) for each subject. Additionally, a random effects multiple regression analysis was conducted using each subjects performance and anxiety scores for the high reward (300 yen) condition as predictors and each subjects contrast estimates for the 300 yen versus 0 yen condition as observations.

Region of Interest Analysis

The regions of interest were defined by MNI (Montréal Neurological Institute) coordinates given in previous research articles: Hippocampus (encoding) [Meltzer and Constable, 2005], (26, −24, −18) search radius 5 mm; Amygdala (reward based goal‐directed behavior) [Elliott et al., 2004], (−30, −2, −20) search radius 5 mm; SN/VTA (reward anticipation) [Kirsch et al., 2003], (−8, −18, −16) search radius 5 mm; Anterior cingulate (anticipatory anxiety) [Chua et al., 1999], (12, 26, 30) search radius 10 mm; and the right middle frontal gyrus (MFG) (divided attention) [Iidaka et al., 2000], (28, 28, 24) search radius 5 mm.

Functional Connectivity Analysis

Functional connectivity among brain regions of interest was investigated using a psycho‐physiologic interaction (PPI) analysis [Friston et al., 1997; Gitelman et al., 2003] (SPM2). Seed activity within the regions of interest upon which the PPI analysis was conducted was determined by a PCA of active voxels (P < 0.05) in a 5‐mm radius centered at the peak voxel based on the analysis given in Table II. Regions of interest included the hippocampus, amygdala, and ACC. After contrast images of the PPI for each subject was determined for the high reward when compared with no reward condition, a random effects multiple regression analysis was conducted using the same contrast as in the regression analysis.

Table II.

Multiple regression for the high reward vs. no reward condition: Small Volume Correction Analysis

Brain region	Contrasta			Contrastb
	T	pFDR	x,y,z	T	pFDR	x,y,z
Hippocampus	3.92	0.043	Right 22, −22, −16
SN/VTA	4.35	0.010	Left −8, −16, −18
Amygdala	4.07	0.018	Left −32, 0, −20
	3.29	0.037	Right 34, −2, −18
ACC				4.81	0.017	Right 14, 28, 26
MFG				3.82	0.013	Right 24, 26, 22

The MNI (x,y,z) coordinate denotes the peak voxel in the region of interest upon which the fitted linear responses were derived.

^a‘+’, performance; ‘−’, anxiety.

^b‘−’, performance; ‘+’, anxiety.

RESULTS

Behavioral Results

There was no main effect of rewards on post‐test recall performance (1 way ANOVA; F(1,14) = 0.28; P > 0.1; paired two‐tailed t‐tests: 300 yen > 0 yen: T = 0.63, P > 0.1; 300 yen > 100 yen: T = 0.70, P > 0.1; 100 yen > 0 yen: T = 0.01, P > 0.1)—see Figure 1A. However, qualitative anxiety ratings showed significant differences between the reward conditions and the no reward condition (1 way ANOVA; F(1,14) = 5.5, P < 0.05, using the Greenhouse Geisser correction for nonsphericity; paired two‐tailed t‐tests: 300 yen > 0 yen: T = 2.5, P < 0.05; 100 yen > 0 yen: T = 2.3, P < 0.05), but no significant difference between the rewarded conditions (300 yen > 100 yen: T = 0.69, P > 0.1)—see Figure 1B. An analysis of covariance of recall performance using anxiety rating as a covariate indicated a significant difference between the high reward condition and the no reward conditions (300 yen > 0 yen; F(1,14) = 9.21; P < 0.01), but no significant (n.s.) difference between the high reward and low reward conditions (300 yen > 100 yen; F(1,14) = 1.4; P > 0.1 n.s.), and no significant difference between the low reward and no reward conditions (100 yen > 0 yen; F(1,14) = 1.9; P > 0.1 n.s.). The 300‐yen condition showed a significant negative correlation between recall performance and anxiety rating (r = −0.65, P < 0.01)—see Figure 1C. Correlations were not significant for the other two conditions (100 yen: r = −0.25, P > 0.1 n.s.; 0 yen: r = 0.29, P > 0.1 n.s.). Correlation between anxiety rating for the different reward amounts are as follows: (300 yen and 100 yen: r = 0.79, P < 0.001); (100 yen and 0 yen: r = 0.27, P > 0.1 n.s.); (300 yen and 0 yen: r = 0.12, P > 0.1 n.s.).

Figure 1.

Behavioral results. (A) Percent correct recall performance for the 300, 100, and 0 yen reward conditions (standard errors are plotted above each bar). No significant difference was found between the conditions. (B) Qualitative rating score of reward‐induced anxiety for the 300, 100, and 0 reward conditions (standard errors are plotted above each bar). The 0 yen reward condition (*) showed significantly lower anxiety rating than both reward conditions. (C) Analysis of covariance of recall performance using anxiety as a covariate. The 300 yen condition showed a significant difference in recall performance when accounting for anxiety rating when compared with the 0 yen condition. The fitted linear plots of recall performance by anxiety rating are shown for the 300, 100, and 0 yen conditions. The 300 yen condition (*) showed a significant negative correlation between performance and anxiety rating (r = −0.65, P < 0.01).

Brain Imaging Results: One Sample t‐test

We then assessed the neural correlates of rewards. In line with our behavioral results, we found no significant difference (even at P < 0.05 one‐tailed uncorrected) in the hippocampus in a direct comparison between the high reward group and the no reward group. Only the left SN/VTA showed a trend toward activation for the direct comparison between the high reward and no reward conditions out of all of our regions of interest (P < 0.05 one‐tailed uncorrected).

Brain Imaging Results: Multiple Regression

To study the neural correlates of the observed negative correlation between performance and reward‐induced anxiety (Fig. 1) in the high reward condition but not in the no reward conditon, we performed a random effect multiple regression analysis (SPM2). The contrast estimate of the 300 yen minus the 0 yen condition for each subject was entered as observations into the multiple regression analyses with each subject's performance and anxiety scores for the 300 yen condition used as predictor variables. The use of the 300 yen minus the 0 yen contrast estimate in the multiple regression analysis controls for many within subject confounds, such as short term memory, working memory, processes related to reading, verbal processing, etc., so that the neural processes involved with reward related encoding can be specifically assessed. The multiple regression analysis in SPM2 allows for the predictor variables to be weighted for the analysis. To investigate the behavioral negative correlation between performance and reward‐induced anxiety we weighted the recall performance variable +1 and the anxiety variable −1. As can be seen in Table I and Figures 2 and 4A, there was significantly greater activity in the left and right SN/VTA, right hippocampus, left amygdala, left and right superior temporal gyrus/sulcus, left and right middle temporal gyrus, right temporal pole, right temporal occipital junction, left middle occipital gyrus, left and right inferior parietal lobule, and right postcentral gyrus (P < 0.005 one‐tailed uncorrected; spatial extent threshold = 50 voxels; in small regions such as the amygdala and midbrain areas a spatial extent threshold of 15 voxels was used. The only additional cluster of activity that was present was in the left parahipocampal gyrus). A small volume correction analysis for multiple comparisons, using the false discovery rate (FDR) procedure [Genovese et al., 2002], showed significant differential activity (FDR P < 0.05 one‐tailed) in the left SN/VTA, the right hippocampus, and the left and right amygdala (Figs. 2 and 4A, Table II).

Table I.

Multiple regression for the high reward vs. no reward condition

Brain region	Contrasta	Contrastb
Hippocampus	Right 28, −26, −6
	Right 22, −22, −16
Parahippocampus	Right 20, −40, −12
	Right 16, −30, −12
Amygdala	Left −32, −1, −17
SN/VTA	Left −6, −16, −14
	Right 10, −24, −11
Anterior cingulated		Right 14, 28, 24
Posterior cingulated		Left −4, −34, 18
Superior temporal gyrus/sulcus	Left −56, −22, 8
	Right 50, −40, 8
Middle temporal gyrus	Left −58, −36, 0
Temporal pole	Right 50, 4, −40
Temporal occipital junction	Right 52, −68, 2
Middle occipital gyrus	Left −18, −88, 10
Inferior parietal lobule	Left −48, −30, 48
	Right 58, −36, 40
	Right 46, −34, 48
Postcentral gyrus	Left −28, −32, 42
	Right 60, −24, 48
	Right 62, −8, 18
Middle frontal gyrus		Right 30, 24, 26

The MNI (x,y,z) coordinate denotes the peak voxel in the region of interest upon which the fitted linear responses were derived.

^a‘+’, performance; ‘−’, anxiety.

^b‘−’, performance; ‘+’, anxiety.

Figure 2.

Significant differential activity (P < 0.005 uncorrected; spatial extent threshold = 50 voxels) identified by random‐effects multiple regression analysis of the 300 yen relative to the 0 yen condition (weighted predictor variables: + performance, − anxiety for the 300 yen condition). Top: For each of the three regions of interest, SN/VTA, hippocampus, and amygdale, a coronal or sagittal slice is shown. Bottom: the fitted linear responses of the contrast estimate show how these brain regions exhibit increased activity for high performance and low anxiety. Horizontal slices covering the entire brain are given in Figure 4.

Figure 4.

(A) Horizontal slices covering activity found throughout the brain for the same analysis given in Figure 2: Significant differential activity (P < 0.005 uncorrected; spatial extent threshold = 50 voxels) identified by random‐effects multiple regression analysis of the 300 yen relative to the 0 yen condition (weighted predictor variables: + performance, − anxiety for the 300 yen condition). (B) Horizontal slices covering activity found throughout the brain for the same analysis given in Figure 3: Significant differential activity (P < 0.005 uncorrected; spatial extent threshold = 50 voxels) identified by random‐effects multiple regression analysis of the 300 yen relative to the 0 yen condition (weighted predictor variables: − performance, + anxiety for the 300 yen condition). MNI Z‐axis coordinates are given below the horizontal slices.

We then performed a second random effect multiple regression analysis for the high reward condition (300 yen) relative to the no reward condition (0 yen), in which we weighted the predictor variables −1 for recall performance and +1 for anxiety level. As can be seen in Table I and Figures 3 and 4B, there was significantly greater activity in the right anterior cingulate (ACC), left posterior cingulate, as well as right MFG (P < 0.005 one‐tailed uncorrected; spatial extent threshold = 50 voxels). A small volume correction analysis for multiple comparisons showed significant differential activity (FDR P < 0.05 one‐tailed) for the right ACC and the right MFG (Figs. 3 and 4B, Table II).

Figure 3.

Significant differential activity (P < 0.005 uncorrected; spatial extent threshold = 50 voxels) identified by random‐effects multiple regression analysis of the 300 yen relative to the 0 yen condition (weighted predictor variables: − performance, + anxiety for the 300 yen condition). Top: activity in the two regions of interest: anterior cingulate and middle frontal gyrus. Bottom: the fitted linear responses of the contrast estimate show how these brain regions exhibit increased activity for higher anxiety and lower performance. Horizontal slices covering the entire brain are given in Figure 4.

To further illustrate the opposite effects of recall performance and reward‐induced anxiety levels in the ROI for the high reward condition, we plotted the fitted response obtained as a function of recall performance or anxiety level ratings for each subject. Figure 2 shows the results for the left VTA/SN, right hippocampus, and left amygdala. Figure 3 shows the results for the right ACC and MFG. The left VTA/SN, the right hippocampus, and left amygdala showed significant positive correlation with performance and negative correlation with anxiety rating (Performance: left VTA/SN r = 0.78, P [lt] 0.001; right hippocampus: r = 0.76, P < 0.005; left amygdala: r = 0.77, P < 0.001. Anxiety rating: VTA/SN r = −0.71, P < 0.005; right hippocampus: r = −0.69, P < 0.005; left amygdala: r = −0.70, P < 0.005). Conversely, the right ACC and MFG showed positive correlation with anxiety rating and negative correlation with performance (Performance: ACC: r = −0.81, P < 0.0005; MFG: r = −0.66, P < 0.01; anxiety: ACC: r = 0.73, P < 0.005; MFG: r = 0.73, P < 0.005).

Brain Imaging Results: Psycho‐Physiological Interaction Analysis

We finally investigated functional connectivity among brain regions of interest, using a PPI analysis [Friston et al., 1997; Gitelman et al., 2003] (SPM2—see Material and Methods). The left SN/VTA showed significant positive functional connectivity (FDR P < 0.05 one‐tailed) with the right hippocampus and the left amygdala associated with higher performance and lower anxiety rating (predictor variable weighting: + performance, − anxiety) for the high reward (300 yen) versus low reward (0 yen) condition (Fig. 5). Conversely, the right ACC showed a trend towards negative connectivity (P < 0.05 one‐tailed uncorrected) with the left amygdala associated with lower performance and higher anxiety rating (predictor variable weighting: + performance, − anxiety) for the high reward (300 yen) versus low reward (0 yen) condition (Fig. 5).

Figure 5.

Results of the functional connectivity analysis between the regions of interest. Both the hippocampus and the amygdala show significant (P < 0.05 FDR corrected) positive connectivity (green) with the VTA/SN, but not with each other for high recall performance and low anxiety. The anterior cingulate shows a trend (P < 0.05 uncorrected) of inhibitory connectivity (red) with the amygdala for low recall performance and high anxiety.

DISCUSSION

Our behavioral and brain imaging results confirmed our hypotheses that reward anticipation has a beneficial modulatory effect on word learning, but only when reward‐generated anxiety levels are not too high. Specifically, our behavioral results did not show a significant difference in performance between the rewarded and unrewarded items, unless measures of the subjects' reward‐induced anxiety levels were taken into account: in this case, for highly rewarded words, we found a negative correlation between recall performance and qualitative anxiety rating. Thus, subjects less prone to reward‐induced anxiety tended to recalled items learned in the high reward condition better than those learned in the no reward condition. For those subjects more prone to reward‐induced anxiety, high rewards tended to be detrimental for encoding.

Our brain imaging results are also consistent with our hypothesis: for the high reward versus the no reward condition, high performance and low anxiety ratings correlated with activity in the SN/VTA, the hippocampus, and the amygdala (Figs. 2 and 4A, Tables I and II). Previous studies have shown involvement of these brain regions during encoding (SN/VTA and increased dopaminergic levels: [Adcock et al., 2006; Fried et al., 2001; Schott et al., 2004, , 2006; Wittmann et al., 2005]; Hippocampus: [Adcock et al., 2006; Casasanto et al., 2002; Greicius et al., 2003; Halsband et al., 2002; Maguire and Frith, 2004; Meltzer and Constable, 2005; Menon et al., 2000; Schott et al., 2004; Weis et al., 2004; Wittmann et al., 2005]; Amygdala: [Kensinger and Schacter, 2006; Phelps, 2004]). These same regions have also been implicated during processing of anticipatory reward (SN/VTA: [Adcock et al., 2006; Kirsch et al., 2003; Schultz, 1998; Wittmann et al., 2005]; Hippocampus: [Adcock et al., 2006; Wittmann et al., 2005]; Amygdala: [Elliott et al., 2004; Gottfried et al., 2003]). The amygdala in particular in some situations also shows a decrease in activity to reward omission [Ernst et al., 2005] and emotional conflict resolution [Etkin et al., 2006]. In our experiment, the possibility of not receiving a high reward (reward omission) may result in a decrease in amygdala activity during high anxiety.

Our results further support the hypothesis according to which high rewards can induce anxiety, which leads to divided attention and reduced encoding. The lack of an objective and on‐line measure of reward‐induced anxiety levels associated with each word during fMRI scanning, such as galvanic skin response is a potential shortcoming of our study. However, a corroboration of the validity of our qualitative anxiety rating arises from our finding of significant activation in the region of the right ACC associated with anxiety [Chua et al., 1999] for the multiple regression analysis of high reward (300 yen) relative to no reward (0 yen) weighting the predictor variables −1 for recall performance and +1 for anxiety (Figs. 3 and 4B, Tables I and II). Furthermore, in this same comparison, differential activity was also present in the region of the MFG (Figs. 3 and 4B, Tables I and II) associated with divided attention [Iidaka et al., 2000]. Thus, reward‐induced anxiety may divide attention between the reward cue and the item to be learned, and may cause less efficient encoding. This explanation is consistent with the “competing response model” [Reiss and Sushinsky, 1975], according to which the anticipation of performance‐dependent rewards leads to divided attention.

Consistent with the view that anxiety induced divided attention reduces performance under the high reward condition, the results of the multiple regression analysis weighting the predictor variable performance +1 and anxiety −1 (Figs. 2 and 4A, Table I) reveals trends in differential activity in brain regions involved with semantic processing (MTG: [Menon et al., 2000]; temporal pole; [Patterson et al., 2006]; inferior parietal lobe, including SMG, [Binder et al., 2003]), auditory/phonological processing (STG/MTG; [Lee, 2004; Scott, 2005], and orthographic—phonological processing (occipital‐temporal junction; [Thuy et al., 2004]). Enhanced encoding by relatively greater use of processing the semantic relationship between the Japanese word and corresponding English word is facilitated by reward when attention is not divided as a result of reward‐induced anxiety.

A direct comparison between the high‐reward with the no‐reward conditions, i.e., without considering performance and anxiety ratings, showed no increased activity in the hippocampus; only a trend in differential activity between conditions was present in the left SN/VTA. Thus, our results may seem at odds with those of Wittmann et al. [ 2005] and those of Adcock et al. [ 2006], who found greater activation in both hippocampus and SN/VTA for high rewarded versus neutral or low rewarded items that were later recognized. An important difference in our study and the Wittmann et al. [ 2005] study is the dissociation of the reward and the item to be implicitly learned in the Wittmann et al. [ 2005] study, and the direct relationship between anticipatory reward and the item to be explicitly learned in our study. The detrimental effect of anxiety on modulation of encoding by anticipatory reward (as found in our study) may not be present when the task that is rewarded is not directly related to the encoding of the item to be learned (as in [Wittmann et al., 2005]. However, Adcock et al. [ 2006] found performance related changes in the hippocampus and SN/VTA in an explicit learning task such as ours. A major difference between the two studies however is the timing of reward cue and items to be learned. In our study the reward cue is given simultaneously with the item to be learned, while in the Adcock et al. [ 2006] study the reward cue is presented several seconds before the item to be learned. Simultaneous presentation may result in greater anxiety related divided attention, which can have a detrimental effect on performance according to the “competing response model” [Reiss and Sushinsky, 1975]. Additionally, presentation of the reward cue before the item to be learned may result in greater item‐specific attention biases that mediate enhanced performance [Loftus, 1972; Nilsson, 1987]. Another difference between our study and that of Adcock et al. [ 2006] and Wittmann et al. [ 2005] is the use of all female subjects in our study whereas the other studies used predominantly males. It is possible (however we believe this to be unlikely) that the divergence in our findings from theirs is that the interaction of performance with reward anxiety exists only for females.

The connectivity analysis revealed that the SN/VTA showed significant functional connectivity to both the hippocampus (consistent with findings by [Adcock et al., 2006]) and the amygdala, for the high reward versus (300 yen) the no reward (0 yen) condition weighting the predictor variables +1 for recall performance and −1 for anxiety. Although one cannot determine the reciprocality or causal influence that one brain region has over another with functional connectivity analysis, this pattern of connectivity suggest that the projections from the amygdala to the SN/VTA [Fudge and Haber, 2000] may modulate the dopaminergic projections between SN/VTA and the hippocampus [Gasbarri et al., 1996] and facilitate encoding. Further, the amygdala may be modulated by anxiety, as we found a trend in inhibitory connectivity between the ACC and the amygdala, but not to the other regions (Fig. 5). This is consistent with the known anatomical connectivity between the anterior cingulate and the amygdala [Cunningham et al., 2002]. Thus, our results suggest that reward‐generated anxiety, may not only result in dividing attention, but also may directly reduce the value of the anticipated reward, as coded by the dopaminergic neurons. In this way, it would directly reduce the beneficial effect of reward anticipation on hippocampal encoding.

It is interesting to point out that connectivity between the rostral ACC and the amygdala are important in resolving emotional conflict [Etkin et al., 2006]. Conflict‐related Rostral ACC activity is correlated with a simultaneous reduction of amygdalar activity. Activity in the amygdala has also been shown to be implicated with encoding [Kensinger and Schacter, 2006; Phelps 2004] and working memory performance [Schaefer et al., 2006]. It is reasonable to conjecture that anxiety induced activity in the ACC found in our study may inhibit amygdala reward related activity that is important for facilitating dopamine release in SN/VTA leading to enhanced hippocampal encoding.

Our study gives an account to the discrepancies observed in neuroscience experiments and in psychological behavioral experiments on the effects of rewards on declarative memory encoding. On one hand, mounting evidence suggests that rewards enhance declarative learning via dopaminergic release in the hippocampus via pathways with the VTA/SN [Adcock et al., 2006; Schott et al., 2004; Wittmann et al., 2005]. On the other hand, these results are somewhat at odds with a large number of behavioral studies that show null or even detrimental effects of rewards on performance and learning [Deci et al., 1999; Sansone and Harackiewicz, 2000]. Consistent with the “competing response model” [Reiss and Sushinsky, 1975], our results suggest that under conditions of high rewards and low anxiety (less divided attention), increased SN/VTA activity, via increased amygdala activity, enhances dopaminergic release in the hippocampus. However, under conditions of high anxiety, the activity in the amygdala is suppressed by inhibitory connections from the ACC, in turn reducing SN/VTA activation, and thus reducing the dopaminergic release in the hippocampus. In sum, both our behavioral and imaging results suggest that, in learning situations, performance‐based rewards should be used parsimoniously, as rewards can generate anxiety that can cancel the potential benefit of rewards on encoding.

Word List