Neuronal Signals in the Monkey Basolateral Amygdala during Reward Schedules

Abstract

The amygdala is critical for connecting emotional reactions with environmental events. We recorded neurons from the basolateral complex of two monkeys while they performed visually cued schedules of sequential color discrimination trials, with both valid and random cues. When the cues were valid, the visual cue, which was present throughout each trial, indicated how many trials remained to be successfully completed before a reward. Seventy-six percent of recorded neurons showed response selectivity, with the selectivity depending on some aspects of the current schedule. After a reward, when the monkeys knew that the upcoming cue would be valid, 88 of 246 (36%) neurons responded between schedules, seemingly anticipating the receiving information about the upcoming schedule length. When the cue appeared, 102 of 246 (41%) neurons became selective, at this point encoding information about whether the current trial was the only trial required or how many more trials are needed to obtain a reward. These cue-related responses had a median latency of 120 ms (just between the latencies in inferior temporal visual area TE and perirhinal cortex). When the monkey was releasing a touch bar to complete the trial correctly, 71 of 246 (29%) neurons responded, with responses in the rewarded trials being similar no matter which schedule was ending, thus being sensitive to the reward contingency. Finally, 39 of 246 (16%) neurons responded around the reward. We suggest that basolateral amygdala, by anticipating and then delineating the schedule and representing reward contingency, provide contextual information that is important for adjusting motivational level as a function of immediate behavior goals.

Introduction

The amygdala is important in generating emotional context for situations and events (LeDoux, 2000; Everitt et al., 2003; Phillips et al., 2003). For example, functional neuroimaging studies in humans show that the amygdala is activated during recognition of fearful and happy facial expressions (Morris et al., 2002; Hare et al., 2005). The amygdala also appears to be involved in evaluating the reward value of a stimulus (Malkova et al., 1997; Parkinson et al., 2001). All of these findings are consistent with the view that the amygdala is important for assessing the affective (or emotional) valence of a stimulus and/or production of a specific affective state (Everitt et al., 2003; Phillips et al., 2003).

We have been studying how the brain uses external stimuli, particularly visual cues, to predict how much work (compare with number of trials) is needed to get a reward (visually cued reward schedule task) (Bowman et al., 1996). The visual cue, which predicts the work load remaining in the current schedule, affects the monkey's motivational and probably emotional states; because the visual cue progressively indicates that more work remains, the error rates and reaction times increase. The increases in error rates and reaction times indicate the monkeys are less motivated. This decrease in motivation in turn shows that the cues indicating that more work remains negatively affect the monkey's emotional state in that the monkeys become less inclined to complete the current trial correctly. Single neuron recording studies have identified robust signals related to this type of reward expectancy in the ventral striatum (Shidara et al., 1998), the perirhinal cortex (Liu and Richmond, 2000), and the anterior cingulate cortex (Shidara and Richmond, 2002).

Here we report on single neuronal recordings from monkeys performing this reward schedule task in the basolateral complex of the amygdala, a region that receives input from the anterior temporal lobe, including area TE (Cheng et al., 1997) and rhinal cortex (Stefanacci et al., 1996), and is high in dopamine (Sadikot and Parent, 1990). The signals carried by amygdala neurons, by coding for schedule and the reward contingency, could be used to resolve ambiguity about the upcoming reward. The anatomical connections and timing relations of the signals suggest that the amygdala may be part of a temporal lobe network that takes information about stimulus identity from temporal lobe sensory areas and generates signals that interpret the external stimuli or events by providing an emotional context, allowing the animal to organize its behavior.

Materials and Methods

Animals, behavioral task, and visual stimuli

Two adult rhesus monkeys (Macaca mulatta, 9 and 5 kg, one male and one female, respectively) were used. During our experiments, the monkey squatted in a primate chair facing a 21 inch computer monitor.

The monkeys were first trained to perform a sequential red-to-green color discrimination task (Fig. 1 A). The monkeys were trained without a fixation requirement. A display was 43.5 cm in front of the monkey. A black-and-white random dot background covered the whole screen of the display.

Figure 1.

Visually cued reward schedule task. A, Visually cued two-trial reward schedule. Time sequence of stimuli in an individual color discrimination trial is shown. Visual stimuli are centered. B, Visual cues.

When the monkey made contact with a touch lever to initiate a trial, a visual cue (rectangle 38 × 2° in visual angle) appeared in the center of the screen. Four hundred milliseconds later, a small red spot (0.3 × 0.3°) appeared on the center of the visual cue. After a random period between 1 and 2 s, the color changed to green. If the monkeys released the touch lever within a 0.2-1.3 s period after the appearance of the green spot, the spot became blue, indicating that a correct response had occurred. After another 100-200 ms, the visual cue and the blue spot disappeared, and a drop of liquid was delivered. The intertrial interval was 1.1 s.

If the monkey did not release the bar within the designated period (i.e., within a 0.2-1.3 s period after the green spot onset), an error was registered, the stimuli disappeared, and the intertrial interval before the next trial started.

After the monkeys learned to perform the color discrimination task (with 90% correct trials per session), we introduced the visually cued reward schedules. In the visually cued reward schedule task (Fig. 1 A), the monkey had to complete one, two, or three color discrimination trials correctly to obtain a reward, i.e., schedules of one, two, or three trials. We identify the trials in the schedules as schedule states. For labeling of the schedule states (Fig. 1), the numerator indicates a progress in the schedule, and the denominator indicates the schedule length (in number of trials). The schedule states are 1/1 for the one-trial schedule, 1/2 and 2/2 for the two-trial schedule, and 1/3, 2/3, and 3/3 for the three-trial schedule. A reward was delivered only after the successful completion of the final trial of a schedule. A new schedule of one, two, or three trials was picked randomly after a reward.

If the monkeys released the touch lever too early, i.e., earlier than 200 ms after the green spot appeared, or too late, i.e., later than 1.3 s after the green spot appeared, the stimuli all disappeared, an error was registered, and the normal intertrial interval began. There was no explicit punishment for errors, but the schedule state advanced only after a correct trial. After an error trial, the same trial of the same schedule state was repeated.

We tested valid and random cue conditions. In the valid cue condition, the cue became brighter as the rewarded trial approached and was brightest in the rewarded trial (Fig. 1). The brightness was proportionally related to the schedule state [pixel intensities in eight bit; 85 (33%) for the 1/3 state, 127 (50%) for the 1/2, 170 (67%) for the 2/3, and 255 (100%) for all three rewarded trials, namely the 1/1, 2/2, and 3/3 states] (Fig. 1 B). In the random cue condition, the reward schedule sequence remained, but the brightness of the cue was chosen randomly from the four brightness levels trial by trial so that there was no relationship between the cue brightness and the schedule state.

In another no-work, no-cue control condition, designated as free-reward condition, a reward was delivered at a regularly irregular rate of between 5 and 8 s (chosen pseudorandomly) with no work required, nor was any related stimulus shown.

Each task was run as a block of trials.

Surgical preparation

After the monkeys were trained to perform the reward schedule task, a cylinder for microelectrode recording and a head holder (Crist Inc., Hagerstown, MD) were affixed to the skull above the dorsal surface of the brain during an aseptic surgical procedure with the animal under isoflurane anesthesia. A scleral magnetic search coil for measuring eye movement was implanted during the same surgery (Robinson, 1963; Judge et al., 1980). The monkeys were given a 2 week postoperative recovery period. After recovery, the monkeys were given practice in the tasks with a loose fixation requirement.

Recording of single neuronal activity

Single-neuron data and behavioral data were collected while the monkeys performed the reward schedule task in both the valid and random cue conditions. A hydraulic microdrive (Narishige, Tokyo, Japan) or an electromechanical microdrive (Alpha Omega, Nazareth, Israel) was mounted on the recording cylinder, and tungsten microelectrodes [impedance, 1.5-1.7 MΩ; Frederick Haer Company (Bowdoinham, ME) or Roboz-Microprobe (Rockville, MD)] were inserted vertically through a stainless steel guide tube that was placed in a hole of a grid within a recording chamber (Crist et al., 1988) (Crist Inc.). Experimental control and data collection were performed by a personal computer, using the REX real-time data-acquisition program adapted for the QNX operating system (Hays et al., 1982). Single-neuron activity was isolated by discrimination using time-voltage window discriminator (Frederick Haer Company). All event and spike times were stored at 1 ms time resolution. All of the experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the Animal Care and Use Committee of the National Institute of Mental Health.

Recording sites localization

In each recording session, the depth of the recording area was measured from the upper tip of a guide tube. Before we changed the location of a guide tube in the grid, a magnetic resonance (MR) image was acquired to document the recording location (Saunders et al., 1990). A tungsten microelectrode was inserted into the monkey's brain to use as a landmark to indicate the recording location on the MR images (susceptibility artifact).

For histological examination of our recording sites, electrolytic microlesions (30 μA, 40 s) were made in the brain of one monkey (monkey A). One week later, the monkey was deeply anesthetized with sodium pentobarbital and perfused through the heart with saline and 10% Formalin fixative. The brain was sectioned into 50 μm slices in the coronal plane and stained with cresyl violet. The sites of guide tubes and microlesions were identified on the sections under microscopic examination. The histological reconstructions matched the locations determined using the MR, so the MR images alone were used for localization of recording sites in the second monkey.

Data analysis

Behavioral measure. Behavioral performances of the monkeys were measured using reaction times and error rates. Reaction times were defined to be the time interval from the appearance of the green spot (go signal) to the time of bar release.

Neuronal response measure. Neuronal responses were analyzed with respect to several discrete events (Fig. 1 A): touching a bar by the monkeys, appearance of the cue (cue), appearance of the red spot (wait), change of the red spot to a green spot (go), release of the bar by the monkey (bar release), change of the green spot to a blue spot (correct), the time when both the cue and the spot disappeared, and the time of reward apparatus activation, i.e., reward delivery, in the final trial of the schedule (reward). We counted number of spikes within different perievent periods as neuronal response measure.

Precue activity was measured by counting the number of spikes during a 300 ms period just before the cue onset. Control activity for the precue activity was measured during a 300 ms period starting 600 ms before the cue onset. Go/prebar-release activity was measured by counting the number of spikes during a 300 ms period just before the bar release. Reward-related response was measured by counting the number of spikes during a 400 ms period after the reward. Control activity for the reward-related response was measured during a 400 ms period just before the reward.

Conventional statistical procedures (mentioned in Results) were used to assess significance. All data analyses were done in the R statistical computing environment (Ihaka and Gentleman, 1996).

Response latency. The latency of the cue-related responses was determined by comparing spike counts before cue presentation (background activity) and spike counts in a test window that moved starting from the time of the cue onset. To determine response latency, we used two time windows with different durations. We first used a wider test window (100 ms) and then used a narrow test window (50 ms). Background activity was set as spike counts in a 100 ms period just before the cue onset for the 100 ms test window and as spike counts in a 50 ms period just before the cue onset for the 50 ms test window. The 100 ms sliding time window was moved in 5 ms steps, and the 50 ms window was moved in 3 ms steps. Using the 100 ms time window, we found first four consecutive windows that showed significantly different activity compared with the background activity (i.e., activity in a 100 ms period before the cue onset; paired t test, p < 0.05). The first of the four consecutive windows was picked. Starting at the beginning of the first window, we then moved a 50 ms window to find first four consecutive windows that showed significantly different activity compared with the background activity (i.e., activity in a 50 ms period before the cue onset). The first window of the consecutive four windows was picked again, and the middle of the window was defined as the latency for the neuronal response. Values that are measured with this procedure are consistent with values we would have chosen by eye.

Results

Behavioral and electrophysiological data were collected while two adult rhesus monkeys performed the visually cued reward schedule task (Fig. 1).

Behavioral analysis

As seen previously (Bowman et al., 1996; Shidara et al., 1998; Liu and Richmond, 2000; Shidara and Richmond, 2002), the monkeys were sensitive to the cue and the work schedule in the valid cue condition (Fig. 2A). The error rates and reaction times were indistinguishable across the three rewarded trials, 1/1, 2/2, and 3/3 (single-factor ANOVA to the two monkeys together; mean error rates, F_(2,735) = 1.79, p > 0.05; mean reaction times, F_(2,735) = 0.0022, p > 0.05). The error rates and reaction times were significantly different across the first trials, 1/1, 1/2, and 1/3 (single factor ANOVA; mean error rates, F_(2,735) = 46.72, p < 0.0001; mean reaction times, F_(2,735) = 73.7, p < 0.0001), showing that the monkeys were sensitive to the information provided by the cue about the schedule.

Figure 2.

Error rates and reaction times in visually cued reward schedule task in both the valid cue and random cue conditions. A, Relationship between monkeys' behavioral performance and the schedule states. The abscissa shows the schedule states. The ordinate shows the proportion of error trials in all trials (left) and mean reaction times (right) in the correct trials. Behavioral performances in the valid cue condition are shown with black symbols, and performances in the random cue condition are shown with gray symbols. The data include all of the recording sessions for each monkey (131 for monkey A and 115 for monkey B). Error bars indicate SE. B, Relationship between monkey's behavioral performance and the cue brightness.

Both monkeys reacted progressively more quickly as the rewarded trial approached in the two- and three-trial schedules (single-factor ANOVA with repeated measures for effect of schedule progress, two-trial schedule, F_(1,245) = 314.3, p < 0.0001; three-trial schedule, F_(2,490) = 324.8, p < 0.0001). Error rates became smaller as the schedule progressed through the three-trial schedule (single-factor ANOVA with repeated measures for effect of schedule progress, F_(2,490) = 79.4, p < 0.0001). In the two-trial schedule, error rates changed significantly for monkey B only (F_(1,130) = 1.3, p > 0.05 for monkey A; F_(1,114) = 109.7, p < 0.0001 for monkey B).

As expected in the random cue condition, the error rates and reaction times were the same in all three first trials (1/1, 1/2, and 1/3; single-factor ANOVA; mean error rates, F_(2,390) = 0.81, p > 0.05; mean reaction times, F_(2,390) = 0.18, p > 0.05). In the random cue condition, some information about the reward contingency is available because the longest possible sequence of unrewarded trials was fixed (two in the three-trial schedule); the probability of a reward is 1/3 in the first trial after a preceding reward, 1/2 if one unrewarded trial had passed, and 1, i.e., the reward is certain, if two unrewarded trials have passed. In previous studies, the monkeys had not shown any sensitivity to the number of trials that had passed (Bowman et al., 1996; Shidara et al., 1998; Liu and Richmond, 2000; Shidara and Richmond, 2002). Here, however, both monkeys were sensitive to number of trials that had passed, with a stronger trend in the three-trial schedule. For monkey A, the error rates increased as the schedule progressed (single-factor ANOVA with repeated measures for effect of schedule progress, F_(2,146) = 5.8, p < 0.01) whereas the reaction times were not modulated by the schedule state (F_(2,146) = 0.63, p > 0.05). For monkey B, the reaction times were related to the schedule states, becoming shorter as the schedule state progressed (F_(2,112) = 46.6, p < 0.0001), whereas the error rates were not modulated by the schedule state (F_(2,112) = 1.5, p > 0.05). In the states of the two-trial schedule, neither reaction times nor error rates were significantly different for monkey A. Reaction times decreased for monkey B (F_(1,56) = 13.6, p < 0.001).

Both monkeys were also sensitive to the cue brightness in the random cue condition (Fig. 2B). The brightest cue was treated differently from other cues in both reaction times and error rates (p < 0.01, t test). Thus, the monkeys were sensitive to two features of the random cue condition, the brightness of the cue, and the number of unrewarded trials that had passed.

To examine behavioral variability, we took data from 84 of 246 recording sessions (46 for monkey A, 38 for monkey B) for the valid cue condition in which at least 20 trials had been performed for each schedule state. Data were available from the random cue condition in 40 of these sessions. In the valid cue condition, the reaction times were significantly different across three first trials (1/1, 1/2, and 1/3) in 43 of 84 (51%) sessions (single-factor ANOVA, p < 0.01). The reaction times were indistinguishable across the three rewarded trials, i.e., 1/1, 2/2, and 3/3, for 79 of 84 (94%) sessions (single-factor ANOVA, p > 0.05). In the two-trial schedule, the reaction times were significantly different along progress for 43 of 84 (51%), and in the three-trial schedule for 41 of 84 (49%; single-factor ANOVA with repeated measures for effect of schedule progress, p < 0.01). All 84 recording sessions showed at least one of the following four reaction time characteristics: significantly different across the three first trials, not significantly different across the three rewarded trials (i.e., not influenced by the post-reward number as described above for the random cue condition), significantly different along the progress in the two-trial schedule, or significantly different along the progress in the three-trial schedule. Twenty-three sessions showed one, 18 sessions showed two, another 25 showed three, and 18 showed a significant effect of all four reaction time characteristics. Thus, the valid cue seemed to affect the animals in every recording session, although the effect changed its pattern from session to session. For 18 of the 23 sessions showing only one criterion, the criterion met was not significantly different across the three rewarded trials. The neurons recorded in these 18 sessions did not seem to react differently from neurons recorded in the other sessions, suggesting that there was no substantial difference between the data collected when the behavior was affected by the schedule in a large way compared with the data when the behavior was less affected. In the random cue condition, the reaction times of three first trials were not significantly different for any sessions (single-factor ANOVA, p < 0.01), and the reaction times decreased significantly as the schedule state progressed in 2 of 15 sessions of monkey B (single-factor ANOVA with repeated measures for effect of schedule progress, p < 0.01, a result indistinguishable from a random effect).

We could not find clear relationships between neuronal responses that are presented in the following and these session-to-session variability regarding the reaction times.

Electrophysiology

Activity was recorded from 246 (131 from monkey A, 115 from monkey B) single neurons located in the basolateral complex [including lateral (L), basal (B), and accessory basal (AB) nuclei] of the right amygdala (Fig. 3) of two monkeys during the visually cued reward schedule task (Fig. 1). Responses were collected while single neurons were well isolated, which gave from 6 to 80 repetitions (median 21) of each schedule state (1/3, 2/3, 3/3, 1/2, 2/2, and 1/1) in valid cue condition, and 5-43 repetitions (median 22) in random cue condition. We were unable to identify systematic response differences related to recording site location, so we present the neurons as one set.

Figure 3.

Histological localization of recorded neurons. Location of recorded neurons in one monkey (monkey A) are plotted in circles with coronal sections positioned at A26, A25, A24, and A23-A22 of the right hemisphere. A26 represents anterior 26 mm in Horsley-Clarke coordinates (namely, distance from the plane having the external auditory meatus). AA, Anterior amygdaloid nuclei; AB, accessory basal nucleus; B, basal nucleus; Ce, central nucleus; L, lateral nucleus; ac, anterior commissure; cd, caudate nucleus; cl, claustrum; lv, lateral ventricle; opt, optic tract; rs, rhinal sulcus; D, dorsal; L, lateral; M, medial; V, ventral.

We examined neuronal activity over different task events, namely before and after presentation of visual cue (precue activity and cue-related response, respectively), before monkey's bar release (go/prebar-release activity), and around reward apparatus activation (reward related response). We found that 188 of 246 neurons (76%) showed a response in one (59 of 188) or more (129 of 188) of these time periods (Table 1). Thus, for more than one-half of neurons (129 of 246, 52%), task-related activity occurred at more than one point in the behavioral task.

Table 1.

Number of neurons responding

	Neuronal response
Number of responses per cell	Precue activity	Cue-related response	Go/prebar-release activity	Reward-related response	Number of cells	Total number of cells
1	○				20
		○			26
			○		4
				○	9	59 (31%)
2	○	○			47
	○		○		2
	○			○	5
		○	○		18
		○		○	4
			○	○	5	81 (43%)
3	○	○	○		32
	○	○		○	6
	○		○	○	3
		○	○	○	3	44 (24%)
4	○	○	○	○	4	4 (2%)
Total number of cells	119	140	71	39		188 (100%)

Circles indicate significant response (p < 0.01).

Signals related to reward schedules

Precue activity

In the valid cue condition, 119 of 246 (48%) of the neurons in the basolateral complex of the amygdala showed precue activity that is selective for differentiating interschedule from intraschedule periods (t test between activity in the 300 ms precue period before the first trials and activity in the precue period before the non-first trials, p < 0.01) (as in the example in Fig. 4A). A total of 88 of 119 neurons (74%) showed this type of preschedule activity building toward the end of the interschedule period, i.e., appearance of the first cue (one-tailed t test between activity in the precue period before the first trials and before the non-first trials, p < 0.005). There was virtually no precue activity preceding non-first trials (2/2, 2/3, and 3/3).

Figure 4.

Precue activity. A, In the valid cue condition, neuronal activity before the cue presentation increased only in the first trial (i.e., preschedule activity). The abscissa shows times from the cue onset (0 ms, vertical line). The ordinate shows a firing frequency per trial (spikes per second per trial). Each line of the dots represents a trial in the task. The earliest trial is placed at the bottom. Each curve shows instantaneous firing frequency averaged over all trials and smoothed with a Gaussian filter (σ = 20 ms). The one-, two-, and three-trial schedules are placed on the top, middle, and bottom rows, respectively. Progress in each schedule goes from left to right. The rewarded trials are on the diagonal. The gray triangles pointing down indicate the time when the visual cue and the blue fixation spot disappeared in the previous trial. The black triangles pointing down show the time of reward apparatus activation at the previous trial. The median latency of the preschedule activity from the preceding reward apparatus activation was 742 ms (interquartile range, 451-1018 ms). B, In the random cue condition, preschedule activity disappeared.

The time onset for the preschedule activity varied from trial to trial (compare with Fig. 4A), usually starting well after the reward (≥700 ms here) and long before (≥300 ms here) the cue signaling the beginning of the next schedule. The preschedule activity generally continued until shortly after cue onset (the ending defined as the time when a cue-triggered response began). Only 2 of 88 neurons showed a significant increase in activity just after the cue onset compared with just before the cue onset (paired t test between activity just before and after the cue onset; activity was measured in either a 100 ms period or, if the latency was shorter than 100 ms, a period before latency of the after cue related response, p < 0.01).

The other 31 of 119 neurons (26%) showed a complementary response pattern, i.e., the precue activity in the non-first trials (2/2, 2/3, and 3/3) was larger than precue activity in the first trials (one-tailed t test, p < 0.005). No single neuron had precue activity before every cue (t test between an activity in the precue period and an activity in the control period, p < 0.01).

When we divided the 300 ms precue period into two 150 ms periods, the activity of 33 of 119 neurons (all 33 neurons for preschedule activity) was significantly different in the two periods (one-tailed paired t test on increase, p < 0.005), showing that the precue activity of many neurons was higher just before the appearance of the cue than it had been earlier.

A total of 82 neurons were tested in the random cue condition. The difference in activity between preschedule and intraschedule periods disappeared (49 neurons) (Fig. 4B) or became significantly smaller (11 neurons; t test, p < 0.005). Thus, the precue activity was, in general, not simply related to the reward in the preceding trial. The preschedule activity in the population was small in the random cue condition (Fig. 5). Thus, the preschedule activity is related to receiving the information about the next reward schedule that will be indicated by upcoming valid visual cue, possibly attributable to increased arousal, attentiveness, or some other mechanisms.

Figure 5.

A, B, Strength of precue activity of a neuronal population in the valid cue (A; n = 88) and random cue (B; n = 68) conditions. In the box plots, middle line indicates the median; notches indicate the 95% confidence interval. Whiskers extend to the most extreme data point that is not >1.5 times the interquartile range. The strength of preschedule activity (i.e., 1/1, 1/2, and 1/3 states) was significantly larger in the valid cue condition (A) than in the random cue condition (B; Wilcoxon's test, p < 0.0001).

Because amygdala lesions interfere with reinforcer devaluations (Malkova et al., 1997), we looked for neuronal firing changes related to the progress through a recording session (presumably the monkeys become satiated as the session progresses, although each recording session might be too short to detect effects related to satiation). We did not identify any trend in the precue activity.

Cue-related response

From inspection, it appeared that many neurons had responses after the appearance of the cues. To identify the period with most cue-selective response, a 350-ms-wide analysis window was moved across the responses in 10 ms steps (Fig. 6). In the range starting with the time window from -285 to 75 ms relative to cue appearance and ending with the time window from 115 to 465 ms relative to cue appearance, 102 of 246 neurons showed at least three consecutive time windows with significant selectivity across the three first cues or the three non-first cues (single-factor ANOVAs, p < 0.01). Of these 102 neurons, 81 showed significant differential activity across the responses elicited by the three first cues (1/1, 1/2, and 1/3) (Fig. 7), six across the three non-first cues (2/2, 2/3, and 3/3), and 15 to both (Fig. 8). The time window from 100 to 450 ms after cue onset, which was initially chosen by eye, was used to approximate the time period with greatest effect (Fig. 6).

Figure 6.

Time course of the effect of either the three first cues or three non-first cues on the responses around the cue presentation measured by single-factor ANOVA. The abscissa shows time from the cue onset relative to the end of 350 ms analysis window. A, Number of neurons that showed significant selectivity across the three first or non-first cues (single-factor ANOVAs, p < 0.01). The solid vertical line shows the cue onset. The largest number of neurons (86) with a significant differential activity was found in the 85-435 ms time window after cue onset. Starting with the time window from -185 to 165 ms after the cue onset, the number of cells with significant differential response increased to 12 (a number significantly different from chance; χ² test, p < 0.01). B, Mean percentage response variance explained for 102 neurons that showed at least three consecutive time windows with significant selectivity across the three first cues or the three non-first cues (single-factor ANOVAs, p < 0.01). The gray curves show SE of the percentage response variance explained. The largest average variance was explained in the time window from 105 to 455 ms after cue onset (33 ± 2%, mean ± SE; n = 102).

Figure 7.

Cue-related response that delineates the shortest schedule length. A, In the valid cue condition, this neuron showed unstable preschedule activity starting before the cue onset (0 ms, vertical line). The largest response followed the 1/1 cue. Two comparisons, namely 1/1 versus 1/2 and 1/1 versus 1/3, were significant for this neuron (Tukey's test, p < 0.01). B, In the random cue condition, none of the comparisons between responses to two different first trials (1/1 vs 1/2, 1/1 vs 1/3, and 1/2 vs 1/3) was significantly different (Tukey's test, p > 0.01). C, In the random cue condition, the cue-related responses sorted by brightness did not depend on cue brightness (single-factor ANOVA, p > 0.05). Fiducial marking as in Figure 4.

Figure 8.

Cue-related response. A, In the valid cue condition, after preschedule activity, this neuron showed an excitatory response after the cue onset (0 ms, vertical line) of the one-trial schedule but stopped firing after the cue presentation of the two- or three-trial schedule. In the two- and three-trial schedules, the neuron responded again after the cue onset in the rewarded trials, namely both 2/2 and 3/3. Response latencies are 90 ms for 1/1 cue, 173 for 1/2 cue, 93 for 2/2 cue, 153 for 1/3 cue, and 93 for 3/3 cue. B, In the random cue condition, the neuron showed a small response only to the first cue; however, the response pattern across the cues was significantly different from the responses in the valid cue condition. The strength of the excitatory response in 1/1 was significantly smaller compared with the response in the valid cue condition (t test, p < 0.0001). Response latencies are 145 ms for 1/1 cue, 108 for 1/2 cue, and 113 for 1/3 cue. C, Cue-triggered responses in the random cue condition shown in B were sorted by the brightness of the cue. Brightness value of each cue is shown as percentage brightness (the value 255 is the maximum in 8-bit grayscale and shown as 100%). The strength of the response was modulated by the brightness (single-factor ANOVA, F_(3,88) = 17.7, p < 0.0001), with the brightest cue eliciting the strongest response. The excitatory response to the brightest cue was significantly smaller compared with the response either to the 1/1, 2/2, or 3/3 cues in the valid cue condition (t test, p < 0.001). Fiducial markings as in Figure 4.

In the interschedule or intraschedule precue period, only 3 of 102 neurons (3%, i.e., <5% level) showed a significant differential response related to the upcoming schedule during the anticipatory precue period (starting with the time window from -575 to -225 ms relative to cue appearance ending with the time window from -355 to -5 ms relative to cue appearance) (Fig. 6A). Thus, the precue activity had no systematic effect on the postcue selectivity. The effect of the three first or non-first cues on neuronal activity increased starting at ∼100 ms after the cue onset (Fig. 6). The activity in a 100 ms period just after the cue onset, i.e., the continuation of the precue activity after the cue onset, was indistinguishable across the three first cues or the three non-first cues for all 102 neurons (single-factor ANOVA, p > 0.01).

To determine whether the responses to the first cues might be related to the predicted schedule length, pairwise comparisons (1/1 vs 1/2, 1/2 vs 1/3, and 1/1 vs 1/3; Tukey's test to adjust for multiple comparisons, p < 0.01) were performed. In this analysis, 84 of 102 (82%) neurons showed differential activity across the first trials (compatible with, but not identical to, the ANOVA above) (Table 2). For 43 of 84, the responses elicited by 1/1 cue were different from the responses to the 1/2 or 1/3, whereas the responses to these latter cues were indistinguishable (Fig. 7A); for 23 of 43 neurons, the lack of difference in the responses between the 1/2 and 1/3 cues occurred because there was no response to the 1/2 and 1/3 cues. Thus, there was a significant emphasis of the 1/1 schedule in the population. Nonetheless, the selectivity of the neurons within the population provides information about which schedule is starting (Table 2).

Table 2.

Significant discrimination among cue-related responses across first trials of different schedules (Tukey's test, p < 0.01; see Results)

Comparison
1/1 versus 1/2	1/1 versus 1/3	1/2 versus 1/3	Number of cells
○	○	○	8
○	○		43
○		○	7
	○	○	11
○			3
	○		11
		○	1
Total number of cells			84

Circles indicate significant comparisons.

None of the comparisons across first trials was significant for the random cue condition (Figs. 7B,8B), as expected if the monkey was unable to predict the schedule length. The 1/1 cue often induces a significantly different response from the other two cues in the valid cue condition, perhaps attributable to the monkey anticipating an immediate reward. However, it is not quite straightforward to attribute this effect to a simple difference in arousal level as inferred from the behavioral measure. The behavioral results (compare with Fig. 2) suggest that the monkeys might be more aroused (or at least trying harder, i.e., more motivated) in the 1/2 state than in the 1/3 state, yet there does not seem to be any systematic difference between the cue-related responses in the 1/2 and 1/3 states for many neurons. This makes it difficult to invoke a simple connection between probable state of arousal, motivation, or effort and the first-cue elicited responses.

Pairwise comparisons (2/3 vs 2/2, 2/3 vs 3/3, and 2/3 vs 3/3; Tukey's test to adjust for multiple comparisons, p < 0.01) showed that 14 of 102 (14%) neurons exhibited differential activity across the non-first trials (Table 3, Fig. 8), revealing that the responses in non-first trials can on occasion be related to the reward contingency. Of the 43 neurons that showed a significantly different response to the 1/1 cue than to the other two first cues (see above), only five showed differential activity across the non-first trials, suggesting that the neurons responding to the 1/1 cue were not simply responding because the cue signaled a rewarded trial (i.e., the response to the 2/3 cue was generally not different from the response to the 2/2 or 3/3 cues). It may be that the responses to the 1/1 cue are special because the cue indicates the conjunction of a first and rewarded trial, the most favorable message possible in this task.

Table 3.

Significant discrimination among cue-related responses across non-first trials of multiple-trial schedules (Tukey's test, p < 0.01; see Results)

Comparison
2/3 versus 2/2	2/3 versus 3/3	2/2 versus 3/3	Number of cells
○	○		5
○			5
	○		2
○		○	2
Total number of cells			14

Circles indicate significant comparisons.

Sixty-seven of the 102 neurons were tested in the random cue condition. The cue-related responses of 13 neurons were influenced according to two-level first vs non-first trial discrimination, with 8% of response variance being explained [single-factor ANOVA, p < 0.01; f test with 65-190 and 1 degree of freedom; n = 13; interquantile range (iqr) of 6-11%].

The responses of 28 of 67 (42%) neurons were related to the cue brightness (single-factor ANOVA, p < 0.01; f test with 54-213 and 3 degrees of freedom) (Fig. 8C), with 18% (median, iqr of 11-23%) of response variance being explained. Of the 28 neurons, the response was strongest to the brightest cue for 11 neurons (Fig. 8C) or to the darkest cue for 17. For 18 of 28 neurons, the strength of the cue-related responses in the valid cue condition were significantly different from the responses to the same brightness cue in the random cue condition (t test, p < 0.01) (an example shown in Fig. 8), indicating that the cue-related responses in the valid cue condition was not simply related to the cue brightness for majority of neurons (the 18 plus 39 neurons without the brightness effect, i.e., 57 of 67 neurons).

For another 38 of 246 neurons, the activity changed significantly after the cue appearance, but the activity was indistinguishable across 1/1, 1/2, and 1/3 states (single-factor ANOVA, NS; t test between rate in a 100-450 ms window after cue appearance and a 150 ms window before the appearance, p < 0.01). These responses might be simply related to arousal induced by the appearance of the cue.

The latency of the cue-triggered response was 119 ms (median, iqr of 98-158 ms; n = 312; measured for the 113 neurons that showed a clear phasic increase or decrease in activity after the cue onset). The median firing rate was 13 spikes/s (iqr of 8-23 spikes/s; n = 312). The latency was slightly but significantly shorter when precue activity preceded the response (median of 110 ms; iqr of 93-153 ms; n = 103) compared with the response latency when there was no precue activity before the response (median of 123 ms; iqr of 103-163 ms; n = 209; Wilcoxon's test, p < 0.05). These latencies are longer than those reported for area TE and shorter than those reported for perirhinal cortex (Liu and Richmond, 2000).

Signals related to reward

Go/prebar-release activity

In the valid cue condition, we suspected from inspection that the activity after the go signal presentation and just preceding bar release was the same in the final trials of all schedules (Fig. 9A). A total of 71 of 246 neurons showed significant modulation according to two-level rewarded versus nonrewarded trial discrimination (single-factor ANOVA, p < 0.01; f test with 45-440 and 1 degree of freedom). Fourteen percent of the response variance was explained (median, n = 71; iqr of 9-25%). When the responses of these 71 neurons were analyzed using four levels in which the levels were the three nonrewarded states (1/3, 2/3, and 1/2) plus a level for any rewarded trial (thus grouping 1/1, 2/2, and 3/3 together; reward proximity in the schedule), the increased degrees of freedom significantly improved the explanatory power for 10 neurons (an f test comparing the amount of variability accounted for by the two models, p < 0.01) (Venables and Ripley, 2002), and 33% of the response variance was explained (median, n = 10; iqr of 30-40%). When the responses of these 71 neurons were analyzed using the six schedule states, the increased degrees of freedom significantly improved the explanatory power for only two neurons, confirming our suspicion that the responses were related to the reward contingency or the number of trials remaining before the rewarded trial.

Figure 9.

Activity during go/prebar-release period. A, In the valid cue condition, the activity increased gradually as the bar release approached. The vertical lines show time of bar release. The black triangles represent the onset of the go signal. Trials are sorted according to reaction times, and the trial with longest reaction time is placed at the bottom. Responses before bar release were selective for the four levels (trials in schedule before reward; single-factor ANOVA, F_(3,138) = 21.7, p < 0.0001). The responses of the rewarded trials, namely 1/1, 2/2, and 3/3 states, were indistinguishable (F_(2,68) = 0.21, p > 0.05). The activity became stronger as the two- or three-trial schedules progressed (single-factor ANOVA with repeated measures for effect of schedule progress, two-trial schedule, F_(1,22) = 25.4, p < 0.0001; three-trial schedule, F_(2,46) = 9.9, p < 0.0005). B, In the random cue condition, the activity was indistinguishable across the schedule states (single-factor ANOVA, F_(5,116) = 1.17, p = 0.32, p > 0.05). C, Go/prebar-release activity in the random cue condition (B) according to the cue brightness. There was no modulation depending on the cue brightness (single-factor ANOVA, F_(3,118) = 1.30, p > 0.05). Fiducial markings as in Figure 4.

The responses were time locked to either the bar release (45 of 71 neurons) or the go signal (26 of 71 neurons; responses classified according to the alignment giving the biggest peak response in the time period between the go signal appearance and bar release). The activity was also modulated by progress in both the two- and three-trial schedules for 25 of 71 neurons (p < 0.01, single-factor ANOVA with repeated measures for effect of schedule progress; f test with 11-37 and 1 degree of freedom for the two-trial schedule; f test with 16-66 and 2 degrees of freedom for the three-trial schedule). The activity increased as the monkey approached the rewarded trial in the schedules (19 of 25 neurons; one-tailed t test, p < 0.005) (Fig. 9A), suggesting that the go/prebar-release activity is related to the expectation of forthcoming reward and/or proximity of reward, possibly via increased arousal.

Forty-five of these 71 neurons were tested in the random cue condition. All but one neuron (44 of 45) lost the differential activity depending on the reward contingency (single-factor ANOVA, p > 0.01; f test with 67-220 and 1 degree of freedom). Go/prebar-release activity in the population was present in every trial (Fig. 10) (example of one neuron Fig. 9B). For 33 neurons, the activity was independent of the cue brightness (single-factor ANOVA, p > 0.01; f test with 65-218 and 3 degrees of freedom) (Fig. 9C), suggesting that the go/prebar-release activity in the valid cue condition did not encode the cue brightness.

Figure 10.

Strength of go/prebar-release activity of a neuronal population. A, In the valid cue condition, the activity of a neuronal population was larger in rewarded trials (n = 71; paired Wilcoxon's test; ^*p < 0.01; ^**p < 0.001; ^***p < 0.0001). B, In the random cue condition, the population activity was indistinguishable across schedule states (n = 45). The strength of the rewarded trials (1/1, 2/2, and 3/3) was not significantly different between the valid and random cue conditions. Fiducial markings as in Figure 5.

Trial-by-trial reaction time and go/prebar-release activity was significantly related only for 9 of 71 neurons (linear regression, 10 of 426 cases, i.e., 6 states by 71 neurons; p < 0.01), showing that the strength of the go/prebar-release activity was only occasionally related to the reaction time. In addition, monkey B showed shorter reaction times as number of trials that had passed increased in the random cue condition, but no neuron (0 of 13 neurons) of the monkey showed significant modulation of the go/prebar-release activity related to reward proximity in the random cue condition (single-factor ANOVA, p > 0.01; f test with 80-218 and 3 degrees of freedom). Thus, the go/prebar-release activity generally did not appear related to motor aspects of the monkey's bar release.

Force measurements taken from the drinking spout (Ravel et al., 2001) for one monkey (monkey A) revealed that the monkey started to lick ∼300 ms after onset of the wait signal in the rewarded trials, long before (by at least 700 ms) the onset of the go signal. Thus, it is unlikely that the go/prebar-release activity was related to the monkey's mouth movements.

Reward-related response

Thirty-nine of 246 neurons responded at the time the reward apparatus was activated (which only occurred in rewarded trials). For 32 neurons, there was an excitatory response (Fig. 11A), and, for the other seven, there was an inhibitory response (one-tailed t test against strength of activity in a 400 ms period before the reward, p < 0.005). For seven neurons, the response started before reward apparatus activation. For 12 of 39 neurons (including one inhibitory neuron) that had responses in the rewarded trial, we also found responses in nonrewarded trials at the time when the reward would have been delivered if this had been a rewarded trial. For these 12 neurons, the strength of responses in the rewarded and nonrewarded trials were not consistently different (t test, p < 0.005; larger to the rewarded trials, 4 of 12; smaller to the rewarded trials, 4 of 12; not significantly different, 4 of 12).

Figure 11.

A-C, Response triggered by reward delivery (1/1, 2/2, and 3/3 states) in the valid cue (A), random cue (B), and free-reward (C) conditions. The vertical lines show time of reward delivery in rewarded trials (black) and equivalent timing without reward delivery in nonrewarded trials (gray). This neuron showed reward-related response in all three conditions. Fiducial markings as in Figure 4.

Of 21 of 39 neurons tested with the random cue condition, 19, including four that showed anticipatory activity before the reward, responded to the reward (one-tailed t test against strength of activity in the control period, p < 0.005) (Fig. 11B). Thus, most of the reward-related signals were found in both the valid and random cue conditions (Fig. 12A), suggesting that they are related to reward delivery itself in some way. Seven neurons responded to the reward only in the random cue condition.

Figure 12.

Distribution of the neurons that responded to reward in different task conditions. A, Valid cue and/or random cue conditions. B, Valid cue and/or free-reward conditions.

The free-reward condition provided evidence that some reward-related responses might become associated with other task events during the reward schedule task. Among 113 neurons tested with both the valid cue and free-reward conditions, 20 responded in the free-reward condition only and 16 responded in both conditions (Fig. 12B) (an example in Fig. 11). Of the 20 neurons that showed the reward-related response only in the free-reward condition, 14 responded either to the cue presentation (Fig. 13) or around the bar release in the valid cue condition. If the neuron shown in Figure 13 had been tested only in the free-reward condition (Fig. 13A), we would have concluded that the response was a “reward-”related response. However, in the valid cue condition, the response occurred to the first cue (Fig. 13B), suggesting that the neuron has a type of signal similar to the prediction signal described for dopamine neurons (Mirenowicz and Schultz, 1994; Schultz et al., 1997). The response transfer to the cue presentation or to the bar release in the valid cue condition might be related to dopaminergic inputs to basolateral complex (Sadikot and Parent, 1990). A similar type of response transfer has been found in other brain areas that receive dopaminergic inputs, such as the orbitofrontal cortex (Matsumoto et al., 1995; Tremblay and Schultz, 2000) and the striatum (Apicella et al., 1997; Ravel et al., 2001).

Figure 13.

Response transfer. The black vertical lines show time of reward delivery, and the gray vertical lines show cue onset. A, In the free-reward condition, this neuron showed a response after reward. B, In the valid cue condition, this neuron showed a response to the first cue (1/1, 1/2, and 1/3). The response to the 1/1 cue was larger than the response to the 1/2 cue and 1/3 cue (t test, p < 0.005). Fiducial markings as in Figure 4.

Discussion

More than 70% of recorded neurons showed response modulation for one or more aspects of this visually cued reward schedule task (Table 1). The selectivity of this population changed dynamically across the schedules and trials. The population begins firing before the cue presentation; for individual neurons, this activity occurs during either the interschedule or the intraschedule periods but not both, signaling whether the next trial begins a new schedule or not. When the first cue appears, the population encodes which schedule is starting, with the greatest proportion of neurons indicating whether or not the one-trial (1/1) schedule has begun. Starting just before the bar release, i.e., from the go/prebar-release period, the responses become selective for the reward contingency, i.e., they indicate whether a reward is forthcoming or not. A few neurons responded to reward delivery. The dynamics, going from uncertainty to more and more focus on the reward, may be related to resolution of an emotionally laden (or arousing) reward situation.

Possible origins of the signals

The basolateral complex of amygdala receives inputs from a late stage of ventral visual processing, having reciprocal connections with the area TE (Aggleton et al., 1980; Amaral and Price, 1984; Cheng et al., 1997; Stefanacci and Amaral, 2002). Area TE has reciprocal connections with the perirhinal cortex (Suzuki and Amaral, 1994; Saleem and Tanaka, 1996), and perirhinal cortex has reciprocal connections with the amygdala (Amaral and Price, 1984; Saunders and Rosene, 1988; Stefanacci et al., 1996). Presumably visual information that is first processed in area TE is sent to both perirhinal cortex and amygdala, and these three areas influence each other.

During the visually cued reward schedule task, only cue-related activity has been observed in area TE and perirhinal cortex. In contrast, amygdala neurons show responses before the cue presentation, before the bar release, and to the reward delivery as well as to the cue. Thus, visual, reward-related, and other signals merge in the amygdala.

In the amygdala, in the valid cue condition, the population is biased toward the 1/1 schedule. In the random condition, there is small but significant response related to brightness of the cue, somewhat like what has be seen in area TE (Liu and Richmond, 2000). In perirhinal cortex, individual neurons show responses that are idiosyncratically related to subsets of the cues, with the cue-related responses encoding information about the schedule states in the population (Liu and Richmond, 2000).

The latency of the cue-triggered responses in the amygdala neurons is longer than those in area TE and shorter than those in perirhinal cortex. These results imply that information processing in the amygdala lies between TE and perirhinal cortex and might contribute to form the associative signals in perirhinal cortex in which the D₂ receptor is crucial for learning the relation of the cue to the schedule states (Liu et al., 2000, 2004). There is, however, a difficulty. At approximately the time of bar release and/or a reward delivery, perirhinal cortex neurons did not respond, so either the cue-triggered amygdala responses do not contribute to perirhinal neuronal responses or cue-triggered activation of perirhinal neurons requires other input in conjunction with the amygdala inputs.

Another possible source for some of the signals in the amygdala are dopamine neurons in substantia nigra pars compacta/ventral tegmental area and/or neurons in orbitofrontal cortex, both of which project strongly to the basolateral complex of amygdala (Sadikot and Parent, 1990; Ghashghaei and Barbas, 2002). Dopamine neurons are known to respond to visual stimuli that precede and predict reward delivery and to motor acts such as bar release that lead to a forthcoming reward (Schultz and Romo, 1987). Some amygdala neurons respond to the cue or around bar release in the valid cue condition but to reward in the free-reward condition, a situation related to reward anticipation or prediction. In the amygdala, the response elicited by the 1/1 cue is often different from the response to the other two first cues (Table 2), a condition that is different from what is found for the dopamine neurons, in which the responses to the first cue are indistinguishable (S. Ravel and B. J. Richmond, unpublished observation). This suggests that the dopamine neurons might play a role in eliciting these cue-related responses but are probably not adequate to explain why the cue in the 1/1 condition elicits a different response than the 1/2 or 1/3 cues.

Recent lesion studies have shown that reinforcer devaluation depends on the amygdala and orbitofrontal cortex (Malkova et al., 1997; Baxter et al., 2000; Izquierdo and Murray, 2004), suggesting that both the amygdala and orbitofrontal cortex play a role in association of the conditioned stimulus with the current value of a reward. Orbitofrontal neurons respond to a visual stimulus that is associated with a reward and the reward itself (Tremblay and Schultz, 2000). Orbitofrontal cortex has a heavy projection to presumed inhibitory neurons (intercalated masses located between lateral and basal nucleus) in the amygdala (Ghashghaei and Barbas, 2002), and signals in the orbitofrontal cortex are thought to modify signal processing within the amygdala. Inhibition after the preschedule activity, such as is shown in Figure 8 for the 1/2 and 1/3 cues, might have been influenced by such processing. The basolateral amygdala is important for the orbitofrontal cortex to represent the significance of a reward-predicting cue (Schoenbaum et al., 2003); thus, the signals in the amygdala could influence signals in orbitofrontal cortex, too.

Other inputs such as those from the posterior thalamus might also play a role in the cue-related response (Komura et al., 2001; Shi and Davis, 2001).

Functional roles of the neuronal responses in the amygdala

The basolateral complex of amygdala receives sensory inputs from sensory association cortices and reward-related information from substantia nigra pars compacta/ventral tegmental area and has reciprocal connections with areas that are important for motor selection depending on reward, e.g., the anterior cingulate cortices (Amaral and Price, 1984; Stefanacci and Amaral, 2002), and the medial and orbitofrontal cortex (Amaral and Price, 1984; Ghashghaei and Barbas, 2002). It also has projections to the ventral striatum (Friedman et al., 2002; Fudge et al., 2002). The responses we recorded show integration of information from these modalities in a time-multiplexed manner (Table 1). Perhaps the amygdala plays a role when association among sensory signals, motor outputs, and reward is required for task performance.

It is known that amygdala neurons (including the basolateral complex) are responsive to sensory events associated with reward and/or to reward itself in monkeys (food, juice, or intracranial self-stimulation) (Nishijo et al., 1986, 1988a,b; Nakamura et al., 1992) and that neurons in basolateral nucleus show anticipatory activity before getting a reward in rats (Pratt and Mizumori, 1998; Schoenbaum et al., 1998). Responses of amygdala neurons to an unconditioned stimulus change depending on satiety (Yan and Scott, 1996) or taste (Nishijo et al., 1988b). Based on these previous studies, we had expected responses related to reward and possibly sensory events predicting the reward (e.g., cue). The precue activity and the modulations around the bar release were unexpected. Comparing our results in other brain regions with those presented here, the amygdala is the only area in which this strong and somewhat prolonged precue activity has been seen thus far (Bowman et al., 1996; Shidara et al., 1998; Liu and Richmond, 2000; Shidara and Richmond, 2002). The amygdala is the only area studied thus far for which the population responses emphasizes the shortest schedule.

A number of previous results suggest that the amygdala plays an important role in evaluating the value or valence of objects. The anatomical connections and timing relations of the signals suggest that the amygdala is in a temporal lobe network that takes information about stimulus identity from temporal lobe sensory areas and generates signals predicting the behavioral path to be taken for obtaining a reward. Given the anatomical connections with the amygdala and a suggested role of the amygdala in emotions, it seems likely that emotions aroused by salient stimuli participate in determining the behavioral path. We have seen a preschedule signal that grows as the time for cue appearance approaches, possibly because the monkey becomes progressively more aroused while awaiting the appearance of the first cue. When the first cue appears, the signal becomes specific, encoding information about which schedule started with emphasis on the shortest schedule, presumably the best (for the monkey) situation. This is compatible with the role of the amygdala in assessing the valence associated with a stimulus (Everitt et al., 2003), in which the most desirable information, that this is the one-trial schedule, is associated with the most selectivity in the population. Later in each trial, the responses are better explained in terms of the reward contingency in the current schedule. These signals seem to provide information that interprets external events and stimuli so that the motivational level can be adjusted according to current behavioral goals. Such signals might be important for forming associative signals related to the reward schedules in perirhinal cortex and/or ventral striatum or forming long-term reward expectation signals in anterior cingulate cortex.