Time Context of Cue-Outcome Associations Represented by Neurons in Perirhinal Cortex

Abstract

The perirhinal cortex (PRh), which has extensive connections with diverse brain sites, may contribute to semantic memory by associating various types of information about objects. However, the extent of the types of associations in which PRh participates is unknown. In the present study, we let monkeys experience a consistent contingency between visual cues and different types of outcomes (water reward and sound-only acknowledgment) in a particular time context for many days and then recorded neuronal activities from PRh and area TE, which is the major source of visual inputs to PRh. We found that PRh cells represented the outcome type in their responses to the visual cues only in the time context in which the monkeys had experienced the cue-outcome contingency. In contrast, TE cells represented the outcome information whenever the cue appeared (i.e., independently from the related time context). These results showed that PRh cells represented not only the cue-outcome contingency but also the time context in which the monkeys had experienced the contingency. We conclude that PRh is not specific to the representation of sensory and associative properties of objects themselves but may represent broader information about objects, including the time context in which the objects are associated with particular outcomes.

Introduction

The perirhinal cortex (PRh) is a polymodal area that has reciprocal connections with later cortical stages of various sensory modalities as well as the orbitofrontal cortex, amygdala, and entorhinal cortex (ERh) (Suzuki and Naya, 2014) and receives dopaminergic projection from the ventral tegmental area (Goldsmith and Joyce, 1996; Wise, 2004). There is a considerable amount of evidence in macaque monkeys showing that PRh is involved in associations of various types of information about objects. Cells in macaque PRh respond to both of paired visual stimuli that the monkey has learned to associate with each other (Miyashita and Chang, 1988; Sakai and Miyashita, 1991; Higuchi and Miyashita, 1996; Erickson and Desimone, 1999; Messinger et al., 2001). Furthermore, monkeys with lesions in PRh have difficulty in learning the visual paired association (Buckley and Gaffan, 1998). Combined lesions of PRh and ERh also produce deficits in visual-taste (Parker and Gaffan, 1998) and visual-tactile associative learning (Goulet and Murray, 2001). Not only associations between sensory stimuli, but also associations between visual stimuli and reward-related events are represented in PRh. PRh cells, in their responses to associated visual cues, represent the position of a current trial in reward schedules or forthcoming outcome types (Liu and Richmond, 2000; Mogami and Tanaka, 2006; Ohyama et al., 2012). Combined lesions of PRh and ERh or suppression of D2 dopamine receptors in PRh and ERh reduce the dependence of monkeys' behavior on the association between visual cues and positions in reward schedules (Liu et al., 2000, 2004). Associations of various types of information about objects may be the core of semantic memory (McClelland and Rogers, 2003), and PRh may contribute to the semantic memory by associating various types of information about objects. Involvement of PRh in semantic memory has also been suggested in humans (Davies et al., 2004; Taylor et al., 2006; Holdstock et al., 2009; Wang et al., 2010; Chan et al., 2011).

Several groups of authors have commonly proposed that the information of objects, or items, conveyed by PRh is integrated in the hippocampus with the information of contexts, especially that of spatial contexts, conveyed by the parahippocampal areas (Davachi, 2006; Eichenbaum et al., 2007; Pascalis et al., 2009; Ranganath and Ritchey, 2012). According to these ideas, associations that occur in PRh are limited to those for objects or items. Therefore, it is important to determine the extent of the types of associations that occur in PRh. The present study focused on the time context of cue-outcome associations and aimed at determining whether it is represented by activities in PRh cells. We compared activities in PRh with those in area TE (Tanaka, 1996), which is the main source of visual inputs to PRh. We expected the comparison with TE to reveal unique functions that emerge at the stage of PRh. We let monkeys experience a consistent contingency between visual cues and different types of outcomes (water reward and sound-only acknowledgment) in a particular time context for many days and then recorded neuronal activities from PRh and TE.

Materials and Methods

Subjects.

Two male macaque monkeys (Macaca mulatta) were used. All experiments were approved by the RIKEN Animal Experiment Committee, and all procedures were performed in accordance with the guidelines of the Japan Neuroscience Society. A titanium head-fixating bar and a titanium recording chamber were implanted with titanium screws and dental acrylic onto the dorsal aspect of the skull in aseptic surgeries under general anesthesia with isoflurane (1.0%–2.0%).

Stimuli.

We used images of computer-generated meaningless objects as visual stimuli (Fig. 1). The stimuli were created in the same way as those in our previous study (Wang et al., 2005). There were 6 sets of stimuli, each of which consisted of 4 stimuli (together 24 stimuli). The 4 stimuli in each set were created by deforming a parent stimulus of the set in four different directions in a feature space. Human psychophysics was used to make the subjective similarity between any stimulus pair comparable within each set and to make the subjective similarity within each set comparable between sets. Differences between sets were much larger than differences within each set.

Figure 1.

Visual stimuli used in the experiments. The 24 stimuli consisted of 6 sets of 4 stimuli. The stimuli were associated with the two outcome types as noted at the top for Monkey 1. The stimuli in the first and third columns from the left were associated with the water reward and those in the second and forth columns with the sound-only acknowledgment for Monkey 2. Differences within each set were much smaller than differences between sets.

Experiments with fixed cue-outcome contingency.

The monkeys conducted a task that required both continuous lever-pressing and continuous eye-fixation while a visual cue was presented. The cue indicated the outcome type that would occur at the end of trial if lever-pressing and eye-fixation were maintained. The monkey sat in a chair, with the head fixed to the chair by the titanium bar implanted to the skull, facing a cathode ray tube display placed 50 cm from the monkey's eyes. A lever was placed in front of the monkey's body. The monkey pressed the lever to start a trial. A fixation point (a white square subtending 0.7° of visual angle) immediately appeared at the center of the display. The monkey had to fixate eye gaze within ±2.5° from the fixation point within 500 ms. When the monkey had continued the lever-pressing and eye-fixation for 400 ms, the fixation point was replaced with a visual cue. The cue was pseudo-randomly selected from the 24 stimuli with a constraint that two stimuli from the same set never appeared consecutively. The visual cue was presented for 350 ms, and the fixation point reappeared after that. The fixation window was set relatively large to realize a good success rate in unrewarded trials.

Twelve of the 24 stimuli (two in each set) were consistently associated with water reward and the other 12 stimuli (two other stimuli in each set) with no reward (Fig. 1). In trials in which the visual cue was one of those associated with reward (“reward-associated cues,” R), the continued lever-pressing and eye-fixation was rewarded with a drop of water twice: first at 600 ms and second at 1000 ms after the visual cue onset (see Fig. 3A). The monkey had to keep the lever-pressing and eye-fixation until the second reward onset. In trials in which the cue was one of those associated with no reward (“no-reward-associated cues,” NR), the successful continuation of lever-pressing and eye-fixation was acknowledged only by a click sound identical to that associated with the water reward delivery. The click sound was provided twice at the timings at which the reward would have been delivered if the trial had been a rewarded trial. The monkey had to continue lever-pressing and eye-fixation until the second click sound even during unrewarded trials to move to the next trial, which was possibly rewarded. There was a 200 ms intertrial interval (ITI) after the onset of the second outcome before the monkey could start the next trial. When the monkey broke the lever-pressing or eye-fixation during a trial, the trial was immediately terminated and a longer (700 ms) ITI was initiated. An unsuccessful trial was followed by a trial with the same visual cue. Each of the monkeys performed 900–1000 trials per daily session.

Figure 3.

Trial structure and behavioral results for the experiments with fixed cue-outcome contingency. A, Events in a trial. There was one sequence of visual-cue presentation and outcome delivery in each trial. Depending on the cue, a water reward or a clicking sound was provided. The same outcome was repeated twice in each trial. Monkeys were only required to maintain lever-pressing and eye-fixation throughout the trial. B, Daily calculated success rate of Monkey 1 plotted against the experimental day. B–D, Red and blue lines indicate values in rewarded (R) and unrewarded trials (NR), respectively. Black lines indicate values in no-cue rewarded trials (ncR). The success rate was calculated by 1 − break rate, where the break rate was the proportion of trials in which the monkey broke the lever-pressing or eye-fixation during a period from the cue onset to the first outcome onset. C, The pressure within the tube for water reward delivery in Monkey 2. The negative deflection near the first outcome onset in rewarded trials represented the monkey's anticipatory sucking, and the positive deflection that followed reflected the arrival of the water drop to the monkey end of the tube. D, Daily calculated mean strength of anticipatory sucking in Monkey 2, measured over a 150 ms window starting 100 ms before the electrical onset of the first outcome, plotted against the experimental day.

The monkeys were first trained for the lever-pressing and eye-fixation with visual stimuli different from those used in the main experiments, and then the 24 stimuli were introduced at once. Neuronal recordings started after the proportion of trials with successful performance of continuous lever-pressing and eye-fixation (success rate, sr) or the pressure to suck the reward-delivery tube before the onset of reward delivery (anticipatory sucking pressure, sp) showed clear dependence (see Analyses of monkey behavior) on the outcome type associated with the cue (water reward vs sound-only acknowledgment), indicating that the monkey had learned the cue-outcome contingency. The eye position was monitored with an infrared pupil position monitoring system (i-rec; http://staff.aist.go.jp/k.matsuda/eye/indexe.html).

No-cue rewarded trials (ncR).

After a part of recordings was completed, we introduced an additional type of trials, which were identical to ordinary rewarded trials, except that visual cues were not presented. The fixation point disappeared during the 350 ms period in which a visual cue was presented in the ordinary trials. No-cue rewarded trials were provided with a probability of 1 of 14.

Experiments with context-dependent cue-outcome contingency.

After cell recordings with the task having fixed cue-outcome contingency were completed, the trial structure was changed so that the cue-outcome contingency depended on the time context. A visual cue presentation and outcome delivery sequence was repeated twice during a continuous lever-pressing and eye-fixation period in each trial (see Fig. 6A). In the first part of the trial, there was a fixed contingency between visual cues and the outcome type that occurred following successful task performance (continuous lever-pressing and eye-fixation). This part of the trial was the same as the previous task with a single cue-outcome sequence in each trial. In particular, the mapping between visual cues and outcome types remained the same as that in that previous task. The cue-outcome mapping was constant throughout all the experiments described in this paper. In the second part of the trial, there was no fixed contingency between the visual cues and outcome types. A water reward was randomly provided to successful performance in 50% of trials, and only a click sound was provided to successful task performance in the remaining trials.

Figure 6.

Trial structure and behavioral results in the experiments with context-dependent cue-outcome contingency. A, Events in a trial. The sequence of a visual-cue presentation and outcome delivery was repeated twice in each trial. Before the first context switch, there was a fixed cue-outcome contingency in the first part of the trial (contingent part), whereas two types of outcome were randomly provided in the second part of the trial (noncontingent part). The order of the contingent and noncontingent parts was reversed after the first context switch and returned to the original order after the second context switch. B, Difference in daily calculated success rate of Monkey 1, after reward-associated cues minus after no-reward-associated cues, in the first and second parts of the trial. C, Difference in daily calculated mean anticipatory sucking strength of Monkey 2, after reward-associated cues minus after no-reward associated cues, in the first and second parts of trial. D, Difference in daily calculated success rate of Monkey 2, after reward-associated cues minus after no-reward associated cues, in the first and second parts of trial. B–D, Black and gray lines indicate values in the first and second parts of the trial, respectively. The values were smoothened over 5 d with equal weights. BS, AS, and A2S indicate the period before the first context switch, that between the first and second context switches, and that after the second context switch, respectively.

As in trials with fixed cue-reward contingency, the monkey pressed the lever to start a trial, and then the fixation point appeared. After continuous lever-pressing and eye-fixation for 400 ms, a visual cue was presented for 350 ms, and an outcome was provided 600 ms after the first cue onset. After a delay of 800 ms starting at the onset of the outcome in the first part, another visual cue was presented for 350 ms and an outcome was provided 600 ms after the second cue onset. We refer to the first part of trial as “contingent part” and the second part as “noncontingent part.” The overall frequency of water reward delivery was identical between the two parts. The monkey had to keep the lever pressed and eye fixed throughout the trial (2800 ms), including both the first and second parts. The fixation spot remained at the center of the screen, except during the cue presentation periods. The stimulus used as a cue in the second part of a trial was always different from that presented in the first part of the trial. A successful trial was followed by an ITI of 800 ms. When the monkey broke eye-fixation or released the lever during a trial, the trial was immediately terminated and a longer ITI of 1000 ms started. An unsuccessful trial was followed by a trial in which the last stimulus associated with the error was presented again as the first cue. Each of the monkeys performed 600–800 trails in each daily session.

We refer to the cues that were associated with the reward and no reward in the contingent part of trails as “reward-associated cues” and “no-reward-associated cues,” respectively. We use these terms even in descriptions of the monkeys' behavior and cell activities in the noncontingent part of the trial, for the sake of convenience. Cell recordings started after the monkey showed clear evidence either in success rate or in anticipatory sucking that it had learned the context dependence of cue-outcome contingency.

Switches of the context for cue-outcome contingency.

After conducting cell recordings for 18 weeks for Monkey 1 and 16 weeks for Monkey 2 with the above-described order of the contingent and noncontingent parts in each trial (the contingent part followed by the noncontingent part), the order of contingent and noncontingent parts was reversed (the “first context switch”). After this first context switch, one of the two types of outcome was randomly provided to successful performance in the first part of the trial, whereas the two outcome types were given on contingency to visual cues in the second part of the trial. After a second series of cell recordings for 20 weeks for Monkey 1 and 17 weeks for Monkey 2, the order of contingent and noncontingent parts in a trial was returned to the original (the “second context switch”). Cell recordings were conducted for 4 weeks for Monkey 1 and 2 weeks for Monkey 2 after the second context switch. The mapping from the 24 visual cues to the 2 outcome types remained the same across the two context switches. There was no sign given to the monkeys indicating the context switches.

Cell recordings.

Single-cell activities were recorded from the PRh and area TE with tungsten microelectrodes (Frederick Haer) while the monkeys were performing the behavioral tasks. Two tungsten microelectrodes were penetrated simultaneously from the dorsal surface of the brain through stainless steel guide tubes to one of the two areas, at an exactly vertical angle for perirhinal recordings and at an angle (15 degrees) tilted to the lateral (as the electrode descended ventrally) for TE recordings. Locations of the perirhinal cortex and area TE were determined from MRIs taken before the preparatory surgery. The dura positions at the ventral surface of the brain were determined for each of the recording penetrations by observing the characteristic noises that occurred when the electrode made contact with the dura. The recorded locations ranged from anterior 10 to anterior 17 mm with respect to the interaural line (Fig. 2). All the recordings were made from the left hemisphere. PRh and TE recordings were conducted on alternative days, except days after the second context switch in the experiments with context-dependent cue-outcome contingency. In these latter days, we focused on recordings in PRh. Spikes from a single cell were isolated by MSD (Multi Spike Detecting system, Alpha Omega). The rastergrams were monitored online for isolated spikes; and when the recorded cell did not appear to change the activity in relation to the task events, we advanced the electrode to find another cell. Cells recorded for a minimum of 100 trials were used for neuronal analyses unless otherwise noted.

Figure 2.

Recording locations in TE and PRh shown on bottom views of the monkey brains. Right inset, MRI image at frontoparallel plane 13 mm from the ear bar position in Monkey 1. rs, Rhinal sulcus; amts, anterior middle temporal sulcus.

Analyses of monkey behavior.

We calculated the proportion of trials in which the monkey broke the lever-pressing or eye-fixation during the period extending from the onset of cue presentation to the onset of outcome (break rate) and then defined the success rate (sr) by 1 − break rate. We used the success rate as an indicator of the monkey's motivation to complete the trial or the trial part, which in turn indicates the monkey's level of anticipation of the water reward provided at the end of the trial or trial part.

The pressure in the water-reward delivery tube was continuously monitored by a pressure sensor. The pressure was influenced by both the water delivery and the monkey's sucking of the tube. There was an ∼50 ms delay from the onset of the electrical signal to the actual supply of water to the monkey's end of the tube. We found that one of the two monkeys (Monkey 2) started to suck the tube ∼100 ms before the electrical onset of reward delivery in trials in which one of the stimuli associated with water reward was presented as the cue. We measured the pressure of this anticipatory sucking as another indicator of monkey's anticipation of the water reward. The mean pressure was measured over a 150 ms window starting 100 ms before the electrical signal onset of the outcome, with subtraction of the mean baseline pressure measured over a 150 ms window placed immediately before the onset of visual cues (the first visual cue for trials with repeated cue-outcome sequence).

To examine the significance of the difference within each day of behavioral responses to reward-associated and no-reward-associated cues, we also calculated the success rate and anticipatory sucking separately for individual stimuli, and then averaged them across the two stimuli belonging to the same set and associated with the same outcome type. We thus obtained six pairs of success rates and anticipatory sucking pressures (note that we used six stimulus sets) following reward-associated and no-reward-associated cues. We then applied a paired t test to the six pairs of success rates or anticipatory sucking pressures. All successful trials (continuous lever-pressing and eye-fixation throughout the trial) were included in the analyses of anticipatory sucking.

Analyses of cell data.

Trials with successful performance (continuous lever-pressing and eye-fixation throughout the trial) were used for the analyses of cell activities, both in relation to the cue presentation and to the outcome delivery. Responses to the visual cue presentation were measured by the mean firing rate in a 500 ms window starting 100 ms after the cue presentation onset, whereas baseline activities were determined by the mean firing rate in a 400 ms window placed immediately before the cue onset (the first cue onset for trials with repeated cue-outcome sequences in each trial). The responsiveness of a cell to the cues was examined by two-way ANOVA with baseline/response and stimulus as factors. For cells recorded in trials with a single cue-outcome sequence, a cell was regarded as cue-responsive if it showed a significant baseline/response main effect or significant interaction (p < 0.025 each). For cells recorded in trials with repeated cue-outcome sequences, responses were measured for two response windows placed in the first and second parts of the trial, and a cell was regarded as responsive if it showed a significant baseline/response main effect or significant interaction in either response window (p < 0.0125 each).

Responses to visual cues were further analyzed for their selectivity to visual properties and associated outcome types (water reward vs sound-only acknowledgment) by using a nested ANOVA. The 24 stimuli were composed of 6 stimulus sets and 2 associated outcome types (Fig. 1). The sets were crossed with the associated outcome types, and each combination of set and associated outcome type contained two stimuli. The selectivity for sets and that for associated outcome types were examined by comparing the mean SS (sum of the squared deviation from the mean) for sets or associated outcome types with the mean SS for stimuli within each combination of set and associated outcome type.

For cells recorded during a period before the first context switch in the second series of experiments, the periods in which each cell showed significantly larger activities in response to reward-associated cues and water reward delivery in the first (contingent) part of the trial were determined by examining the significance of the difference in a 200 ms window moving in 20 ms steps (p < 0.05, two-tailed Mann–Whitney U test). The periods with significant differences were determined with respect to the center of the window. The center of the 200 ms window was moved from 100 ms after the first cue onset to the offset of the second cue. When a cell showed significant response differences in two periods that were each longer than 300 ms, the overall period was determined from the beginning of the first period to the end of the second period.

Activities following the outcome onset were analyzed in a 500 ms window starting 200 ms after the electrical signal onset of the outcome. The click sound immediately followed the electrical signal, whereas the water drop arrived at the monkey's end of the water delivery tube after a delay of ∼50 ms. Because the click sound was provided, following successful task performance, in both rewarded and unrewarded trials, the monkeys could discriminate the positive from negative feedback only after the water drop arrived.

Results

Experiments with fixed cue-outcome contingency

We first examined representations of cue-outcome associations in PRh and TE cells. We let two monkeys experience a fixed contingency between visual cues and outcome types (water reward and click sound only) and then recorded activities of cells from PRh and TE during the task. Each trial had a single cue-outcome sequence in the task (Fig. 3A).

Monkeys' behavior

After the monkey had experienced the cue-outcome contingency for 35 d (in Monkey 1) or 27 d (in Monkey 2), cell recordings were conducted for 164 d (in Monkey 1) or 157 d (in Monkey 2). Clear evidence for the monkey's anticipation of outcome type was found in the success rate for Monkey 1 and in the anticipatory sucking for Monkey 2 during the period of cell recordings. The success rate was higher (in Monkey 1) or the anticipatory sucking was stronger (in Monkey 2) in trials in which a water reward was delivered for successful performance (continuous lever-pressing and eye-fixation) (“rewarded trials”) than in trials in which a sound-only acknowledgment was provided for successful performance (“unrewarded trials”) (Fig. 3B–D).

We measured the success rate for the period from cue presentation onset to the first outcome onset. Monkey 1 maintained the lever-pressing and eye-fixation almost perfectly in rewarded trials (the mean success rate averaged over the 164 d in which cell recordings were conducted, 0.98), whereas it frequently broke eye-fixation or lever-pressing in unrewarded trials (average success rate, 0.81). When a Wilcoxon signed rank test was applied to 164 pairs of daily calculated success rates in rewarded and unrewarded trials, the difference was highly significant (p < 0.00001, two-tailed; Fig. 3B). Moreover, when the success rates in rewarded and unrewarded trials were calculated within each day separately for the six sets of stimuli and a paired t test was applied to the six pairs (see Materials and Methods), the success rates in rewarded trials were significantly greater than those in unrewarded trials (p < 0.05, one-tailed) for all 164 d.

The success rates were more similar between rewarded and unrewarded trials in Monkey 2 (0.95 for rewarded trials and 0.91 for unrewarded trials, averaged over the 157 d). Although the difference over the 157 d was significant (p < 0.00001, two-tailed Wilcoxon signed rank test), the difference within each day was significant only in a half of the days (p < 0.05, one-tailed paired t test, 72 of 157 d).

Monkey 2 showed clear anticipatory sucking in rewarded trials but not in unrewarded trials. The negative pressure in the reward delivery tube, reflecting the anticipatory sucking, started ∼100 ms before the electrical signal onset of the reward delivery (Fig. 3C). The positive deflection that started ∼50 ms after the electrical signal onset of reward delivery reflected the arrival of the water drop to the monkey's end of the tube. Therefore, we measured the strength of anticipatory sucking by the mean negative deflection of pressure in a 150 ms window starting at 100 ms before the electrical onset of outcome from the baseline pressure measured in a 150 ms window before the cue onset. The pressure was then averaged over all successful rewarded trials or over all successful unrewarded trials within each day. The magnitudes of mean sucking pressure in rewarded trials were significantly larger than those in unrewarded trials, when a Wilcoxon signed rank test was applied to 157 pairs of daily calculated mean pressure values in rewarded and unrewarded trials (p < 0.00001, two-tailed; Fig. 3D). When a paired t test was applied to the six pairs of mean pressure measurements calculated within each day for the six stimulus sets, the anticipatory sucking pressure in rewarded trials was significantly larger than that in unrewarded trials for most days (136 of 157 d; p < 0.05, one-tailed). Most of the remaining 21 d (19 of 21 d) occurred around the 140th day, in which the monkey did not show anticipatory sucking either in rewarded or unrewarded trials. We removed these 21 d from the analyses of cell activities for Monkey 2.

Cell activities

Cell responses to the visual cue presentation were analyzed in a 500 ms window starting at 100 ms after the cue onset (“cue-response window”). This window did not overlap the outcome onset. Approximately 60% of the tested PRh cells (224 of 371 in the two monkeys together: 119 of 203 in Monkey 1 and 105 of 168 in Monkey 2) and ∼80% of the tested TE cells (432 of 543: 233 of 319 in Monkey 1 and 199 of 224 in Monkey 2) significantly responded to cues: they showed either significant baseline/response main effects or significant interactions (p < 0.025 each, two-way ANOVA with baseline/response and stimulus as factors) (“cue-responsive cells”). These responses to visual cues were further analyzed in terms of the selectivity for outcome types associated with the cues and for visual properties of the cues. Selectivity for associated outcome types was found in both PRh and TE (Fig. 4A,B): 45% of the cue-responsive PRh cells (100 of 224: 68 of 119 in Monkey 1 and 32 of 105 in Monkey 2) and 22% of the cue-responsive TE cells (96 of 432: 34 of 233 in Monkey 1 and 62 of 199 in Monkey 2) showed significant outcome-type main effects in nested ANOVA (p < 0.05; see the Materials and Methods). The selectivity for visual properties of stimuli was more prevalent in TE than in PRh (Fig. 4B). Significant selectivity for stimulus sets was found in 52% of the cue-responsive TE cells (223 of 432: 112 of 233 in Monkey 1 and 111 of 199 in Monkey 2) (p < 0.05, stimulus set main effects in nested ANOVA), whereas the proportion in PRh (13%, 29 of 224, 8 of 119 in Monkey 1 and 21 of 105 in Monkey 2) was significantly smaller (p < 0.00001 in either monkey, z test).

Figure 4.

Responses of cells in PRh and TE to visual cues. A, Mean activities averaged over cells that showed significant selectivity for associated outcome types in their cue responses. Red and blue lines indicate activities in rewarded (R) and unrewarded trials (NR), respectively. Activities were separately averaged for PRh and TE cells and for the cells with larger and smaller responses in rewarded trials than in unrewarded trials. Bin width was 10 ms, and values were smoothened over five bins with equal weights. Shading represents SEM across cells. B, Proportions of cells that showed significant selectivity for associated outcome types (Outcome) and stimulus sets (Set) in PRh and TE. C, Proportions of cells having a value within the bin range of the ratio of (the variance due to associated outcome types)/(the variance due to associated outcome types + the variance due to stimulus sets) in PRh and TE.

Both the selectivity of cue responses for associated outcome types and for stimulus sets appeared in PRh cells with longer onset latency than in TE cells. Nested ANOVA was applied to activities of individual cells in 10 ms bins moving in 10 ms steps, and the onset latency of selectivity was determined in individual cells by the first of three consecutive bins with significant differences. Based on these calculations, the onset latency in TE cells was 137.4 ± 4.9 ms (mean ± SEM across cells) for the selectivity for stimulus sets and 195.4 ± 12.1 ms for the selectivity for associated outcome types, whereas in PRh cells it was 162.5 ± 25.5 ms for the selectivity for stimulus sets and 241.5 ± 7.0 ms for the selectivity for associated outcome types. The selectivity for associated outcome types appeared in TE cells on average 46.1 ms earlier than in PRh cells.

To further compare the nature of selectivity in individual cells between PRh and TE, we calculated the ratio of the response variance due to associated outcome types to a sum of the variance due to associated outcome types and that due to visual properties in individual cells. The ratio was almost evenly distributed from 0 to 1 in PRh (with a mean of 0.50, 0.63 in Monkey 1 and 0.35 in Monkey 2), whereas values near 0 dominated in TE (with a mean of 0.20, 0.19 in Monkey 1 and 0.21 in Monkey 2) (Fig. 4C). The difference between the two distributions was highly significant (p < 0.00001 in either monkey, two-tailed Mann–Whitney U test). These results indicated that the translation of the cues from visual properties to the meanings (associated outcome types) of the cues occurred much more strongly in PRh than in TE.

The selectivity of cells for the associated outcome types could not be explained by differences in the preciseness of gaze fixation between rewarded and unrewarded trials. The length of movements of gaze position per trial was calculated within the cue-response window: the averaged gaze position was determined in 30 ms windows and the change of the position from the previous window to the current window was summed over the 500 ms window. It was generally larger in rewarded trials than in unrewarded trials. For example, in the experiments with the fixed cue-outcome contingency, it was 0.698 ± 0.0196 degrees (visual angle, mean ± SEM across cells) in rewarded trials and 0.568 ± 0.020 degrees in unrewarded trials during recordings from the 96 TE cells that showed significant selectivity for associated outcome types and 0.645 ± 0.011 degrees in rewarded trials and 0.513 ± 0.011 degrees in unrewarded trials during recordings from the 100 PRh cells that showed significant selectivity for associated outcome types. The differences were highly significant (p < 0.00001 both, two-tailed Wilcoxon signed rank test). To examine the possibility that the differences in eye movements caused the differences in cue responses between rewarded and unrewarded trials, we calculated the correlation between neuronal activities and the length of gaze movements across trials, after the means of neuronal activities and gaze movement length calculated for each stimulus were subtracted from the values in individual trials. Significant correlation in the direction that can explain the cell's selectivity for associated outcome type (positive for cells with larger activities in rewarded trials and negative for cells with smaller activities in rewarded trials) were found only in 5 of the 96 TE cells and 5 of 100 PRh cells. These proportions were not significantly larger than those expected by chance (p = 0.18 in TE and p = 0.18 in PRh, Poisson test). The differences in gaze movements could not cause the selectivity of cells for the associated outcome type.

No-cue rewarded trials

To examine whether the selectivity of cue responses for associated outcome types was a mere reflection of the monkey's expectation of outcome type or it depended on the visual inputs, we introduced trials in which no visual cue (except a disappearance of the fixation spot) was presented while the water reward was provided to successful performance (“no-cue rewarded trials”).

Monkeys' behavior

In Monkey 1, daily calculated success rates in no-cue rewarded trials (mean, 0.99) were significantly larger than those in unrewarded trials (mean, 0.80) when the pair of success rate was pooled over the 72 d in which no-cue rewarded trials were provided (p < 0.00001, two-tailed Wilcoxon signed rank test). The success rates in no-cue rewarded trials were even significantly larger than those in ordinary rewarded trials (mean, 0.98, p < 0.00001; Fig. 3B). The success rate in unrewarded trials was also calculated separately in each day for each of the 12 stimuli. When these 12 rates were compared with the rate in no-cue rewarded trials obtained for the same day, the success rate in no-cue rewarded trials was significantly larger than those in unrewarded trials for all 72 d (p < 0.05, one-tailed t test).

In Monkey 2, the mean pressure of anticipatory sucking averaged over no-cue rewarded trials for each day was significantly larger than that averaged over unrewarded trials (p < 0.00001, two-tailed Wilcoxon signed rank test) and was not significantly different from that in rewarded trials (p = 0.49), when the comparison was pooled over the 109 d in which no-cue rewarded trials were provided (Fig. 3D). When the pressure in unrewarded trials was separately averaged in each day for each of 12 stimuli and those 12 values were compared with the averaged pressure in no-cue rewarded trials of the same day, the averaged pressure in no-cue rewarded trials was significantly larger than the values in unrewarded trials (p < 0.05, one-tailed t test) on the sixth and all later days after the introduction of no-cue rewarded trials, but not in 4 of the 5 initial days (Fig. 3D). Therefore, we decided to exclude the initial 6 d, in both monkeys, after the introduction of no-cue rewarded trials from the analyses of cell activities in no-cue rewarded trials.

Thus, the success rate (in Monkey 1) or anticipatory sucking pressure (in Monkey 2) indicated that both monkeys anticipated the reward in no-cue rewarded trials as strongly as they did in the ordinary rewarded trials. Then, if the selectivity of cue responses had merely reflected the monkey's expectation of outcome type, activities in no-cue rewarded trials should have been identical to those in ordinary rewarded trials.

Cell activities

Cue responses of PRh and TE cells almost disappeared in no-cue rewarded trials (Fig. 5). Of 87 cue-responsive PRh cells recorded in the sessions including no-cue rewarded trials, 33 PRh cells showed significant differences in their cue responses between unrewarded and ordinary rewarded trials (p < 0.05, main effects of outcome type, nested ANOVA): 23 and 10 cells showed larger and smaller responses, respectively, in rewarded trials than in unrewarded trials. In both groups of cells as populations, activities in no-cue rewarded trials (1.55 ± 1.55 spikes/s, mean ± SEM across cells, for the 23 cells, −3.26 ± 2.75 spikes/s for the 10 cells, after subtraction by the baseline activity) were not significantly different from zero (p = 0.18 for the 23 cells and p = 0.69 for the 10 cells, two-tailed Wilcoxon signed rank test). Although the averaged responses of the 23 cells in no-cue rewarded trials appear to have some positive activities around the cue offset, their activities were not significantly different from zero even in 100-ms-long windows (100–200, 200–300, 300–400, 400–500 ms after the cue onset) (p > 0.18, two-tailed Wilcoxon signed rank test). In the 23 cells, the activities in no-cue rewarded trials were significantly smaller than their cue responses in ordinary rewarded trials (6.15 ± 1.82 spikes/s) (p = 0.000077, two-tailed Wilcoxon signed rank test) and larger than their activities in unrewarded trials (−1.03 ± 1.21 spikes/s) with a marginal significance (p = 0.089) (Fig. 5, top left). In the other 10 PRh cells, probably because the number of cells was small, the differences between activities in no-cue rewarded trials and those in ordinary rewarded (−5.77 ± 3.58 spikes/s) and unrewarded trials (−1.02 ± 2.99 spikes/s) did not reach a significance level (p = 0.084 and p = 0.99, respectively) (Fig. 5, top right).

Figure 5.

Mean activities of cue-responsive cells in PRh and TE in no-cue rewarded trials (ncR) compared with those in unrewarded (NR) and ordinary rewarded trials (R). Cue-responsive cells in each area were divided into two groups of cells that showed either larger or smaller responses in ordinary rewarded trials relative to unrewarded trials. Red lines indicate values in ordinary rewarded trials. Blue lines indicate values in unrewarded trials. Black lines indicate values in no-cue-rewarded trials. Bin width was 10 ms, and values were smoothened over five bins with equal weights. Shading represents SEM across cells.

Of 162 cue-responsive TE cells recorded in the sessions including no-cue rewarded trials, 45 cells showed significant differences in their cue responses between unrewarded and ordinary rewarded trials (p < 0.05, main effects of outcome type, nested ANOVA): 24 and 21 cells showed larger and smaller responses in rewarded trials than in unrewarded trials. In both groups of cells, activities in no-cue rewarded trials (−1.33 ± 0.90 spikes/s, mean ± SEM across cells, for the former group, −0.13 ± 0.26 spikes/s, for the latter group) were not significantly different from zero (p = 0.14 for the former group and p = 0.96 for the latter group, two-tailed Wilcoxon signed rank test). In the former group, the activities in no-cue rewarded trials were significantly smaller than their activities in ordinary rewarded trials (8.85 ± 2.19 spikes/s) (p = 0.00012, two-tailed Wilcoxon signed rank test) and unrewarded trials (3.41 ± 1.48 spikes/s) (p = 0.0011) (Fig. 5, bottom left). In the latter group, probably because the activity differences between ordinary rewarded and unrewarded trials were small, the differences between activities in no-cue rewarded trials and those in ordinary rewarded (−0.30 ± 0.54 spikes/s) and unrewarded (1.38 ± 0.69 spikes/s) trials did not reach a significance level (p = 0.74 and p = 0.26, respectively) (Fig. 5, bottom right).

The dissociation between behavioral and cell results in no-cue rewarded trials suggests that the differential components of cue responses of PRh and TE cells selective for associated outcome types did not merely reflect the monkey's outcome anticipation or subsequent change in motivation level but depended on visual inputs.

Experiments with context-dependent cue-outcome contingency

After confirming the results that cue responses of PRh and TE cells represented the outcome types associated with the cues, we then moved to examine whether the representations depended on the context in which the monkeys had experienced the cue-outcome contingency. To answer this question, we modified the task so that the cue-outcome sequence was repeated twice in each trial (Fig. 6A). Although the outcome type (water reward or sound only) was contingent to the cue in the first part, one of the two types of outcome was randomly provided, regardless of the cue, in the second part. Thus, the consistent cue-outcome contingency appeared only in the first part of the trial (“time context” of the cue-outcome contingency). We will refer to the cues that were associated with reward and no reward in the first part of the trial as “reward-associated cues” and “no-reward-associated cues,” respectively, even when they appeared in the second part of trial, for the sake of convenience. The probability of rewards was identical in the first and second parts of a trial (Hammond, 1980). After experiencing the new task paradigm for 24 d (in Monkey 1) or 20 d (Monkey 2), cell recordings started. The following two sections will describe the behavioral and cellular results obtained in a period of 89 d (in Monkey 1) or 80 d (in Monkey 2) of cell recordings in this task condition.

Monkeys' behavior

Both monkeys had become adapted to the time context of cue-outcome contingency during the period of cell recordings. The monkeys' behaviors showed clear signs of anticipating the outcome type in the first part of the trial, whereas the signs were much less clear in the second part of the trial.

In Monkey 1, the mean success rate in the first part of the trial was 0.98 and 0.96 after the presentation of reward-associated and no-reward-associated cues, respectively. When trials were sorted according to the cue presented in the second part of the trial, the success rate in the second part of the trial was 0.98 after both reward-associated and no-reward associated cues. The success rates after reward-associated and no-reward-associated cues will be denoted as sr(R) and sr(NR), respectively, and sr(R) – sr(NR) in the first and second parts of the trial will be denoted as dsr(1) and dsr(2), respectively. When a Wilcoxon signed rank test was applied to 89 pairs of daily calculated dsr(1) and dsr(2), dsr(1) was significantly larger than dsr(2) (p < 0.00001, two-tailed) (Fig. 6B).

To examine the significance of this difference for individual days, mean success rates after reward-associated and no-reward-associated cues were calculated in each day for the six stimulus sets separately, and a paired t test was applied to the six pairs of values. dsr(1) was significantly larger than dsr(2) in 31 of the 89 d (p < 0.05, one-tailed). The differences between dsr(1) and dsr(2) were insignificant (p > 0.05, one-tailed) for the other 57 d. Significant differences in the opposite direction, dsr(2) > dsr(1), were found in only 1 d.

In Monkey 2, the mean anticipatory sucking pressure in the first part of the trial was 30.47 (in arbitrary units) after reward-associated cues and −5.12 after no-reward-associated cues, whereas the difference in the mean anticipatory sucking pressure was smaller in the second part of the trial (5.30 after reward-associated cues and 2.91 after no-reward-associated cues). The anticipatory sucking pressure after reward-associated and no-reward-associated cues will be noted as sp(R) and sp(NR), respectively, and sp(R) – sp(NR) in the first and second parts of the trial will be noted as dsp(1) and dsp(2), respectively. When a Wilcoxon signed rank test was applied to 80 pairs of daily calculated dsp(1) and dsp(2), dsp(1) was significantly larger than dsp(2) (p < 0.00001, two-tailed) (Fig. 6C). Moreover, when the magnitudes of mean pressure of anticipatory sucking after reward-associated and after no-reward-associated cues were calculated in each day separately for 6 stimulus sets, dsp(1) was significantly larger than dsp(2) for all of the 80 d (p < 0.05, one-tailed paired t test).

The adaptation of the monkeys' anticipatory behavior to the time context of the cue-outcome contingency was incomplete, although strong. In Monkey 1, whereas dsr(2) was much smaller than dsr(1), sr(R) in the second part of the trial was significantly larger than sr(NR) in that part of the trial when compared over 89 d (p = 0.00069, two-tailed Wilcoxon signed rank test). Also, in Monkey 2, whereas dsp(2) was much smaller than dsp(1), sp(R) in the second part of the trial was significantly larger than sp(NR) in that part of the trial when compared over 80 d (p < 0.00001, two-tailed Wilcoxon signed rank test).

PRh cell activities

The question here was whether the selectivity of cue responses for associated outcome types adapted to the time context of the cue-outcome contingency. We found different answers for PRh and TE cells. Of 113 cue-responsive PRh cells (61 in Monkey 1 and 52 in Monkey 2; 217 PRh cells were recorded in the period), 85 and 28 cells (40 and 12 in Monkey 1, 36 and 16 in Monkey 2) showed numerically larger and smaller cue responses, respectively, after reward-associated cues than after no-reward-associated cues (for mean activities averaged over the first and second parts of the trial). Activities of an example of the 85 cells are shown in Figure 7, and activities averaged over the 85 cells are shown in Figure 8A. In these figures, the activities in the second part of the trial were sorted by the cue presented in that part of the trial, but not by the outcome type given in that part of the trial. The mean firing rates in the cue-response window after reward-associated and no-reward-associated cues will be denoted as fr(R) and fr(NR), respectively. We normalized fr(R) and fr(NR) by a sum of fr(R) and fr(NR) over both the first and second parts of the trial (a sum of four values). The normalized fr(R) and fr(NR) will be referred to as nfr(R) and nfr(NR), respectively. The selectivity of cue responses for associated outcome types, which was represented by nfr(R) − nfr(NR), was much larger in the first part of the trial than in the second part (p < 0.00001, p < 0.00001 in Monkey 1 and p = 0.018 in Monkey 2, two-tailed Wilcoxon signed rank test). However, for the 28 PRh cells that showed numerically smaller cue responses after reward-associated cues, nfr(R) − nfr(NR) was not significantly different between the first and second parts of the trial (p = 0.49, p = 0.68 in Monkey 1 and p = 0.16 in Monkey 2), although the signed differences in both parts of the trial were significantly smaller than zero (p = 0.00075 and 0.0013 for the first and second parts, respectively). Thus, the PRh cells with smaller cue responses after reward-associated cues had a different nature from the PRh cells with larger cue responses after reward-associated cues.

Figure 7.

Cue responses of an example PRh cell recorded in the experiments with context-dependent cue-outcome contingency before the first context switch. Trials were sorted by the cue presented in each trial part. Red and blue lines indicate activities in trial parts in which reward-associated and no-reward-associated cues were presented, respectively. The “reward-associated” and “no-reward-associated” categories were defined by the contingency that existed only in the contingent part of the trial. Activities were averaged over trial parts with stimuli belonging to the same stimulus set and associated with the same outcome type. Shading represents SEM across trials. The bin width was 10 ms, and values were smoothened over five bins with equal weights.

Figure 8.

Mean activities of cue-responsive PRh cells that showed numerically larger cue responses after reward-associated cues than after no-reward-associated cues (in responses averaged between the first and second parts of the trial). Left graphs, Activities in the first part of the trial sorted by the first cue. Right graphs, Activities in the second part of the trial sorted by the second cue. Red and blue lines indicate activities after reward-associated and no-reward-associated cues, respectively. The “reward-associated” and “no-reward-associated” categories were defined by the contingency that existed only in the contingent part of the trial. A, For the cells recorded before the first context switch (BS). B, For the cells recorded between the third and 12th days after the first context switch: AS (3–12). C, For the cells recorded between the 13th day after the first context switch and the last day before the second context switch: AS (13-end). D, For the cells recorded on the third and later days after the second contest switch: A2S (3-end). A–D, Bin width was 10 ms, and values were smoothened over five bins with equal weights. Shading represents SEM across cells.

To statistically confirm the dependence of PRh cells' selectivity on the time context, we compared nfr(R) − nfr(NR) between the first and second parts of the trial (referred to as dnfr(1) and dnfr(2), respectively) over the cell population. All of the 113 cue-responsive PRh cells (61 in Monkey 1 and 52 in Monkey 2) were put into this analysis. As can be seen in Figure 9A, values of dnfr(1) were much more shifted from zero to the positive side (black line) than values of dnfr(2) (gray line). A Wilcoxon signed rank test applied to 113 pairs of dnfr(1) and dnfr(2) indicated that dnfr(1) was significantly larger than dnfr(2) (p < 0.00001, p < 0.00001 in Monkey 1 but p = 0.072 in Monkey 2, two-tailed Wilcoxon signed rank test). The difference between the first and second parts of a trial appeared only in the positive (i.e., nfr(R) > nfr(NR)), part of the cumulative histogram (Fig. 9A), corresponding to the absence of time-context dependence in the PRh cells with smaller activities after reward-associated cues.

Figure 9.

Quantitative comparison of the selectivity of cue responses for associated outcome types between the contingent and noncontingent parts of the trial. A, Cumulative proportions of cells plotted against nfr(R) − nfr(NR): the normalized cue responses after reward-associated cues subtracted by those after no-reward associated cues, in the first and second parts of the trial (black and gray lines, respectively) for the cells recorded before the first context switch (BS). B, The same as in A but for the cells recorded between the third and 12th days after the first context switch: AS (3–12). C, The same as in A, B but for the cells recorded between the 13th day after the first context switch and the last day before the second context switch: AS (13-end). D, The same as in A–C but for the cells recorded between the third day after the second context switch and the last recording day: A2S (3-end). E, Cumulative proportion of cells plotted against dnfr(1) − dnfr(2): the reward/no-reward difference in the first part of the trial subtracted by that in the second part of the trial, in BS (black line) and AS (3–12) (green line). F, The same as in E but for AS (3–12) (black line) and AS (13-end) (green line). G, The same as in E, F but for AS (13-end) (black line) and A2S (green line).

The dependence of individual cells' selectivities on time context can be quantitatively examined by an interaction in a three-way ANOVA with the associated outcome type (reward, R vs no reward, NR), the time context (contingent vs noncontingent), and stimulus as factors. Of the 113 cue-responsive PRh cells, 36 cells (31 in Monkey 1 and 5 in Monkey 2) showed a significant interaction between the associated outcome type and time context or a significant three-way interaction (p < 0.025 each). The R/NR differences were larger in the first part of the trial in almost all cells (35 of 36, 31 of 31 in Monkey 1 and 4 of 5 in Monkey 2), and activities that occurred after reward-associated cues were larger than those after no-reward-associated cues in most cells (33 of 36, 29 of 31 in Monkey 1 and 4 of 5 in Monkey 2).

These results on both cell population and individual cells indicated that the selectivity of PRh cells' cue responses for associated outcome types was largely specific to the first part of the trial, in which the monkeys had experienced a constant cue-outcome contingency.

TE cell activities

There was no significant difference between the first and second parts of the trial in the selectivity of cue responses for associated outcome types in TE cells. Of 120 cue-responsive TE cells (71 in Monkey 1 and 49 in Monkey 2) (213 cells were recorded during the period), 56 and 64 cells (32 and 39 in Monkey 1, 24 and 25 in Monkey 2) showed numerically larger and smaller activities after reward-associated cues than after no-reward-associated cues, respectively, for mean activities averaged between the first and second parts of the trial. Activities of an example of the 64 cells are shown in Figure 10, and averaged activities of these two groups of cells are shown in Figure 11A. The selectivity of cue responses for associated outcome types did not differ between the first and second parts of the trial for either group. A Wilcoxon signed rank test applied to the 56 and 64 pairs of dfr(1) and dfr(2) both showed no significant difference (p = 0.58, p = 0.56 in Monkey 1 and p = 0.82 in Monkey 2, for the cells with numerically larger activities after reward-associated cues; p = 0.69, p = 0.21 in Monkey 1 and p = 0.51 in Monkey 2, for the cells with numerically smaller activities after reward-associated cues, two-tailed).

Figure 10.

Cue responses of an example TE cell recorded in the experiments with context-dependent cue-outcome contingency before the first context switch. The formats are identical to those in Figure 7.

Figure 11.

Mean activities of cue-responsive TE cells. Left graphs, Activities in the first part of the trial sorted by the first cue. Right graphs, Activities in the second part of the trial sorted by the second cue. The cells were divided into two groups that showed numerically larger and smaller cue responses after reward-associated cues than after no-reward-associated cues (in responses averaged between the first and second parts of the trial). Red and blue lines indicate activities after reward-associated and no-reward associated cues, respectively. Bin width was 10 ms, and values were smoothed over five bins with equal weights. Shading represents SEM across cells. A, For the cells recorded before the first context switch (BS). B, For the cells recorded between the 13th day after the first context switch and the last day before the second context switch: AS (13 − end).

At the individual cell level, three-way ANOVA with the associated outcome type, time context, and stimulus as factors detected either a significant main effect of associated outcome type (p < 0.025) without a significant interaction between the associated outcome type and stimulus (p > 0.05) or a significant interaction between the associated outcome type and stimulus (p < 0.025) without a significant three-way interaction (p > 0.05) in 56 of the 120 cue-responsive TE cells (19 of 71 in Monkey 1 and 37 of 49 in Monkey 2). This ANOVA pattern means that the cells showed consistent R/NR differences in the first and second parts of the trial. Activities after reward-associated cues were larger and smaller in 20 and 36 cells (12 and 7 in Monkey 1, 8 and 29 in Monkey 2), respectively, than those after no-reward-associated cues. Significant interactions between the associated outcome type and time context or significant three-way interactions (p < 0.025 each) were found in only 10 of the 120 cells (2 of 71 in Monkey 1 and 8 of 49 in Monkey 2). This proportion is not significantly larger than those expected by chance (p = 0.13, p = 0.27 in Monkey 1 and p = 0.14 in Monkey 2, Poisson test).

These results on both cell population and individual cells indicated that the selectivity of TE cells' cue responses for associated outcome types was independent from the time context of the cue-outcome contingency.

Context switches

Given that the selectivity of PRh cells' cue responses for associated outcome types was largely specific to the first part of the trial in which the monkeys had consistently experienced the cue-outcome contingency, we moved the cue-outcome contingency to the second part of each trial to examine how fast this context dependence of PRh cells follows changes in the context. After the context switch, there was now fixed cue-outcome contingency in the second part of the trial, whereas one of two outcome types was randomly provided in the first part of the trial. Furthermore, after cells were recorded for 100 d (in Monkey 1) or 85 d (in Monkey 2) in this new task arrangement, we moved the cue-outcome contingency back to the first part of each trial and recorded cells for 18 d (in Monkey 1) or 12 d (in Monkey 2). We will refer to these changes in task as the first and second context switches.

Monkeys' behavior

Both monkeys quickly changed their behavior after the first context switch. In Monkey 1, the R/NR difference in success rate in the second part of the trial, dsr(2), became larger than that in the first part of the trial, dsr(1), from the first day after the switch (Fig. 6B) and continued to be so for 11 consecutive days. Although the difference between dsr(1) and dsr(2) for each day (see Materials and Methods) did not necessarily reach a significance level during those days, a paired t test applied to daily calculated differences in the first 5 d showed a significant difference (p = 0.0012, one-tailed). In most of the 5 consecutive-day windows in the period between the first and second context switches, dsr(2) was significantly larger than dsr(1) as well (82 of 96, p < 0.05, one-tailed paired t test).

After the second context switch, Monkey 1 started to show a larger R/NR difference in success rate in the first part of the trial than in the second part of the trial from the fifth day after the switch (Fig. 6B) and continued to do so for the remaining 14 d until the end of recordings. All sets of 5 consecutive days during this period (fifth to 18th days) showed significantly larger differences in the first part of the trial (p < 0.05, one-tailed paired t test).

In Monkey 2, the R/NR difference in anticipatory sucking pressure in the second part of the trial, dsp(2), became larger than that in the first part of the trial, dsp(1), from the third day after the first context switch (Fig. 6C) and continued to be so for 37 d. A paired t test applied to daily calculated dsp(1) and dsp(2) in the third to seventh days showed a significant difference (p = 0.0056, one-tailed), and dsp(2) was significantly larger than dsp(1) in most sets of 5 consecutive days during the period between the third and 63rd days after the first context switch (52 of 57, p < 0.05, one-tailed paired t test). After the 63rd day until the last day before the second context switch (the 85th day after the first context switch), the differences in anticipatory sucking were small in both the first and second parts of the trial. However, in many of these 23 d, the difference in success rate showed clear differences between the first and second parts of the trial (Fig. 6D). A paired t test applied to success rate differences during 5 consecutive-day windows in this period showed significantly larger differences in the second part than in the first part of the trial (p < 0.05) in a majority of 5 day windows (11 of 19).

After the second context switch, Monkey 2 started to show a larger difference in anticipatory sucking pressure in the first part of the trial than in the second part from the third day after the switch (Fig. 6C) and continued to do so for 10 d until the end of recordings. dsp(1) was significantly larger than dsp(2) in all of 5 consecutive-day windows during this period (p < 0.05, one-tailed paired t test).

The adaptation of the monkeys' anticipatory behavior to the new time context after the first context switch was incomplete, although strong. In Monkey 1, although dsr(1) was significantly smaller than dsr(2) (p < 0.00001, two-tailed Wilcoxon signed rank test), sr(R) in the first part of the trial was larger than sr(NR) in that part of the trial during most of the 100 d between the first and second context switches. Comparison between sr(R) and sr(NR) in the first part of the trial over the 100 d showed a highly significant difference (p < 0.00001, two-tailed Wilcoxon signed rank test). In Monkey 2, although dsp(1) was significantly smaller than dsp(2), sp(R) in the first part of the trial was larger than sp(NR) in that part of the trial in most of the 85 d between the first and second context switches. Comparison between sp(R) and sp(NR) in the first part of the trial over the 85 d showed a highly significant difference (p < 0.00001, two-tailed Wilcoxon signed rank test).

PRh cell activities

The selectivity of PRh cells' cue responses for associated outcome types moved to the second part of the trial after the first context switch, but with a delay of ∼13 d.

Over a period from the 13th day after the first context switch to the last day before the second context switch, the selectivity of PRh cells' cue responses for associated outcome types was mostly limited to the second part of the trial (Fig. 8C). The distribution of R/NR differences in the second part of the trial, dnfr(2), was much more shifted from zero to the positive side (gray line) than that of R/NR differences calculated for activities in the first part, dnfr(1) (black line) (Fig. 9C): dnfr(2) was significantly larger than dnfr(1) (p = 0.000016, p = 0.00092 in Monkey 1 and p = 0.018 in Monkey 2, two-tailed Wilcoxon signed rank test). However, the cells recorded between the third and 12th days after the first context switch showed a pattern more similar to those recorded before the first context switch than to those recorded on later days after the first context switch (Figs. 8B and 9B): dnfr(1) was significantly larger than dnfr(2) in 12 PRh cells recorded between the third and 12th days after the first context switch (p = 0.034, two-tailed Wilcoxon signed rank test). When the difference between the first and second parts of the trial, dnfr(1) − dnfr(2), was compared among the three recording periods, dnfr(1) − dnfr(2) for third to 12th days after the first context switch was not significantly different from that before the first context switch (p = 0.89, two-tailed Wilcoxon signed rank test; Fig. 9E), whereas it was significantly larger than that on the later days after the first context switch (p = 0.0019; Fig. 9F).

After the second context switch, however, the selectivity of PRh cells' cue responses for associated outcome types quickly moved back to the first part (Fig. 8D). In the 19 cue-responsive cells recorded during a period of 12 (in Monkey 1) or 18 d (in Monkey 2) after the second context switch, dnfr(1) was significantly larger than dnfr(2) (p = 0.016, two-tailed Wilcoxon signed rank test) (Fig. 9D), and dnfr(1) − dnfr(2) for this period of recordings was significantly larger than that in the period between the 13th day after the first context switch and the last day before the second context switch (p = 0.00011, two-tailed Wilcoxon signed rank test) (Fig. 9G). Similar conclusions were obtained even for 17 cells recorded during the first 10 d after the second context switch (p = 0.0034 for the differences between dnfr(1) and dnfr(2); p = 0.000048 for the differences in dnfr(1) − dnfr(2) from those cells recorded between the 13th day after the first context switch and the last day before the second context switch).

TE cell activities

Consistent with the observation that selectivity of TE cells' cue responses for associated outcome types did not differ between the first and second parts of the trial before the first context switch, there was no significant difference between dnfr(1) and dnfr(2) in 97 cue-responsive TE cells recorded in the period from the 13th day after the first context switch to the last day before the second context switch (Fig. 11B). Of the 97 cue-responsive TE cells (46 in Monkey 1 and 51 in Monkey 2), 51 and 46 cells (20 and 26 in Monkey 1, 31 and 20 in Monkey 2) showed numerically larger and smaller activities after reward-associated cues than after no-reward-associated cues, respectively. dnfr(1) was not significantly different from dnfr(2) in either group of cells (p = 0.23, p = 0.079 in Monkey 1 and p = 0.83 in Monkey 2, for the cells with numerically larger activities after reward-associated cues; and p = 0.10, p = 0.052 in Monkey 1 and p = 0.68 in Monkey 2, for the cells with numerically smaller activities after reward-associated cues, two tailed Wilcoxon signed rank test).

Cell activities after the outcome onset

Many PRh cells continued to show differential activities selective for outcome type (water reward or sound only) after the outcome onset. The experiments in which the cue-outcome sequence was repeated twice in each trial provided a good opportunity to examine the nature of these differential activities after the outcome onset.

PRh cell activities

We first examined activities after the outcome onset in the noncontingent part of each trial to determine whether the cells differentially responded to the occurrence of two types of outcome even when there was no systematic cue-outcome contingency. Of the 217 PRh cells (109 in Monkey 1 and 108 in Monkey 2) recorded before the first context switch, 48 and 25 cells (37 and 7 in Monkey 1 and 11 and 18 in Monkey 2) showed significantly larger and smaller activities, respectively, after the water delivery than after the sound-only acknowledgment (in the 500 ms window starting at 200 ms after the electrical onset of outcome, p < 0.05, two-tailed Wilcoxon signed rank test) in the second (noncontingent) part of the trial (Fig. 12A, left). Of the 278 PRh cells (100 in Monkey 1, 178 in Monkey 2) recorded during the period from the 13th day after the first context switch to the last day before the second context switch, 57 and 9 cells (44 and 5 in Monkey 1, 13 and 4 in Monkey 2) showed significantly larger and smaller activities, respectively, after the water delivery than after the sound-only acknowledgment in the first (noncontingent) part of the trial (Fig. 12A, right). Because there was no cue-outcome contingency in the second part of the trial before the first context switch and in the first part of the trial between the first and second context switches, these differential activities in PRh cells appear to have been evoked by the actual delivery of the outcomes.

Figure 12.

Responses of PRh cells to reward delivery. A, Mean activities of PRh cells selected by significantly larger responses to water delivery than to sound-only acknowledgment in the noncontingent part of the trial. Left graphs, Activities of cells recorded before the first context switch (BS). Right graphs, Activities of cells recorded between the 13th day after the first context switch and the last day before the second context switch: AS (13 − end). Trials were sorted by actual outcome types in the noncontingent part of trial in the top graphs and by cues in the contingent part of trial in the bottom graphs. Bin width was 10 ms, and values were smoothened over five bins with equal weights. Shading represents SEM across cells. B, Periods during which each cell showed significantly larger activities in response to the presentation of reward-associated cues and the delivery of water rewards. The analysis was applied to activities in the first (contingent) part of trial. The 70 PRh cells included in the analysis were recorded during the BS period and selected for significantly larger cue responses after reward-associated cues or significantly larger outcome responses to the water reward in the first part of trial. A line indicates a cell. Gray parts in each line represent the periods in which the cell showed significant differences in a 200 ms window moving with 20 ms steps. The cells were aligned from the bottom to the top according to the length of persistent significant differences.

Many of the PRh cells that showed significantly larger outcome responses to the water delivery than the sound-only acknowledgment in the noncontingent part of trial also showed significantly larger cue responses after reward-associated cues than after no-reward-associated cues in the contingent part of the trial (67 of 105 cells: 56 of 81 in Monkey 1 and 11 of 24 in Monkey 2, p < 0.05, two-tailed Wilcoxon signed rank test). The outcome responses of these 67 cells in the noncontingent part of the trial might have represented errors in reward prediction (Schultz et al., 1997). However, most of them (61 of 67: 55 of 56 in Monkey 1 and 6 of 11 in Monkey 2) also showed significantly larger outcome responses to the water delivery than the sound-only acknowledgment in the contingent part of the trial (p < 0.05, two-tailed Wilcoxon signed rank test). Moreover, the differences between outcome responses after the water delivery and sound-only acknowledgment, in the 67 cells, were significantly larger or comparable in the contingent part of the trial compared with those in the noncontingent part of the trial (p = 0.0010, p = 0.00011 in Monkey 1 and p = 0.46 in Monkey 2, two-tailed Wilcoxon signed rank test). Therefore, these activities of the PRh cells could not reflect errors in reward prediction.

In the contingent part of the trial, many PRh cells maintained differential activities selective for outcome types from some time after the cue onset through the outcome delivery until the perception of the next cue (Fig. 12B). Before the first context switch, 70 PRh cells (56 in Monkey 1 and 14 in Monkey 2) showed significantly larger activities after reward-associated cues in the cue-response window or after water reward delivery in the outcome-response window (500 ms starting at 200 ms after the outcome onset) (p < 0.025 each, two-tailed Mann–Whitney U test) in the first part of the trial. When the statistical significance of the differential activity was examined in these cells with a 200 ms window moving at 20 ms steps (see Materials and Methods), the duration of the period (determined by the center of window) in which the cell consistently showed significantly larger activities in response to reward-associated cues and water reward delivery was 1091 ± 56 ms (mean ± SEM across cells; 1192 ± 52 ms in Monkey 1 and 688 ± 143 ms in Monkey 2). More than a half of the cells (42 of 70: 38 of 56 in Monkey 1 and 4 of 14 in Monkey 2) showed persistent significant differences for a period longer than 1000 ms.

TE cell activities

TE cells did not show outcome-dependent activities after outcome onset in the noncontingent part of the trial. Of the 213 TE cells (147 in Monkey 1 and 66 in Monkey 2) recorded before the first context switch, only 10 cells (6 in Monkey 1 and 4 in Monkey 2) showed significant outcome responses in the second (noncontingent) part of the trial (in the 500 ms window starting at 200 ms after the electrical onset of outcome, p < 0.05, two-tailed Wilcoxon signed rank test). Of the 171 TE cells (101 in Monkey 1 and 70 in Monkey 2) recorded during the period from the 13th day after the first context switch to the last day before the second context switch, only 14 cells (10 in Monkey 1 and 4 in Monkey 2) showed significant outcome responses in the first (noncontingent) part of the trial. The proportions of 10 of 213 and 14 of 171 were not significantly larger than those expected by chance (p = 0.13 and p = 0.11, respectively, Poisson test). Thus, TE cells did not respond differentially to reward and no reward when there was no consistent cue-outcome contingency.

Discussion

We found that PRh cells represented, in their responses to visual cues, the outcome type (water reward or sound only) contingent to the cue (Fig. 4) and that this representation appeared only in the trial part in which the monkeys had experienced the stimulus-outcome contingency (Figs. 7, 8, and 9). Thus, PRh cells not only represented the stimulus-outcome contingency but also represented the time context in which the contingency had existed. PRh is not specific to the representation of sensory and associative properties of objects themselves (Eichenbaum et al., 2007; Ranganath and Ritchey, 2012) but may represent broader information about objects, including the time context in which the objects are associated with particular outcomes.

This time-context-dependent representation of stimulus-outcome contingency in PRh cells was not a mere reflection of the monkey's expectation of particular outcomes or the consequent changes in motivation level. We found clear dissociations between the monkey's behavior and PRh cell activities in two situations. First, when the monkeys had experienced a sequence of no visual stimulus and water reward, the monkeys showed a full anticipation of the reward (Fig. 3), whereas activities of PRh cells were not significantly different from their baseline activities (Fig. 5). Second, after the first context switch in the second series of experiments, the monkeys started to show stronger expectation of outcome types in the newly contingent part of the trial than in the previously contingent part of the trial within 3 d (Fig. 6), whereas PRh cell activities started to adapt to the context switch after another 10 d delay (Figs. 8 and 9). The first dissociation also indicated that the representation of associated outcome in PRh was conditional to the existence of complex visual feature inputs. PRh cells may be located in the middle of the translation of visual inputs to the associated outcome types, and not just representing outputs of the translation. The second dissociation suggested that the representation of stimulus-outcome contingency in PRh cells reflected the long-term experience of the contingency for the past 10 days or longer.

PRh has appropriate anatomical connections for the representation of stimulus-outcome contingency in a particular time context. It receives information about visual properties of objects from TE (Webster et al., 1991; Suzuki and Amaral, 1994; Saleem and Tanaka, 1996). It may receive information of reward by the projections from dopamine cells in the ventral tegmental area and those from the amygdala and orbitofrontal cortex (Goldsmith and Joyce, 1996; Stefanacci et al., 1996; Kondo et al., 2005). Information about the time context may originate in the hippocampus. Cells in CA1 of the hippocampus map time points in repeated task sequences (Eichenbaum, 2013). The information of time mapping may propagate from CA1 to PRh directly or via the entorhinal cortex. It has been found that PRh cells represented the time of presentation as well as identity of visual stimuli when their time sequences were encoded into short-term memory (Naya and Suzuki, 2011). The time-context information may also originate in the lateral prefrontal cortex, which contains many cells representing the task structure, including its temporal aspects (Miller and Cohen, 2001; Tanji and Hoshi, 2008; Saga et al., 2011). The requirement of complex visual feature inputs, relatively slow changes after the first context switch and dependence on time context suggest that the time-context-dependent representation of stimulus-outcome contingency is an emergent function in PRh. However, the results in the present experiments do not exclude the possibility that the representations first emerged in another brain site (e.g., the amygdala and orbitofrontal cortex) and are sent to PRh.

Associative properties of TE cells contrasted sharply with those of PRh cells in several aspects. First, responses of TE cells continued to be determined primarily by visual properties of the cues even after training (Fig. 4C). Second, the associated-outcome-dependent components of cue responses in TE cells indiscriminately appeared in both the contingent and noncontingent parts of the trial (Figs. 10 and 11). Therefore, once the system has learned the stimulus-outcome contingency over a long period, the differential responses of TE cells appeared whenever the stimulus occurred. These results, together, suggest that the differential activities in TE cells represent a categorization embedded into the stimulus selectivity of the cells. The translation from visual features to meanings (reward or no reward in this case) developed much more in PRh.

Neuronal activities correlated with animals' expectation of particular outcomes have been observed in various parts of the brain. Their functions may be diverse. Dopamine cells in the ventral tegmental area and substantia nigra, as well as cells in the anterior cingulate cortex, represent errors in reward prediction (Schultz and Dickinson, 2000; Matsumoto et al., 2007; Quilodran et al., 2008; Kuwabara et al., 2014), which may reinforce the learning of value-based action or rule selection (Schultz et al., 1997). Because PRh cells responded to reward delivery at least as strongly when it was expected in the contingent part of a trial as when it was unexpected in the noncontingent part of a trial, their activities do not simply represent reward prediction errors. The differential activities of PRh cells in the contingent part of a trial started after the cue presentation and continued through the outcome delivery until the perception of a new cue in the next trial part (Fig. 12B). These continuing activities may work, not for the simple reinforcement learning of behavioral responses, but as the bases to form an inclusive association incorporating events consistently occurring in the vicinity of the outcome with events already associated with it. Such continuing activities could be advantageous for forming inclusive associations.

There has been plenty of evidence that PRh cells represent associations between visual stimuli (Miyashita and Chang, 1988; Sakai and Miyashita, 1991; Higuchi and Miyashita, 1996; Erickson and Desimone, 1999) as well as associations between visual stimuli and reward-related information (Liu and Richmond, 2000; Mogami and Tanaka, 2006; Ohyama et al., 2012). Information about the context in which those associations will likely occur is as important as that of the associations themselves for guiding animals' behavior. We here propose that various types of information related to objects, including the time context in which stimulus-outcome contingencies occur, are represented by PRh cell activities. This integration of time context with stimulus-outcome contingencies occurred slowly along many repetitions of the experience over more than a week (Figs. 8B,C and 9B,C,E,F). The previous findings of associative representations of object features in PRh were also based on observations after the animals had experienced the contingency many times. Such slow learning that requires many repetitions may be a common characteristic of association representations in PRh cells, which are advantageous to extract regularities existing over multiple events. However, we found that, when a previously learned time context of stimulus-outcome contingency was reexperienced, the representation of the time context in PRh cells quickly recovered (Figs. 8D and 9D,G). Thus, once an association in a context is consolidated, and after its apparent extinction under a different context, it may quickly appear again when the context returns to the original. This flexible switching between consolidated association memories is advantageous for animals experiencing alternations between multiple but repeating environmental constraints.

In conclusion, we found that associations represented by PRh cells are highly inclusive in nature, encoding regularities in past experiences to form representations that are stable but adaptable to repeating regularities. In contrast, association memories represented by TE cells are less inclusive and probably related primarily to their selectivity for visual features. The stable, inclusive associations represented by PRh cells may form a core part of animal's semantic knowledge about the environment.