Monkey Orbitofrontal Cortex Encodes Response Choices Near Feedback Time

Abstract

The primate prefrontal cortex contributes to stimulus-guided behavior, but the functional specializations among its areas remain uncertain. To better understand such specializations, we contrasted neuronal activity in the dorsolateral prefrontal cortex (PFdl) and the orbital prefrontal cortex (PFo). The task required rhesus monkeys to use a visual cue to choose a saccade target. Some cues instructed the monkeys to repeat their most recent response; others instructed them to change it. Responses were followed by feedback: fluid reward if correct, visual feedback if incorrect. Previous studies, using different tasks, have reported that PFo neurons did not encode responses. We found PFo did encode responses in this task, but only near feedback time, after the response had been completed. PFdl differed from PFo in several respects. As reported previously, some PFdl neurons encoded responses from the previous trial and others encoded planned responses. PFo neurons did not have these properties. After feedback, PFdl encoded rewarded responses better than unrewarded ones and thus combined response and outcome information. PFo, in contrast, encoded the responses chosen, rewarded or not. These findings suggest that PFdl and PFo contribute differently to response knowledge, with PFo using an outcome-independent signal to monitor current responses at feedback time.

Introduction

The prefrontal cortex (PF) is important for behavioral flexibility (Platt, 2002; Matsumoto and Tanaka, 2004; Lee and Seo, 2007), but knowledge about its mechanisms remains inadequate. Understanding those mechanisms will require better knowledge about functional specializations within PF. One prevalent idea is that the orbital prefrontal cortex (PFo) is specialized for processing reward and affective information, which reflects the outcome of responses among other factors. This outcome information is thought to be used by the dorsolateral prefrontal cortex (PFdl) for the selection and monitoring of current and future responses (Wallis, 2007). In accord with this idea, PFdl neurons have been reported to encode previous and future responses, as well as their outcome (Wallis and Miller, 2003; Barraclough et al., 2004; Tsujimoto and Sawaguchi, 2004; Genovesio et al., 2006; Seo et al., 2007), and reports on PFo activity have focused mostly on expected or received reward outcomes (Thorpe et al., 1983; Tremblay and Schultz, 1999; Hikosaka and Watanabe, 2000; Wallis and Miller, 2003; Ichihara-Takeda and Funahashi, 2006; Padoa-Schioppa and Assad, 2006, 2008; Simmons and Richmond, 2008). Some neurophysiological studies have suggested that PFo lacks a signal reflecting response choices (Tremblay and Schultz, 1999; Padoa-Schioppa and Assad, 2006). Wallis and Miller (2003), for example, studied the neuronal activity in both PFdl and PFo while monkeys chose between pictures associated with different amounts of reward. They reported that PFo neurons reflected the expected reward outcome, in terms of reward quantity, but did not encode the monkeys' response independent of outcome.

The tasks used in previous studies of PFo, however, did not require monkeys to monitor their responses to perform subsequent trials, which is a common feature of the tasks used to study PFdl (Barraclough et al., 2004; Genovesio et al., 2005, 2006). Accordingly, we contrasted activity in PFdl and PFo using a task with such a requirement.

Materials and Methods

We used two male rhesus monkeys (Macaca mulatta), 10.0 kg and 10.7 kg, respectively. All procedures accorded with the Guide for the Care and Use of Laboratory Animals, and all aspects of the research were approved by the National Institute of Mental Health Animal Care and Use Committee.

Before recordings began, the monkeys were operantly conditioned to perform a saccade task (Fig. 1A). Each monkey was seated in a primate chair with its head facing a video screen 32 cm away. A trial began when a white fixation spot (0.6°) appeared at the center of a video screen, along with two saccade targets (2.0° unfilled white squares, 11.6° left and right from center). Initial fixation was constrained within a ±3° square window centered on the fixation point, but in practice, both monkeys maintained fixation more accurately than required. After the monkey maintained fixation for 1.5 s, a visual cue appeared at the fixation spot for 0.5 s, followed by a delay period of 1.0, 1.25, or 1.5 s (selected randomly). Then the fixation spot disappeared as the “go” or “trigger” signal for a saccade. Once the monkey made a saccade to one of the squares (±3.75°), both targets became white. After 0.5 s of target fixation, feedback was delivered: a drop of fluid after correct responses or the presentation of red squares over both targets after errors. After errors, the cue from that trial was repeated on correction trials, which were presented until the monkey obtained a reward by responding in accord with the most recent correct choice.

Figure 1.

Behavioral task, recording locations, and representative eye-position records. A, Sequence of task events. Each gray rectangle represents the video screen. A central white circle (the fixation point) and two white squares appeared on each trial. The dashed lines show the monkey's gaze angle and fixation target. Next, the cue appeared, followed by a delay period. Offset of the fixation point triggered a saccade (red arrow). Feedback was delivered after the saccade. Fix, Fixation period; ITI, intertrial interval. B, Cues and the response strategies each instructed. The vertical, gray bar and yellow square instructed the monkeys to stay with their previous response; the horizontal bar and purple square instructed a shift to the alternative response. C, Coronal section based on MRI. Angle of penetration (black lines) allowed sampling of neuronal activity in both PFdl (blue) and PFo (pink). LO, Lateral orbital; MO, medial orbital; P, principal sulcus. D, Coronal MRI from monkey 1, ∼35 mm rostral to the interaural plane. Arrow, Electrode location at the center of the recording chamber. Cg, Cingulate sulcus; L, monkey's left; R, monkey's right; otherwise as in C. E, Representative horizontal (x) and vertical (y) eye-position records, collected from monkey 1 while recording the activity illustrated in Figure 2B. In 96% of the trials, eye position was within ±1° of the fixation point when the cue appeared, and in 91% of the trials, it was within ±1° of the center of the target when reward was delivered. Background shading: feedback period.

For each trial, one cue was selected pseudorandomly from a set of 4 stimuli (Fig. 1B) comprising vertical and horizontal bars (light gray, 1.0° × 4.9°) and squares (yellow and purple, 2.0°). The vertical bar and yellow square instructed the monkeys to select the same response on the current trial as they had on the previous trial: a “stay” instruction. The horizontal bar and purple square instructed a “shift” from the previous response to the alternative.

Single-cell activity was recorded from PF using up to 16 platinum-iridium electrodes (0.5–1.5 MΩ at 1 kHz) inserted into the cortex with a multielectrode drive (Thomas Recording). The recording chambers (18 mm inner diameter) were positioned and angled based on magnetic resonance images (MRI) so that both PFdl and PFo were accessible simultaneously (Fig. 1C,D). Single-cell potentials were isolated off-line using a cluster cutting technique (Off Line Sorter, Plexon). An infrared oculometer (Arrington Research) recorded eye position.

The recording sites were reconstructed by standard histological analysis and MRI (Fig. 1C,D). Near the end of data collection, electrolytic lesions (20 μA for 20 s, anodal current) were placed in selected locations at two depths per penetration. After 10 d, the animal was deeply anesthetized and then perfused with 10% (v/v) formol saline. Frozen, coronal sections were Nissl stained. The PFdl recordings came from both dorsal and ventral banks of the principal sulcus in both monkeys. PFo recording sites differed slightly between monkeys. In monkey 1, they were mainly from area 11 and ventral area 12, more lateral and rostral than in monkey 2, which were mainly from area 13. In both cases, the recording sites were in homotypical cortex.

Neuronal analysis involved standard methods, and correction trials were excluded from all analyses reported here. To identify task-related neurons, we used the Kruskal–Wallis test (α = 0.05) to compare mean firing rate among 4 task periods: the fixation period (0.5–1.0 s after fixation onset), the cue period (0.08–0.50 s after cue onset), the delay period (0–1.0 s after cue offset), and the feedback period (from 0.3 s before feedback onset until 0.2 s afterward). If activity during any task period significantly differed from that during another task period, a neuron was classified as task-related. Then, to examine response-dependent activity, we used a two-factor ANOVA (α = 0.05), with factors response (left vs right) and strategy (stay vs shift). (Results for the strategy effect will be reported elsewhere.) For the population averages, we measured the mean firing rate of each neuron in 20 ms bin aligned on reward onset. To confirm these results, we also calculated the normalized population averages, based on the z-score of each bin's firing rate relative to the mean activity from 1.0 s before cue onset to 0.5 s after reward onset. Using a receiver operating characteristic (ROC) analysis, we computed the area under the ROC curve to measure response selectivity, with 0.5 indicating no selectivity and 1.0 corresponding to maximal selectivity. To test whether the ROC values exceeded those expected by chance, a bootstrap analysis was performed (Wallis and Miller, 2003). For each neuron, we shuffled the response designation (left vs right) for each trial and recalculated the ROC values. This process was repeated 1000 times for each neuron, and shuffled ROC values were compared with observed values (Mann–Whitney U test, α = 0.05). The time course of response selectivity was examined by calculating the area under ROC curve in a 200-ms time window that stepped across the trial in increments of 20 ms (Rainer and Miller, 2000).

Results

Behavior

Both monkeys performed the task proficiently, averaging 93% correct responses (supplemental Table 1, left, available at www.jneurosci.org as supplemental material) with reaction times of ∼310 ms (supplemental Table 2, available at www.jneurosci.org as supplemental material). After errors, the monkeys performed correction trials almost perfectly (supplemental Table 1, right, available at www.jneurosci.org as supplemental material). Furthermore, before and during cue presentation, the monkeys maintained stable and accurate fixation, usually within ±1° (Fig. 1E). At the end of each trial, the monkeys typically continued fixation for a few hundred milliseconds after feedback (Fig. 1E), then shifted gaze in an unpredictable manner.

Neurophysiology

We recorded the activity of 873 PF neurons: 524 from PFdl (270 and 254 from monkeys 1 and 2, respectively) and 349 from PFo (253 and 96). Of these 873 neurons, 671 (77%) were task-related (Kruskal–Wallis test): 393 (75%) in PFdl (207 and 186 from monkeys 1 and 2, respectively) and 278 (80%) in PFo (200 and 78). These task-related cells, which had a mean of 83 ± 29 (SD) trials per neuron, were then tested for response selectivity by two-way ANOVA.

Figure 2A illustrates the activity of a neuron with significant response selectivity near the time of reward feedback. Its feedback-period activity (shaded region) was significantly higher when the monkey had chosen the rightward response than when it had chosen the leftward one (3.8 ± 3.8 spikes/s vs 12.0 ± 7.6 spikes/s; mean ± SD; F_(1,114) = 52.6, p ≪ 0.001). Figure 2B shows a different neuron, one with significantly higher feedback-period activity after leftward responses (14.9 ± 8.4 vs 1.2 ± 1.9 spikes/s; F_(1,67) = 91.8, p ≪ 0.001), a property that was maintained on error trials (13.0 ± 10.8 vs 0.0 spikes/s) (Fig. 2C).

Figure 2.

Two response-selective PFo neurons. A, Neuron from monkey 2. Activity aligned on reward (Rw), sorted chronologically from top to bottom, with saccade onset (sac) and the go cue indicated by marks on each raster line. Raster displays show spike times with spike-density averages above each display. Left and right responses shown separately. Correct trials only are shown. Background shading: feedback period. B, Neuron from monkey 1, in format of A. C, From the cell in B, for error trials, in format of A. Note that on error trials, feedback was visual, in contrast to reward feedback on correct trials. Trials are sorted by responses made (not by correct responses).

Figure 3A shows that a substantial number of PFdl neurons were response selective during the cue, delay (supplemental Fig. 1A, available at www.jneurosci.org as supplemental material), and feedback periods. PFdl neurons also encoded the response that had been made on the previous trial during the fixation period (supplemental Fig. 1B, available at www.jneurosci.org as supplemental material). In contrast, in PFo, the cue, delay, and fixation periods did not show a significantly greater percentage of response-dependent cells than expected by chance: only the feedback period did so (χ² = 54.8, p ≪ 0.001). For the feedback-period (n = 144 and 79 for PFdl and PFo, respectively), separate Mann–Whitney U tests were performed for the 300-ms period before reward onset and for the 200-ms period afterward. Supplemental Table 3, available at www.jneurosci.org as supplemental material, shows that both periods showed substantial response selectivity. None of these results differed significantly between monkeys (supplemental Fig. 2, available at www.jneurosci.org as supplemental material).

Figure 3.

A, Of task-related neurons (n = 393 and 278 for PFdl and PFo, respectively), percentage showing significant response-dependent activity. Dashed horizontal line: percentage expected by chance. B, Response-selectivity for all task-related neurons estimated as the difference in ROC values between observed and shuffled data. Error bars indicate SEM. Fb, Feedback period; Fix, the fixation period of the next trial.

ROC analysis, computed for all task-related neurons (Fig. 3B), yielded similar results. Compared with shuffled data, the PFdl population showed significantly higher ROC values in all task periods examined (Mann–Whitney U test, p < 0.01), which indicates robust encoding of the response selected. In contrast, PFo did so only for the feedback period (p < 0.01). The mean (±SD) ROC values for the feedback period were 0.66 ± 0.11 and 0.63 ± 0.10 for PFdl and PFo, respectively. For the cue period, these values were 0.62 ± 0.08 for PFdl versus only 0.56 ± 0.05 for PFo; for the delay period, they were 0.61 ± 0.09 for PFdl versus just 0.56 ± 0.05 for PFo.

To characterize the results in greater detail, we examined the average population activity for neurons with response-selectivity during the feedback period (n = 144 and 79 for PFdl and PFo, respectively). Figure 4, A and B, shows that both PFdl and PFo neurons began to discriminate their preferred and antipreferred responses ∼0.5 s before feedback, with a similar time course (Fig. 4C; supplemental Fig. 3, available at www.jneurosci.org as supplemental material). ROC analysis confirmed these findings, also showing that neurons in both PFdl and PFo encoded responses near feedback time (Fig. 4D,E), with a similar time course (Fig. 4F).

Figure 4.

Response-selective activity in PFdl and PFo. A, Average population activity (shaded area = SEM), for PFdl neurons with significant response-selective activity during the feedback period (n = 144), computed separately for each neuron's preferred (black) and antipreferred (red) response. Dashed and solid vertical lines show the time of target acquisition (acq) and reward (rw), respectively. B, Population activity as in A, for PFo neurons (n = 79). C, Difference in activity between the preferred and antipreferred responses from A (brown) and B (green). D, Sliding ROC plots for PFdl neurons with response-selective feedback-period activity (as in A), with the area under the ROC curve color coded for each cell, ranked according to the mean ROC value during the feedback period. E, ROC values as in D, for PFo. F, Means, with SEM, for the data in D (brown) and E (green).

To examine whether response selectivity was affected by reward outcome, we compared activity on correct versus error trials for the 200-ms period after feedback (Mann–Whitney U test, α = 0.05). In PFdl, 65 cells met the inclusion criterion for testing (≥ 5 errors); in PFo, 42 neurons did so. A small percentage of these cells showed a significant difference between correct and error trials (26% in PFdl, 33% in PFo), with error-trial activity usually greater (12 of 17 cells in PFdl; 13 of 14 in PFo). supplemental Figure 4, available at www.jneurosci.org as supplemental material, shows the results of this analysis at the population level, and Figure 2, B and C, shows an example neuron from PFo. In PFo, the strength of the response-direction signal did not differ for correct versus error trials (supplemental Fig. 4A,B, right, available at www.jneurosci.org as supplemental material), whereas in PFdl, it was weaker for error trials than for correct trials (supplemental Fig. 4A,B, left, available at www.jneurosci.org as supplemental material).

Discussion

In accord with previous studies (Barraclough et al., 2004; Genovesio et al., 2006; Seo et al., 2007), we found that PFdl neurons encoded responses both retrospectively and prospectively. During the fixation period, some PFdl neurons encoded the response that had been made on the previous trial (supplemental Fig. 1B, available at www.jneurosci.org as supplemental material), retrospective information that was required by our task. During the delay period, a largely separate population of PFdl cells encoded future responses (supplemental Fig. 1A, available at www.jneurosci.org as supplemental material), a prospective code for the current choice. We also replicated previous findings indicating the virtual absence of response coding in PFo during the cue and delay periods (Fig. 3) (Tremblay and Schultz, 1999; Wallis and Miller, 2003; Padoa-Schioppa and Assad, 2006). Unlike those previous studies, however, we found that PFo neurons did encode responses, although this neural signal occurred at a highly selective time, shortly in advance of feedback and immediately afterward (Figs. 2–4).

The fact that PFo encoded responses only after they had been made explains the results of Wallis and Miller (2003), whose analysis did not extend beyond the response period or into the feedback period. The discrepancy between the present report and other previous studies (Tremblay and Schultz, 1999; Padoa-Schioppa and Assad, 2006) probably resulted from task differences. In previous tasks, the monkey's response was guided entirely by the location of a visual cue or the reward value associated with a cue. Hence, the monkeys were not required to attend to or remember their responses after they were made. The present task, in contrast, required that the monkeys remembered their most recent response until next trial, which required attention to and monitoring of their responses as well as memory of them. Taken in the context of previous findings, our results suggest that PFo neurons encoded responses near feedback time in the present task because the monkeys had to attend to and monitor those responses. Future studies are needed to determine if these signals occur in other circumstances, as well.

The response-related signal in PFo resembles that in the dorsal anterior cingulate (AC) cortex (Seo and Lee, 2007), which also occurs primarily at feedback time. Although medial and orbital prefrontal areas compose different networks, some components (e.g., areas 13a and 12o) have substantial connections in common (Carmichael and Price, 1996). Accordingly, the response-coding properties that PFo and AC share could reflect the coordination of function between medial and orbital frontal networks. In this context, it will be of particular interest to study how these two networks interact during decision making (Rushworth et al., 2007).

We can rule out an account of these response signals in terms of rewards, per se. Both rightward and leftward responses were followed by equal rewards (both in terms of magnitude and probability), and the monkeys showed little, if any, spatial response biases (supplemental Table 1, available at www.jneurosci.org as supplemental material). An account in terms of visual stimuli is also unlikely. By design, the task ensured that leftward and rightward responses were equally associated with each stimulus.

The present experimental design did not permit us to distinguish response coding in motor terms from more abstract aspects of decisions and choices such as goals. Additional experiments will be needed to differentiate these possibilities. Nevertheless, the absence of response selectivity at other times during a trial and in other tasks (Wallis and Miller, 2003; Padoa-Schioppa and Assad, 2006) suggests that this signal is unlikely to represent something as simple as a motor command, per se. It more likely reflects the monitoring of choices, in this case a choice of the spatial goal for the most recent saccade. As noted above, an important factor could be the need to monitor the choice or goal. The fact that the monkeys performed nearly perfectly on correction trials (supplemental Table 1, right, available at www.jneurosci.org as supplemental material) supports this idea, because it shows that the monkeys attended to and remembered their choices at the time of feedback and used this information to shift from unrewarded responses.

In contrast to the present data, neurophysiological studies in rats have reported response-dependent activity both before and after the behavioral response (Feierstein et al., 2006; Roesch et al., 2006; Furuyashiki et al., 2008). This apparent discrepancy could result from species differences, but it more likely reflects the difficulty in knowing which, among the many frontal areas in primates (Carmichael and Price, 1994; Öngür and Price, 2000), is the homolog of what is commonly called orbital prefrontal or orbitofrontal cortex in rodents. Some authorities have argued that the like-named areas in rodent and primate orbital cortex are homologous (Uylings et al., 2003), but many doubts remain about such contentions. The rodent orbitofrontal cortex is agranular, but most neurophysiological data from monkeys (Tremblay and Schultz, 1999; Wallis and Miller, 2003; Padoa-Schioppa and Assad, 2006), including those presented here, come from homotypical parts of PFo. Rodents lack homotypical frontal areas, and the areas called orbitofrontal cortex in rodents are poor candidates for homologues of any homotypical prefrontal area in primates, including the parts of PFo studied here (Preuss, 1995, 2007; Wise, 2008). Accordingly, it is not surprising to find physiological differences between what is called orbitofrontal cortex in rodents and primates.

Conclusion

To adapt flexibly to a dynamic environment, it is critical to associate responses with their outcome (Platt, 2002; Matsumoto and Tanaka, 2004; Lee and Seo, 2007). Previous work has emphasized the role of AC cortex in this function (Rushworth et al., 2007), but it is clear that other parts of frontal cortex contribute, as well. For example, previous studies have reported neuronal signals for both responses and outcomes in PFdl (Barraclough et al., 2004; Genovesio et al., 2006; Seo et al., 2007). The present study is the first to report response-encoding signals in the primate PFo, which added to previous reports of outcome coding in PFo (Thorpe et al., 1983; Tremblay and Schultz, 1999; Wallis and Miller, 2003; Ichihara-Takeda and Funahashi, 2006; Padoa-Schioppa and Assad, 2006) indicates that PFo, like PFdl, encodes both responses and outcomes at feedback time.

PFdl appears to play a different role than PFo, however, because many individual PFdl neurons encode both responses and outcomes (Barraclough et al., 2004; Tsujimoto and Sawaguchi, 2004, 2005; Seo et al., 2007) (supplemental Fig. 4, available at www.jneurosci.org as supplemental material), whereas response signals in the PFo cells appear to encode the monkey's response independent of outcome. This conclusion follows from the lack of difference in PFo's response selectivity between correct and error trials (supplemental Fig. 4, available at www.jneurosci.org as supplemental material). Put somewhat differently, the feedback-time response signal in PFdl encodes the monkey's response if it produced a reward, but the simultaneous signal in PFo encodes the response independent of reward. Furthermore, PFdl cells also encode responses and outcomes at the time responses are selected (Barraclough et al., 2004; Genovesio et al., 2006; Seo et al., 2007), unlike PFo cells, which encode response only near feedback time (Fig. 3). PFdl is thus likely to use information about previous responses and their outcomes at the response-selection stage, with both signals combined in individual neurons, whereas PFo seems to function more at the response-monitoring stage, with separate signals for responses and outcomes. This idea refines models suggesting that PFo provides information to PFdl for adaptive decision making (Wallis, 2007).