Neural Correlates of Decisions and Their Outcomes in the Ventral Premotor Cortex

Abstract

Selection of the appropriate action in a changing environment involves a chain of events that goes from perception through decision to action and evaluation of the outcomes. What and where in the brain are the correlates of these events? The ventral premotor cortex (PMv) is a candidate because (1) it is involved in sensory transformations for visually guided actions and in perceptual decisions, and (2) it is connected with sensory, motor, and high-level cognitive areas related to performance monitoring. Therefore, we hypothesized that it would be the site for representing sensory perception for action and for evaluating the decision consequences. Trained monkeys were required to discriminate the orientation of two lines showed in sequence and separated by a delay. Monkeys compared the orientation of the second line with the memory trace of the first and communicated whether the second was to the left or to the right of the first. Here we show that the activity of PMv neurons reflected (1) the first stimuli and its memory trace during the delay and comparison periods, (2) its comparison with the second stimuli, including the strength of the evidence, and (3) the result of the discrimination (choice). After the monkeys reported the choice, there were neurons that only encoded the choices, others only the outcomes, and others the choices and outcomes together. The representation of task cues, decision variables, and their outcomes suggest a role of PMv as part of a supervisory network involved in shaping future behavior and in learning.

Introduction

Selection of the adequate action can be conceptualized as a chain of events that involves perceiving environmental cues, maintaining in or recovering some of them from memory, deciding between alternatives, and executing the decision. Another essential component of this chain of events is the detection of the outcomes. Ventral premotor cortex (PMv) receives information from striate and extrastriate (Markowitsch et al., 1987), parietal (Godschalk et al., 1984; Luppino et al., 1999; Borra et al., 2008), and prefrontal (PFC) (Lu et al., 1994; Boussaoud et al., 2005) cortices and thalamic structures (Schell and Strick, 1984; Rouiller et al., 1999) and sends information to motor areas of the frontal cortex (Ghosh and Gattera, 1995; Dancause et al., 2006), subcortical structures (McFarland and Haber, 2002), and spinal cord (Keizer and Kuypers, 1989; Dum and Strick, 1991; He et al., 1993). This area receives projections from PFC, cingulate cortex, supplementary eye fields (SEFs), and basal ganglia (Hoover and Strick, 1993; Ghosh and Gattera, 1995; Dum and Strick, 2002; Boussaoud et al., 2005; Dancause et al., 2006), all related to performance monitoring. PMv neurons modulate their activity during visuospatially instructed tasks (Mushiake et al., 1991) and complex sensorimotor actions (Gentilucci et al., 1988; Rizzolatti and Luppino, 2001) and share functional properties with the anterior intraparietal area (Raos et al., 2006), which is involved in sensorimotor transformations for visually guided actions (Goodale and Milner, 1992; Borra et al., 2008). PMv lesions affect performance of sensorimotor tasks (Fogassi et al., 2001). Furthermore, work with monkeys performing a vibrotactile discrimination task has revealed that PMv is involved in perceptual decisions (Romo et al., 2004). Therefore, PMv is a candidate for representing sensory perception for action and evaluating the decision consequences. We trained monkeys in a visual discrimination task (Vazquez et al., 2000). In this task, subjects perceived the orientation of a line, maintained a trace of it in working memory during the delay period, perceived the orientation of another line, compared the orientation of the second line with the first, and decided whether the second line was to the left or to the right of the first. After the end of the second stimulus, subjects reported the result of the decision with an eye movement toward one of two targets. Here we report that PMv neurons reflect the chain of events that takes place between perception and action. There are neurons that encoded the orientation of the first stimulus during the stimulus presentation and a trace of it during the delay period between the first and the second stimuli. The neural activity during the comparison period was related to the remembered and current stimuli and also to the monkey's choice and the difficulty of the discrimination. Once the monkeys communicated the result of the decision, the majority of these neurons differentiated correct responses from incorrect ones; interestingly, many of them reflected the decision again. These results suggest that PMv cortex is involved in perceptual decisions and in the evaluation of its consequences.

Parts of this work have been reported previously in abstract form (Pardo-Vazquez et al., 2007, 2008).

Materials and Methods

General.

Experiments were performed on two male monkeys (Macaca mulatta). Animals (BM5, 8 kg; and BM6, 6 kg) were handled according to the standards of the European Union (86/609/EU), Spain (RD 1201/2005), and the Society for Neuroscience Policies and Use of Animals and Humans in Neuroscience Research. The experimental procedures were approved by the Bioethics Commission of the University of Santiago de Compostela (Spain). The basic behavioral tasks were described previously (Vazquez et al., 2000). The monkeys' head was fixed during the task and looked binocularly at a monitor screen placed at 114 cm away from their eyes (1 cm subtended 0.5° to the eye). The room was isolated and soundproof. Two circles (1° in diameter) were horizontally displayed 6° at the right and 6° at the left of the fixation bar (a vertical bar; 0.5° length, 0.02° wide) displayed in the screen center. The visual stimuli-oriented bars were displayed in the center of the screen. In the discrimination task, monkeys used right and left circles to signal, with an eye movement, the orientation of visual stimuli to the right and to the left, respectively. To calibrate the setup, monkeys used the fixation bar in the eye fixation task to signal the tilt of the fixation bar. Eye movements were recorded with the magnetic eye search coil technique (C-N-C Engineering) (Robinson, 1963), sampled at 740 Hz, and acquired with DataWave Experimenter (DataWave Technologies).

Visual stimuli were created in a 90 MHz Pentium personal computer using a 40 MB Matrox MGA Millennium II PCI graphic card driven by MGL Libraries from SciTech and presented in a NOKIA multigraph 445X monitor, with 75 Hz vertical refresh rate, and 1280 × 1024 pixel resolution. The monitor mode when the task was running was 800 × 600 (75 Hz). CORTEX (http://www.cortex.salk.edu/) was used for task control and to generate visual stimuli. Those stimuli were black bitmap images (red–green–blue, [0 0 0]; size, 640 × 480 pixels, 300 dots per inch), with white bars (red–green–blue, [255 255 255]; size, 3 × 280 pixels) at the center of the images.

The bars were stationary bright lines, subtending 8° length and 0.15° wide. Three different reference orientations were used for each monkey: 87°, 90°, and 93° (BM5) and 84°, 90°, and 96° (BM6); all angles referred to the horizontal axis. Different test lines, eight per reference stimulus, were presented: four clockwise and four counterclockwise to the reference line in steps of 1° (BM5) or 2° (BM6). To avoid the use of end points of the line as a cue, the lines were randomly displaced in both horizontal directions. The amount of displacement was less than or equal to twice the value of the maximum separation of the two lines (reference and test) for the minimum change in rotation. For example, the end point of +1° rotation of the test line will be mistaken for the end point of −1° rotation of the test line. This forces subjects to rely only on the orientation of the line to complete the task.

Surgery.

A head holder was implanted on the monkeys' skulls to fix the head during the experiments. A three-turn monocular scleral eye coil was implanted to measure gaze direction (Judge et al., 1980). The recording chamber was fixed to the skull, above the PMv following stereotaxic coordinates (the center was at anterior +23 mm, lateral +26 mm). The chamber, 20 mm in diameter, was used as a pedestal for a hydraulic microdrive (MO-951; Narishige). All surgery procedures were performed under general anesthesia and aseptic conditions. The animals were first anesthetized with ketamine (10 mg/kg, i.m.). This was followed by pentobarbital sodium (15 mg/kg, i.v.) for induction and maintained with a continuous intravenous infusion of pentobarbital sodium (3.5 mg · kg⁻¹ · h⁻¹) and fentanyl (5.25 μg · kg⁻¹ · h⁻¹) in saline at 10 ml/h. Animals were intubated and artificially ventilated with a mixture of oxygen and air. Expiratory CO₂, electrocardiogram trace, and temperature were monitored continuously during surgery. Antibiotics and analgesics were administered after surgery.

Discrimination task.

Each monkey was trained for ∼11 months to perform in three tasks: the continuous discrimination (CD), the control (CT), and the fixed discrimination with implicit reference (Nacher et al., 2006). Here we will report on the results obtained in the CD and CT tasks only.

CD task.

The monkeys were trained to discriminate up to their psychophysical thresholds in the two visual discrimination tasks sketched in Figure 1A. The stimuli were presented in the center of the monitor screen, and eye movements larger than 2.5° aborted the task. The orientation discrimination task was a modified two-alternative forced choice. A masking white noise signaled the beginning of the trial and the fixation target appeared in the center of the screen (FT) (Fig. 1A). The monkey was required to fixate the FT. If fixation was maintained for >100 ms, the FT disappeared, and, after a variable prestimulus delay (100–300 ms), two stimuli (reference and test), each of 500 ms duration, were presented in sequence, with a fixed interstimulus interval (1 s). At the end of the second stimulus, the subject made a saccadic eye movement, in a 1200 ms time window, to one of the two circles, indicating whether the orientation of the second stimulus was clockwise or counterclockwise to the first. Trials lasted ∼3.5 s separated by a variable intertrial interval (1.5–3 s). Fifty milliseconds after the correct response, a drop of liquid was delivered as a reward. A modulation of the masking noise signaled the errors; the modulation started 50 ms after the incorrect response and lasted for 75 ms. Monkeys' weights were measured daily to control hydration, and once a week the animals had access to water ad libitum. The level of training was assessed by the psychometric functions. Once trained, the monkeys performed ∼1000 trials per day.

Figure 1.

Sequence of events during the discrimination and control tasks. A, Discrimination task; the FT and the two circles appeared simultaneously in the center and at both sides of the monitor screen, respectively. The monkey initiated the trial by fixating the FT. Fixation had to be maintained during the trial; otherwise, it was aborted. When the FT disappeared and after a variable prestimulus delay (PSD) (100–300 ms), two stimuli, each of 500 ms duration, appeared in sequence separated by a delay of 1 s. Once the second stimulus had disappeared, the monkey made an eye movement to one of the two circles to indicate whether the orientation of the second stimulus was to the left or to the right of the first. Correct discriminations were rewarded. B, Control task; the FT and one of the circles appeared simultaneously. The monkey had to maintain fixation until the FT had disappeared and made an eye movement to the circle. The duration of the trial was the same as that of the CD task. C, Distribution of the orientation of the stimuli. There were some stimuli that overlapped: some test stimuli had the same absolute orientation (e.g., 88°) but different relative orientation to the reference stimuli (e.g., 90° and 87°), and those test stimuli gave responses to the right and to the left, respectively.

Subjects received three reference stimuli presented randomly and followed by the corresponding test stimuli (Fig. 1C). For BM5, a reference stimulus of 87° was followed by a test stimulus ranging from 83° to 91° in 1° steps. A reference stimulus of 90° was followed by a test stimulus ranging from 86° to 94° in steps of 1°. A reference stimulus of 93° was followed by a test stimulus ranging from 89° to 97° in 1° steps. For BM6, a reference stimulus of 84° was followed by a test stimulus ranging from 76° to 92° in 2° steps. A reference stimulus of 90° was followed by a test stimulus ranging from 82° to 98° in 2° steps. A reference stimulus of 96° was followed by a test stimulus ranging from 88° to 104° in 2° steps. In this task, the first and second stimuli randomly varied continuously from trial to trial. During the CD task (Fig. 1A), animals paid attention to the orientation of the first line (reference), stored some trace of it during the delay between the two stimuli (1 s), paid attention to the orientation of the second stimulus, and compared the stored trace with the orientation of the second stimulus (test). Monkeys had to decide whether the orientation of the second stimulus was to the right or to the left of the reference and to communicate their decision by moving their eyes to one of the two circles. Subjects can only solve the CD task by paying attention to both stimuli in each trial. In fact, when in the CD task the first stimulus was not shown, humans and monkeys were not able to solve the task (Hernandez et al., 1997; Vazquez et al., 2000); this clearly indicates that maintenance of the first stimulus in working memory is crucial to solve the task.

Control task.

The control task was designed to elucidate whether the neural activity was related with motor preparation or response. In the control task, the time duration of the trial was the same as that in the CD task and the monkeys knew in advance where they had to move their eyes. Therefore, no discrimination or categorization was necessary, only the decision to move the eyes and the movement itself. The FT and only one circle (to the left or to the right, depending on the trial) appeared on the screen, and, when the FT disappeared, the monkey was required to make an eye movement toward the circle to be rewarded (Fig. 1B).

Recording sessions and sites.

Extracellular single-unit activity was recorded with tungsten microelectrodes (epoxylite insulation, 1.5–3.5 MΩ, catalog #UEWMGCLMDNNF; FHC) in the posterior bank of the ventral arm of the sulcus arcuatus and adjacent surface in the ventral premotor cortex in the four hemispheres of the two monkeys (Fig. 2B). Microelectrodes were advanced through the intact dura mater to the cortex. Signals were amplified, filtered, and viewed on an oscilloscope screen. Raw extracellular unit activity together with records of the eye position were displayed and stored in a computer for offline analysis (DataWave Experimenter; DataWave Technologies).

Figure 2.

Behavioral results and localization of the recording area. A, Psychometric functions for monkey BM5. Inset shows the minimum perceptible differences (MPD). B, Sketch of the brain showing the PMv region (black circle), in which the microelectrode penetrations were made. Sizes of dots according to the number of microelectrode penetrations in each place (1–4; n = 173) for the two monkeys. ASv, Arcuate sulcus ventralis; PS, sulcus principalis; sca, sulcus subcentralis anterior.

To stabilize the recordings, the chamber was filled with a warm (39°) mixture of agarose in saline, which rapidly solidified. At the end of the recording session, the plug was removed and the chamber was cleaned. To avoid infections and prevent hardening of the dura, the chamber was filled with a warm solution of agarose (80 mg/4 ml saline), polymixin B sulfate (2.4 mg/0.5 ml saline), neomycin (1 ml), and dexamethasone (0.5 ml), which solidified after a few minutes (Shtoyerman et al., 2000). The dura remained in perfect condition for >8 months. In each daily recording session, the monkeys began by working on the eye fixation task (∼50 trials) to calibrate the eye coil system. The extracellular single-unit activity was recorded while the monkeys worked on the tasks, and the tasks order was randomly changed.

The animals were killed with an intravenous overdose of pentobarbital sodium. Four pins were inserted in the brain at the borders of the craniotomies parallel to the microelectrode penetrations. The head was perfused through the heart with saline, followed by 10% formaldehyde. Each microelectrode penetration was marked on the surface map of the brain area delimited by the four pins. The brain was removed and kept in 10% formaldehyde. No electrolytic lesions were made during recording sessions; therefore, we did not have detailed references of the trajectory of the microelectrode penetrations. However, the marks left by the inserted pins, the surface map of the penetrations, together with the readings of the microdriver counter taken during the recording sessions allowed us to reconstruct the area where the recordings took place (Fig. 2B).

Data analysis.

All analyses were performed using custom-made programs in Matlab R2006b (MathWorks). The following task periods were considered for data analysis: prestimulus delay (from the capture of the FT until the presentation of the reference, 100–300 ms variable duration), reference (500 ms duration), delay (from the end of the reference until the presentation of the test, 1000 ms duration), test (500 ms duration), response time (from the end of the test stimuli until the monkey captured the target), and time after reward (from the time the monkey received the reward until 1500 ms afterward). We considered the response of a neuron as task related if, during any of the task periods, the mean firing rate was significantly different from the control period preceding the white noise at the beginning of each trial (Wilcoxon's test, p < 0.01). The control period firing rate was estimated from single trials using the average response during that period. The behavioral response was taken when the monkeys' gaze captured one of the circles. All analysis was conducted on correct trials except to compare correct versus incorrect decisions. To calculate the average firing rate of the population, the activity of each neuron was normalized according to its minimum and maximum values.

The neural activity during the task periods can depend on the stimuli (reference and test periods), their traces (delay and test periods), the interaction among them (test period), or, once the response was emitted, on the correct versus incorrect trials. We compared the neural activity across all task periods using receiving operating characteristics (ROC) (Green and Swets, 1966) and stepwise linear regression (SLR) analyses (Draper and Smith, 1966).

The ROC analysis allows the measure of the degree of overlap between two response distributions, and it is independent of the firing rate of the neuron (Britten et al., 1992; Hernandez et al., 2002; Wallis and Miller, 2003). Each point of an ROC curve represents the proportion of trials of one condition (e.g., the test is to the left of the reference) in which the firing rate was higher than a criterion value against the proportion of trials of the other condition (e.g., the test is to the right of the reference) that exceeded the same criterion; the criterion varied between the minimum and maximum values of both distributions. For each neuron with sufficient data (at least five trials for each condition), we computed the area under the ROC curve within a time bin of 100 ms that was slid in 20 ms steps until the entire periods of the task had been analyzed: ROC index. The analysis began 500 ms before the reference and ended 1500 ms after the response. To establish when the ROC index value significantly deviates from 0.5, we performed a sliding window permutation test. For each neuron, the trials were randomly assigned to different task conditions, and ROC values were calculated for each permutation iteration (n = 2000). Minimum and maximum ROC intervals were computed as the values of the ROC histogram for which the probability of finding random smaller and greater values was less than p < 0.01. This stage produces temporal ROC margins that change as the firing rate does. These margins obtained in this way were approximately in the range of 0.4 and 0.6 (Wallis and Miller, 2003). The criterion for ROC selectivity was defined as the point at which the ROC index exceeded the limits obtained with the permutation test for at least two consecutive bins.

The firing rate dependence on the reference and test is of interest because the first stimulus can be represented throughout the reference, delay, test, and response time periods, and the comparison between the two stimuli can take place sometime during the test period. This dependence for the choice was obtained through ROC analysis. SLR analysis was used to establish the dependence between (1) the firing rate and the orientation of the references, (2) the firing rate and the relative orientation of the test stimuli, and (3) the firing rate and the choice (Draper and Smith, 1966; Hernandez et al., 2002; Romo et al., 2004). Both ROC and SLR analysis for the choice gave similar results; however, SLR analysis allowed studying the conjoint contribution of multiple factors.

The neural activity during the task was fitted to an arbitrary linear function of three factors: the reference (R), the relative test orientation (T), and the choice (C). Therefore, the firing rate can be formulated as firing rate(t) = a₀(t) + a_r(t) × R + a_t(t) × T + a_c(t) × C, where t represents time, and the coefficients a_r, a_t, and a_c quantify the firing rate dependence on the R, T, and C, respectively. Linear fits for the task periods were obtained through SLR analysis; this function models the response variable (firing rate) as a function of the predictor factors (stepwise fit in Matlab R2006b), which in our case were the firing rate as a function of R, T, and C. The result is a vector of estimated coefficient values for R, T, and C. The coefficient a₀ depends on the firing rate and is a parameter independent of a_r, a_t, and a_c. This fit was calculated using a sliding window of 100 ms moving in 10 ms steps throughout the neuron firing rate. The resulting a_r, a_t, and a_c coefficients were plotted as a time function. Coefficients were included in the model if p value for a predictor was lower than 0.02 for at least two consecutive bins. The deviation from the mean (2 SDs) was used to determine the significant time periods for each factor.

Psychometric functions of monkeys' discriminations between different orientations were obtained. We plotted the percentage of test stimuli identified as oriented to the left of the reference stimuli against the orientation of the test stimulus. Data were fitted to a logistic Boltzmann equation:

where p_b(a) is the percentage of trials called “to the left of”, A₁ and A₂ are the maximum and minimum values of p_b(a), respectively, a₀ is the orientation for which p_b(a) = (A₁ − A₂/2), and a₁ represents the width of the function. All regressions fitted the data significantly, with a χ² of p < 0.05. Upper and lower differential limens, or thresholds, were calculated as the values of the minimum angle that elicits 75 and 25% of correct responses, respectively. The minimum perceptible differences were calculated as the mean of the upper and lower limens.

Results

General

Six hundred eighty (98%; n = 691) neurons were found to be task related (see Materials and Methods). During the microelectrode penetrations, we found neurons whose activity increased during spontaneous arm movements (scratching one's own body or reaching with one's hand) or sucking the reward tube (n = 184). These neurons were found in the same microelectrode penetrations and close to the other neurons described here. However, while the animals worked on the tasks, these arm/mouth neurons did not modulate their activity in the same way as in the population referred to below, and they were not included in this description. During the comparison period, the neural activity differentiates choices to the right and to the left.

In the CD task, the discrimination process occurred sometime during the presentation of the test stimuli. This is the comparison period, when subjects compared the orientation of the test stimulus with the memory trace left by the reference stimulus and decided whether the second stimulus was to the left or to the right of the first. If the PMv neurons were involved in discrimination processes, the firing rate had to be modulated during the comparison period. The response of the neurons shown in Figure 3, A, B, E, and F, was analyzed as a function of subjects' two possible choices: the test stimulus was to the left or to the right of the first. During the comparison period, 359 (of 680; 53%) PMv neurons showed significant ROC indices (Fig. 3D,H), reflecting that the neuronal responses were significantly related to the monkeys' choice about the orientation of the second stimulus. For the example neurons of Figure 3, the latency of the significant ROC indices from the beginning of the second stimuli was 60 ms (Fig. 3C,G), and, for the population (n = 359), the mean ± SE was 175 ± 6 ms (Fig. 3I). For the two example neurons, the peak values of the ROC indices occurred 260 ms after the presentation of the test stimuli (Fig. 3C,G), and, for the population (n = 359), they occurred 320 ± 9 ms (mean ± SE) after the presentation (Fig. 3I). For 219 (of 359; 61%) neurons, the choice-related activity was restricted to the comparison period, and, for 140 (of 359; 39%), this effect was also extended until the monkeys gave the response.

Figure 3.

During the comparison period of the discrimination task, the PMv neurons represent the decisions taken by the monkeys. A, E, Rasters sorted by correct decisions: the second stimulus was to the right or to the left of the first one. Each row represents a trial and each tick an action potential; each dot indicates the monkey's response (R). B, F, Average firing rates as a function of time sorted by correct decisions to the left (red lines) and to the right (blue lines). C, G, ROC indices as a function of time for decisions taken to the left versus to the right; thicker traces indicate significant ROC indices. D, ROC indices of each neuron that correctly differentiated decisions to the left from the right during the comparison period are represented in each row (n = 359). Color code from blue to red indicates ROC indices represented between 0.5 and 1. H, Distribution of maximum ROC indices across the 359 decision neurons. I, Distribution of latencies, from test onset, for both the first and maximum significant ROC indices across the 359 decision neurons. Shaded boxes signal the reference (500 ms) and test (500 ms) task periods.

Dynamics of the discrimination process

This task requires a comparison between the orientation of the second stimulus and the trace left by the first one; therefore, the first stimulus has to be available during the test period. When the discharge of the neurons was analyzed as a function of the reference stimuli, 279 neurons (of 680; 41%) significantly modulated their discharge depending on the reference stimuli during the reference, delay, or comparison periods (SLR analysis; see Materials and Methods). There were neurons that varied their firing rate as a function of the reference stimuli during the reference (79 of 279; 28%), the delay (179 of 279; 64%), and the comparison (142 of 279; 51%) periods. The responses during the presentation of the reference, delay, and test periods may carry information about the first stimuli or about future actions (Romo et al., 2004; Gold and Shadlen, 2007). However, in our stimulus set, the reference stimuli can be associated with responses in both directions (left/right), and therefore the monkeys may not anticipate the correct responses before the test presentation. This suggests that the responses during the reference, delay, and comparison periods of these neurons encoded information about the reference stimuli.

We investigated further the dependence between the tests and the traces of the references. During the delay period, the neurons carried information about the orientation of the reference stimuli that was extended into the comparison period (Fig. 4). The average firing rate was fitted (1) as a function of the traces of the references during the delay period, (2) separately as a function of the traces of the references and the relative orientation of the test stimuli during the beginning of the comparison period, and (3) after ∼100 ms of the presentation of the test stimuli, as a function of the relative orientation of the test stimuli; in this moment, the firing rates are grouped in such a way that they reflected the two choices (Fig. 4D,I, 1, 2, 3, respectively). The dynamics of the response during the comparison period occurred very rapidly; during approximately the first 50 ms, the neurons carried information about the references only, during the following 50 ms, they carried information about the references and the relative orientation of the test stimuli, and from that time onward, they carried information about the monkeys' choice. Therefore, the comparison between the trace of the first stimulus and the second occurred ∼50 ms after the presentation of the second stimulus, and, after this, the neurons immediately signaled the choice. The SLR coefficients also clearly showed the dynamics of the decision process for the two example neurons and for the population (Figs. 4E,J, 5). During the comparison period, 255 and 353 neurons showed significant a_t and a_c coefficients, respectively (Fig. 5). The average latencies from the beginning of the test period until the maximum significant a_t and a_c coefficients were (mean ± SE) 196 ± 9 and 246 ± 8 ms, respectively. For the 205 neurons with significant a_t and a_c coefficients during the comparison period, this time lag of 50 ms was significant (t test for dependent samples, p < 0.0001). It is worth noting that 172 neurons (49%) of the 353 choice-related neurons encoded the reference orientations. To explore the congruency between the preferred orientation of the reference and the preferred direction of the choice, we compared the sign of the regression coefficients a_r and a_c. One hundred eighteen neurons of the 172 (69%) that encoded the references and the choices displayed congruent responses, e.g., large response to the leftmost reference was followed by large response “to the left” choices. This congruence suggests that PMv cortex represents perception for action.

Figure 4.

Temporal evolution of neural activity depends on the trace left by the first stimuli, on the comparison between the first and the second stimuli, and on the choice. A, F, Rasters of two single neurons sorted by the reference stimuli and the choice. B, G, C, H, Average firing rates as a function of time across trials sorted by the reference stimuli and by the monkey's choices, respectively. D, I, Temporal evolution of the average firing rates fitted as a function of the reference stimuli during the delay period (1), the references and the relative orientation of the test (2), and the choice (3) during the comparison period. E, J, Stepwise linear regression coefficients, a_r (light gray line), a_t (dark gray line), and a_c (black line), as a function of time; thicker traces indicate significant coefficients that are associated with the orientation of the references, with the relative orientation of the test and with the choice, respectively.

Figure 5.

Dynamics of PMv population response during the discrimination task. Number of neurons with significant SLR coefficients as a function of time. Light gray, dark gray, and black traces correspond to a_r, a_t, and a_c coefficients, respectively.

The neural activity during the comparison period cannot be explained by the motor component of a voluntary action.

Does the neural activity reflect the decision formation or the planned eye movement? In this task, the decision can be taken sometime during the 500 ms test period, making it necessary to distinguish neural activity associated with decision formation from that associated with planned eye movement (Kim and Shadlen, 1999; Roitman and Shadlen, 2002; Glimcher, 2003; Cisek and Kalaska, 2004). Given that there is a relationship between the monkeys' choice and the motor response, it is possible that the differential neural activity observed during the test period represented only the motor component of the task (planning and execution of the saccades). Some aspects of the neural activity suggest that this is not the case: (1) the maximum firing rate for most neurons appeared during the comparison period at least 100 ms before the eye movement, (2) the maximum difference in firing rate between decisions “to the left” and “to the right” was observed for most of the neurons during the first part of the comparison period, this being at least 300 ms before the eye movement, and (3) 219 neurons showed decision-related activity only during the comparison period and not when the monkey made the saccades. Nevertheless, we conducted a series of analyses to rule out whether the motor component of the task alone can explain this activity.

Neural response depends on the difficulty of discrimination

One way to get round the problem of whether the activity is associated with decision formation or with the planned eye movement is to compare trials with the same motor component but different sensory evidence. One case is the modulation of brain neural activity by the difficulty of the discrimination, which is explained by the influence of the perceptual component of the task (Palmer et al., 2005; Gold and Shadlen, 2007).

The easiness of the discriminations depends on the difference in the orientation between the first and the second stimuli. To compare the behavioral performance and neural data for easy and difficult discriminations, we made two groups: the easiest (the three sets of tests and references most apart) and the most difficult (the three closest) (Fig. 1C). The behavioral data from the two monkeys showed that, for the easiest discriminations, the percentage of correct choices was significantly higher than that for the most difficult ones (p < 0.000001, t test for dependent samples): for decisions to the left, the mean ± SE percentages of correct choices were 99.85 ± 0.04 and 80.93 ± 0.67% for easy and difficult trials, respectively; for decisions to the right, the mean ± SE percentages of correct choices were 99.42 ± 0.18 and 79.31 ± 0.70% for easy and difficult trials, respectively. The response time, measured from the end of the test stimuli (the go signal for the subjects) to the response, is significantly faster for the easiest than for the most difficult discriminations: for decisions to the left, the mean ± SE response times were 96 ± 0.74 and 115 ± 1.14 ms; for decisions to the right, the mean ± SE response times were 93 ± 0.82 and 129 ± 1.1 ms. For a single example neuron and for the population (n = 357) (Fig. 6), the comparison of the firing rates for the easiest versus the most difficult discrimination showed that, during the first 250 ms of the decision period and for the preferred direction (preferred direction, the one that provoked the highest firing rate), the firing rate was higher for the easiest than for the most difficult discrimination. For the nonpreferred direction, the firing rate was higher for the most difficult than for the easiest discriminations. Therefore, during this time period, the neural response was graded according to the difficulty and the direction of the decision; the neural activity can reflect both the decision and the strength of the evidence that the monkeys used to make a decision. During the next 250 ms of the comparison period, the firing rate is now graded according to the difficulty of the decision only: it is greater for the easiest discriminations for preferred as well as for nonpreferred directions. The neural activity during this time period might reflect the likelihood of the behavioral action chosen or of the expected outcomes. To further explore the effect of the difficulty or the strength of the evidence in decision-related activity, we computed the temporal ROC indices comparing (1) the easiest trials to the left versus the easiest trials to the right and (2) the most difficult trials to the left versus the most difficult trials to the right. Then we compared ROC indices for the 183 neurons for which we had enough trials. We found significant differences (p < 0.001, t test for dependent samples) both in the magnitude and timing of the ROC indices. The maximum ROC indices reached during the comparison period were greater for the easiest than for the most difficult decisions (mean ± SE): 0.9 ± 0.01 and 0.88 ± 0.01, respectively. Also, the time latencies, taken from the comparison period onset, for both the first and maximum significant ROC indices were faster for the easiest trials (145 ± 7.3 and 253 ± 8.32 ms, respectively; mean ± SE) than for the most difficult ones (182 ± 7.83 and 281 ± 8.01 ms, respectively; mean ± SE). These results further suggest that the neural response represents the strength of the evidence needed to guide the decision.

Figure 6.

During the comparison period, PMv neural activity reflects the difficulty of the discrimination and the decision. A, Rasters of a single neuron sorted by easy and difficult correct decisions to the left and to the right during the test period. B, Average firing rates across time of the rasters displayed in A; thick and thin black and gray traces represent easy and difficult correct decisions for the preferred and nonpreferred directions, respectively. C, Normalized average firing rates as a function of time across 357 neurons sorted by easy and difficult correct decisions for preferred and nonpreferred directions (same color conventions as in B).

Responses during the control task

In this task, no discrimination was involved because the reference and test stimuli were not displayed and the monkeys knew in advance toward which target they had to make an eye movement (see Materials and Methods) (Fig. 1B).

Of 359 choice-related neurons, we studied the 179 neurons in which the CD and control tasks were completed. The results have shown that the firing rate of 98 neurons (of 179; 55%), during the comparison period, that correlated with correct choices to the left and to the right did not respond in the control task (Fig. 7A–D, I for the population). The firing rate of 81 other neurons (of 179; 45%) were modulated according to choices to the left and to the right, and, in the control task, they were modulated well in advance of the behavioral response (Fig. 7E–H, J for the population); the latency of the first significant ROC index measured from the beginning of the fixation is 1360 ms for the example neuron (Fig. 7H) and 1082 ± 83 ms (mean ± SE) for the population.

Figure 7.

Comparison between the discrimination and the control tasks for correct decisions to the right and to the left. A, E, Firing rates of two single neurons during the discrimination task. C, G, Firing rates of the same neurons during the control task; shaded boxes signal the fixation period that has the same duration as that of the three periods (reference, delay, and test) in the CD task (Fig. 1B). B, F, D, H, ROC indices as a function of time during the CD and control tasks, respectively. Thicker lines indicate significant indices. I, J, Each row represents ROC indices for each neuron (n = 179). Firing rate shown in spikes per second.

The neural activity during the comparison period cannot be explained by incoming sensory information alone.

Although the neural activity during the comparison period could be related to the comparison/decision process, it could also reflect the encoding of sensory information, i.e., the orientation of the test stimuli. We investigated these possibilities further by analyzing the neural activity in trials in which the same sensory information was followed by different behavioral responses. We analyzed those reference/test pairs in which the test orientation was the same but the correct decisions were toward different directions (Fig. 1B). If the neural activity depends on the orientation of the test stimuli alone, the firing rate will be the same for all the test stimuli and the ROC index will not be significant (close to 0.5).

For the example neurons of Figure 3 and for the population (n = 359), the results shown in Figure 8 indicate that this is not the case, suggesting that the neuron activity depends on the monkeys' choice.

Figure 8.

Analyses of reference/test pairs in which the test orientations were the same but the correct decisions were toward different directions, provoked different neural responses. A, B, ROC indices as a function of time for decisions taken to the left versus decisions taken to the right for those trials in which the test orientations were the same; same two single neurons as in Figure 3. C, Each row represents the temporal evolution of ROC indices for each decision related neuron (n = 359).

Responses after the report of the decision

Once the behavioral report was made, the neural activity encoded the correct and incorrect outcomes as is shown for two example neurons in Figure 9, A, B, D, and E. The ROC indices as a function of time were calculated for correct versus incorrect trials (Fig. 9C,F). Monkeys received feedback about their execution (the reward for the correct trials and the modulation of the white noise for the incorrect ones) 50 ms after the behavioral response. The latencies of the first and the maximum significant ROC indices were taken from the moment of the behavioral response. For each of the two neurons, latencies of the first significant ROC indices were 460 and 120 ms and latencies of the maximum ROC indices were 680 and 400 ms, respectively. Correct and incorrect trials were compared in the 317 choice-related neurons for which we had sufficient data (see Materials and Methods). For 233 of 317 neurons (74%), the firing rate in the correct trials for both decisions was significantly different (p < 0.01) from the firing rate in the error trials. For 191 neurons of 233 (82%), the firing rate was higher for correct outcomes (Fig. 10A), and the latencies of significant ROC indices at the beginning and at the maximum were (mean ± SE) 384 ± 24 and 685 ± 27 ms on average, respectively (Fig. 10B). For 42 neurons (of 233; 18%), the firing rate was higher for incorrect outcomes (Fig. 10C), and the latencies at the beginning and at the maximum of significant ROC indices were (mean ± SE) 504 ± 29 and 753 ± 31 ms on average, respectively (Fig. 10D). To integrate the choices with their outcomes, a trace of the recently executed action has to be available. To explore this integration, the period after the behavioral response (0–1500 ms) was divided in 250 ms steps, and we combined the ROC indices obtained for choices (to the left vs to the right) and outcomes (correct vs incorrect). For each step, we computed the number of neurons with significant ROC indices for (1) the choices alone, (2) the outcomes alone, and (3) choices and outcomes together (Fig. 10E, note that the same neurons can be represented in more than one time period). During the period after the behavioral response, we found that 80 neurons (of 317; 25%) encoded only the previous choice, 120 neurons encoded only the outcomes (of 317; 38%), and 113 (of 3177; 36%) neurons encoded the choices and outcomes together at least in one 250 ms step. The neural activity for correct and incorrect responses suggests that the PMv cortex is involved in performance monitoring to regulate behavioral action rather than in signaling errors (Carter et al., 1998; Schall and Boucher, 2007).

Figure 9.

After the monkey gave the behavioral response (R), the single neurons differentiated correct from incorrect outcomes. A, D, B, E, Rasters and averaged firing rates of two single neurons sorted by correct and incorrect outcomes. Firing rate shown in spikes per second. C, F, ROC indices as a function of time for correct versus incorrect outcomes; thicker traces indicate significant indices.

Figure 10.

Neuronal population that differentiated correct from incorrect outcomes after the monkey gave the behavioral response (R). A, C, Distribution of maximum ROC indices for the group of neurons that responded more for correct (n = 191) and for the group of neurons that responded more for incorrect (n = 42) outcomes, respectively. B, D, Distribution of latencies; times taken from R for the first (light gray) and maximum (dark gray) significant ROC indices for the two populations. E, Number of neurons that carried significant information (ROC analysis) about the choices only (gray bars), the outcomes only (white bars), and the choices and outcomes together (black bars). Time intervals taken from the behavioral response (R). The same neurons can be represented in more than one time period.

Discussion

During the performance of the discrimination task, the PMv neural activity correlates with the monkeys' decision based on the comparison of a memory trace of recently perceived visual stimuli with the current one, including the strength of the evidence. This was reported previously with a vibrotactile discrimination task (Romo et al., 2004). The novelties of our results are that (1) the same codification occurs for the visual sensory modality, (2) neurons coding the decision rarely reflect the direction of eye movements, and (3) once the monkeys have communicated the results of their discrimination, the same neurons evaluate what has been reported by differentiating correct responses from incorrect ones and many of them reflect the decision again. These results suggest that the PMv could be part of a supervisory network involved in decision and evaluation of its outcomes independently of the sensory modality.

Neural correlates of the decision

The neural activity during the comparison period could be related to the decision, to the incoming sensory signal, or to the future motor action. There are several tests that helped us to elucidate between these possibilities. First, the neural activity was different when the monkeys used the same sensory information to take different decisions, suggesting that it is not related to the encoding of sensory information but to the decision process (Shadlen and Newsome, 2001; Romo et al., 2004). Second, the motor component of the task alone cannot explain the pattern of results we obtained. The neural activity correlates with the difficulty of the discrimination, a quality of the decision variable, that affects the temporal course of the decision process but not the motor report (Gold and Shadlen, 2000; Gold and Shadlen, 2007). Another approach was to compare the neural activity during the CD task and the control task, and the results suggest that (1) the majority of choice-related neurons were not modulated by the movement, and (2) for the remainder of neurons is the decision to move rather than the movement itself that drives them. In fact, goal-directed saccades are sparsely represented in the PMv (Fujii et al., 1998), suggesting an intermediate representation of directional information.

Temporal evolution of neural activity

To reach a decision, the monkeys have to compare the orientation of the second stimulus with the trace left by the first. Primate premotor areas have been associated with temporary storage functions and with being part of a rehearsal circuit involved in maintenance of information about item recognition (Cohen et al., 1997; Smith and Jonides, 1999; Rao et al., 2001; Hernandez et al., 2002; Cisek and Kalaska, 2004; Romo et al., 2004; Xiao, 2005). The memory traces of the references are available during the task periods, and their availability during the test presentation is especially relevant because it is during this period that the comparison between the memory trace and the second stimuli takes place. The PMv neuronal activity reflects the sequence of processing from sensation to action, including the comparison of memory trace and current stimulus (Romo et al., 2004).

The comparison process occurred for ∼50 ms during the first 200 ms of the test period, and, as a result, 250 ms after the beginning of the test period the neural activity signaled the choice, which was correlated with the behavioral response. During the comparison period, there is a representation of the relative orientation of the second stimuli instead of a representation of the absolute frequencies, as is the case in a vibrotactile task (Romo et al., 2004). The information about the visual stimuli in the task periods can come from striate and extrastriate (Markowitsch et al., 1987), parietal (Godschalk et al., 1984; Luppino et al., 1999; Borra et al., 2008), and prefrontal (Lu et al., 1994; Boussaoud et al., 2005) cortices and thalamic structures (Schell and Strick, 1984; Rouiller et al., 1999). During the first 100 ms of the comparison period, there are several interactions that rapidly evolved in time, and, because of this, a representation of the absolute orientation cannot be completely ruled out. Alternatively, it can be a weaker response to the visual modality.

The fact that the preference for the reference stimuli and the preferred choice were congruent for a majority of neurons suggests that the visual information is used for action. The parietal cortex (dorsal stream) is involved in sensorimotor transformations for visually guided actions, whereas the inferior temporal cortex (ventral stream) mediates information associated with perceptual identification of objects (Goodale and Milner, 1992). Our findings can be interpreted in the light of these two streams. First, the PMv is involved in sensory transformations for visually guided actions (Gentilucci et al., 1988; Rizzolatti and Luppino, 2001) and in perceptual decisions (Romo et al., 2004), and its lesions affect performance of sensorimotor tasks (Fogassi et al., 2001). Second, information from ventral visual stream areas involved in object discrimination (Tanaka, 1996) can reach the dorsal streams areas (Borra et al., 2008). Third, PMv receives information from both streams and shares functional properties through a circuit for visuomotor transformations for action (Mountcastle et al., 1975; Jeannerod et al., 1995; Sakata et al., 1995; Murata et al., 2000; Raos et al., 2006; Borra et al., 2008). Therefore, the findings reported here can represent visual perception for action.

Neural correlates of the outcomes

The same PMv neurons that encoded the decision during the comparison period differentiated correct from incorrect outcomes after the monkey reported that decision; interestingly, many of these neurons also encoded the sign of the decision again after the behavioral response. This finding suggests that the PMv cortex can be involved in sensory perception for action and in the evaluation of the decision consequences.

To evaluate performance and adapt future behaviors accordingly, subjects should know when erroneous and correct responses are made and integrate decisions with outcomes.

The results shown here suggest that, if the PMv is involved in supervising the evolution and consequences of actions, it could be part of the network involved in executive control. First, the PMv has anatomical connections with the SEFs, anterior cingulate cortex (ACC), the frontal eye fields (FEFs), and PFC (Hoover and Strick, 1993; Ghosh and Gattera, 1995; Dum and Strick, 2002; Boussaoud et al., 2005; Dancause et al., 2006). Second, single-unit recordings and lesion studies in subhuman primates, brain neuroimaging, and an event-related potential, the error-related negativity, in humans have shown activity related to performance monitoring in the ACC, SEF, and PFC (Falkenstein et al., 1991; Gehring et al., 1993; Dehaene et al., 1994; D'Esposito et al., 1995; Cohen et al., 1997; Carter et al., 1998; Courtney et al., 1998; Stuphorn et al., 2000; Ito et al., 2003; Kennerley et al., 2006; Michelet et al., 2007; Schall and Boucher, 2007; Rushworth and Behrens, 2008). After the behavioral responses, neural activity independently related to decisions and outcomes was described in the caudate nucleus (Lau and Glimcher, 2007), and the integration of decisions and outcomes in the same neurons was found in the SEF (Uchida et al., 2007) and PFC (Lee, 2006). The FEF is an important node in these oculomotor tasks because it participates in the transformation of visual signals into saccade motor commands (for review, see Schall, 1997). Third, the same PMv neurons that reflect correct and incorrect decisions during the comparison period also reflect whether the motor report was correct or incorrect. Furthermore, during the report period, many of these neurons also carried information about the decision taken. The fact that the temporal evolution of activation in the same PMv neurons correlates with signaling every component of the decision process, including the outcomes to correct and incorrect reports suggests that the weighed activity of these two groups of neurons may be involved in performance monitoring to shape behavioral action.

Conclusion

PMv cortex reflects the neural processes necessary to organize an evolving action, including the parametrical representation of memorized sensory stimuli, the comparison process, the choice, and the outcomes. These representations rapidly evolved in time, partially overlapping, suggesting that an accrual of all sources of information needed for a decision formation and evaluation occur in the PMv cortex. These results suggest that the PMv could be part of a supervisory network recording whether an evolving action leads to the desired consequences to shape future behavior.