Neural encoding of auditory discrimination in ventral premotor cortex

Abstract

Monkeys have the capacity to accurately discriminate the difference between two acoustic flutter stimuli. In this task, monkeys must compare information about the second stimulus to the memory trace of the first stimulus, and must postpone the decision report until a sensory cue triggers the beginning of the decision motor report. The neuronal processes associated with the different components of this task have been investigated in the primary auditory cortex (A1); but, A1 seems exclusively associated with the sensory and not with the working memory and decision components of this task. Here, we show that ventral premotor cortex (VPC) neurons reflect in their activities the current and remembered acoustic stimulus, their comparison, and the result of the animal's decision report. These results provide evidence that the neural dynamics of VPC is involved in the processing steps that link sensation and decision-making during auditory discrimination.

An important research theme in neuroscience is to understand how in the brain a sensory representation transforms into a behavioral decision (1–3). The contribution of sensory cortices in this cognitive operation has been investigated before, but the results suggest that they are primarily associated with stimulus encoding (4–9). In contrast, those cortical areas that receive inputs from sensory and memory circuits, and send outputs to the motor circuits appear well suited for elaborating perceptual decisions (10–16). For example, the ventral premotor cortex (VPC) receives projections from sensory areas of the parietal and temporal lobes (17–19) and association areas of the prefrontal cortex (20), and it sends projections to motor areas of the frontal lobe (18), subcortical structures (21) and spinal cord (22, 23). It has also been shown that VPC neurons possess both sensory (24, 25) and motor fields (26), and encode complex sensorimotor actions (27–29). Thus, VPC seems well suited to evaluate sensory events and convert them into a decision or motor report. Consistent with this interpretation, during somatosensory (14) and visual (30) discrimination tasks, the activity of VPC neurons reflects the transformation of sensory information into a perceptual decision. But, whether VPC is involved in a perceptual decision based on sound discrimination is unknown.

We addressed this problem by recording the activity of single neurons in VPC while trained monkeys discriminated the difference in rate of two acoustic flutter stimuli (range of 4–40 Hz) (9). The sensation of acoustic flutter is produced by slow repetitions of an acoustic stimulus. The rate of the acoustic flutter is determined by the interval between each pulse (each pulse lasts 20 ms at 1 KHz) in the stimulus trains (9). In the acoustic flutter discrimination task, monkeys report whether the second stimulus rate (f2) is higher or lower than the first stimulus rate (f1). This cognitive operation requires that subjects compare information of f2 with a stored trace of f1 to form a decision, i.e., whether f2 > f1 or f2 < f1, and to report their perceptual evaluation after a short, fixed delay. Here, we report that the activity of VPC neurons is involved in the entire processing sequence of steps that link the evaluation of auditory information into a decision report.

Results

Two monkeys were trained in the acoustic flutter discrimination task (Fig. 1A) until their psychophysical thresholds were stable (9). We avoided the issue of task difficulty during the recordings by using a stimulus set that had large differences between f1 and f2 (Fig. 1B). In this set, trials can be divided into two types: those in which f2 > f1 and those in which f2 < f1. This corresponds to the monkey's two possible choices. Notice also that, in this set, comparison stimuli can be preceded by base stimuli either higher (8 and 12 Hz) or lower (8 and 12 Hz). In other words, each of the f2 stimuli can be judged higher or lower, depending on f1. Thus, the neuronal responses across trials can be analyzed as functions of f1, f2, f2 - f1, or as functions of the monkey's two possible motor choices. We recorded 475 neurons that had task-related responses (Methods).

Fig. 1.

Discrimination task. (A) Sequence of events during discrimination trials. The mechanical probe is lowered, indenting the glabrous skin of one digit of the restrained hand; the monkey places its free hand on an immovable key (kd); after a variable delay 1–3 s, the first acoustic flutter stimulus (f1) is delivered; after a delay of 3 s, a second acoustic flutter stimulus (f2) is delivered; after another delay of 3 s between the end of f2 and probe up (pu), the monkey releases the key (ku) and presses either a lateral or a medial push-button (pb) to indicate whether the comparison (f2) stimulus rate was higher or lower than the base. (B) Stimulus set used during recordings. Each box indicates a base/comparison repetition rate stimulus pair. The number inside the box indicates overall percentage of correct trials for that (f1, f2) pair. (C) Recording site indicated by the gray circle. CS, central sulcus; ips, intraparietal sulcus; ls, lateral sulcus; VPC, ventral premotor cortex.

Responses of VPC Neurons during the Acoustic Flutter Discrimination Task.

When the discharges of VPC neurons were analyzed as functions of f1 (31), we found 17 neurons (3.5%) that modulated their firing rate as a function of f1 during the stimulation period (Table 1). Fourteen of these neurons varied their firing rate as a positive monotonic function of increasing f1 (Fig. 2) and 3 varied their firing rate as a negative monotonic function of increasing f1. This type of f1 encoding was also observed in 79 (16.6%) neurons that responded during the delay period between f1 and f2. Of these, 42 had rates that increased monotonically with increasing f1 (Fig. 3), and 37 had rates that decreased monotonically with increasing f1. Thus, more VPC neurons encoded f1 through their firing rates during the working-memory period than during the f1 stimulation period.

Table 1.

Database of VPC

	Task component
	f1	Delay f1-f2	f2	Delay f2- pu	mt
Tuned to f1	14 + (2.9%) 3–(0.6%)	42 + (8.8%) 37–(7.8%)	12 + (2.5%) 9–(1.9%)	64 + (13.5%) 54–(11.4%)	16 + (3.4%) 22–(4.6%)
Tuned to f2			28 + (5.9%) 20–(4.2%)	71 + (14.9%) 56–(11.8%)	13 + (2.7%) 21–(4.4%)
d			9 + (1.9%) 5–(1%)	8 + (1.7%) 13–(2.7%)	5 + (1%) 3–(0.6%)
c			10 + (2.1%) 7–(1.5%)	19 + (4%) 27–(5.7%)	8 + (1.8%) 13–(2.7%)

Recorded, n = 542. Responsive, n = 475 (87.6%). f1, first stimulus; f2, second stimulus; pu, probe up; mt, movement time; tuned to encoding stimulus frequency with positive (+) or negative (−) slopes; d, tuned to stimuli and differential activity for f2 > f1 or f2 < f1; c, categorical differential activity to f2 > f1 or f2 < f1.

Fig. 2.

Responses of a VPC neuron during the acoustic flutter discrimination task. (A) Raster plot of the neuron that responded during f1 and f2 stimulation periods. Each row of ticks is a trial, and each tick is an action potential. Trials were delivered in random order (only 5 trials per stimulus pair are shown; all neurons were tested with 10 trials per stimulus pair). Labels at left indicate f1:f2 stimulus frequencies. The stimulus set illustrated in Fig. 1B was used. (B) Spike density functions as a function of f1 (green traces) or f2 (red traces), and as function of the two possible comparison: f2 > f1 (f2 = f1 + 12 Hz) or f2 < f1 (f2 = f1 - 12 Hz). Data for Left and Middle are displayed as a function of f1 (green traces); data for Right are displayed as a function of f2 (red traces). The intensities of the green and red traces are functions of the increased stimulus repetition rate of f1 and f2. (C) Each panel shows the result of fitting equation firing rate (t) = a1(t)f1 + a2(t)f2 + a3*(t). In this formulation, t represents time, and the coefficients a1 (green dots) and a2 (red dots) serve as direct measurements of firing rate dependence on f1 and f2, respectively. These measures were calculated in sliding windows of 200 ms moving in steps of 100 ms. (D) The resulting coefficients a1 (green trace) and a2 (red trace) for this neuron are plotted in as functions of time. Filled circles indicate significant values. Color bars in D indicate periods of significant coefficients values. f1, first stimulus; f2, second stimulus; pu, probe up; pb, push button.

Fig. 3.

Firing rate modulation of a VPC neuron during the discrimination task. This neuron modulated its firing rates as a function of f1 during the delay period between f1 and f2, and during the postponed decision period for trials f2 < f1. Same format as in Fig. 2. (A) Raster plots. (B) Spike density functions as a function of stimulus repetition rates. (C and D) Resulting coefficients for f1 (a1, green) and f2 (a2, red) as function of time. These coefficients were calculated from the firing rate. This neuron encodes information about f1 during the delay between f1 and f2 and during the postponed decision period (end of f2 and the beginning of the cue signal that triggers the decision motor report). In addition, the neuron also responded for condition f2 < f1 at different times during f2 and between f2 and the cue signal that trigger the motor report. These coefficients had significantly different magnitudes (blue dots and bars) and statistically equal magnitudes and opposite signs (black dots and bars). Filled circles indicate significant values.

As the task progressed, responses reflected both f1 and f2 during the f2 stimulation period. We found 100 neurons (21%) that modulated their firing rates during the f2 period, as described below. Forty eight neurons responded selectively to f2: 28 had rates that varied as positive monotonic functions of increasing f2 while 20 had rates that varied as negative monotonic functions of increasing f2. Twenty one of the 100 neurons had firing rates that depended exclusively on f1: 12 had rates that varied as a positive monotonic function of increasing f1 and 9 varied as negative monotonic function of increasing f1. Thus, considerably more neurons had purely sensory responses during the f2 period than during the f1 period. However, the task requires that the difference f2 - f1 be calculated, but few neurons that responded during the f2 period reflected this operation. Thirty-one neurons (6%) discharged differentially during the f2 period; that is, their responses depended on f2 - f1. Nineteen neurons increased their firing rates selectively for f2 > f1 trials and 12 did so for f2 < f1 trials. Thus, few neurons encoded the comparison process between f2 and f1 during the f2 period.

This encoding scheme changed substantially during the postponed decision report period (delay between the end of f2 and beginning of the cue that triggers the beginning of the motor report). Again, during this period, the neuronal responses across trials can be analyzed as functions of f1, f2, f2 – f1, or as functions of the two possible motor choices. However, in principle, once the comparison between f1 and f2 is carried out, the monkeys only need to remember the signs of the difference (f2 > f1 or f2 < f1). If this is indeed the case, VPC neurons should reflect only the outcome of the comparison between f2 and f1, providing simply a categorical signal consistent with the decision motor report. However, many VPC neurons responded in an entirely different manner (Table 1). The large majority of the neurons reflected in their activities the sensory information (51.6%; Table 1) – again, with almost equal number of neurons with positive or negative encoding for both f1 and f2 – on which the decision is based, compared with the number of neurons that reflected the comparison f2 > f1 or f2 < f1 (n = 67, 14. %; Table 1). Thus, the number of neurons that modulated their firing rates during the postponed decision was considerably larger compared with number of neurons that responded during the stimulation periods and working memory period between f1 and f2 (Table 1). The activity of these neurons often displayed complex dynamics. For example, the neuron of Fig. 3A was modulated as a function of f1 during the delay period between f1 and f2, but in addition, it was differentially responsive during the postponed decision report period: it fired at higher rates for stimulus f2 < f1 than for stimulus pairs f2 > f1. These differential responses could be interpreted as encoding the motor choice, because discrimination of f2 > f1 trials and f2 < f1 trials is reported by pressing the lateral and medial push buttons, respectively, and similar responses were also observed in other VPC neurons for f2 > f1 trials (Table 1). This simple interpretation, however, does not hold for this neuron's response and other types of responses observed during the postdiscrimination delay period. For example, the neuron of Fig. 3A responded differentially, but their firing rates were also as a function of the differences between f2 – f1 stimuli. Many neurons had similar dynamics during the postponed decision report (Table 1), so we investigated these dependencies further.

Dynamic Encoding of Acoustic Flutter Discrimination in VPC.

To further quantify the different possible encoding schemes, we modeled the firing rates during the task as arbitrary liner functions of both f1 and f2, such that for each cell, firing rate (t) = a1(t)f1 + a2(t)f2 + a3*(t) (14, 32, 33). In this formulation, t represents time, and the coefficients a1 and a2 serve as direct measurements of firing rate dependence on f1 and f2, respectively. Because the constant associated with coefficient a3 can be an arbitrary value, for each neuron we set it to the mean firing rate calculated in the sample period studied. These measures were calculated in sliding windows of 200 ms moving in steps of 100 ms. To illustrate this analysis, the resulting coefficients a1 and a2 for the neurons of Figs. 2 and 3 are plotted in panels C and D as functions of time. The magnitude and sign of the coefficients reveals the tuning properties of the neurons — i.e., their selectivity — during the stimulus periods (Fig. 2 C and D) and during the working memory components between f1 and f2 and postponed decision report (Fig. 3 C and D). The response illustrated in Fig. 3 turns out to be close to the ideal expected decision motor report. This neuron responded strongly throughout the entire postponed decision period for f2 < f1 trials; furthermore, the analysis shows significant coefficients a1 (green trace) and a2 (red trace) of opposite signs and similar magnitudes, thus confirming a differential or categorical response (black dots and black trace in panels C and D of Fig. 3). The neuron in Fig. 3 C and D displayed, however, some important deviations from the ideal expected decision motor report. It responded briskly at different times during the postdecision delay for f2 < f1 trials, had significant coefficients a1 (green trace) and a2 (red trace) of opposite signs. But in this case a2 was slightly larger in magnitude than a1, indicating an f2 sensory component superimposed on the differential response (blue dots and blue trace). Some other neurons (n = 245) deviated even further from the ideal, and did not exhibit differential activity at all. The coefficients for some of these neurons revealed that, during the postponed delay period, these units carried only information about f1 or f2, because only a1 or a2 were significantly different from zero (Fig. 4C).

Fig. 4.

Dynamics of VPC population responses during the acoustic flutter discrimination task. (A) Values of a1 and a2 coefficients for all neurons selected plotted B. For each point, at least one coefficient is significantly different from zero. Different plots are for various times in A; n = number of neurons. (B) Number of neurons with significant coefficients as a function of time. Green and red traces correspond to a1 and a2, respectively. Blue trace corresponds to neurons with both significant a1 and a2 coefficients of opposite signs, but significantly different magnitudes; these are partially differential (d) responses. Black trace corresponds to neurons with both significant a1 and a2 coefficients of opposite signs and statistically equal magnitude; these are fully differential (c) or categorical responses encoding f2 - f1. (C) Bar graphs of 40 randomly selected neurons from the 475 neurons that contributed to panel B. These bars indicate periods of responses encoding f1 (green bars), f2 (red bars), partially differential f2 - f1 (blue bars) and fully differential or categorical responses encoding f2 - f1 (black bars). Each line of bars represents the dynamics of the responses of one single neuron during the discrimination task. The dynamics of these coefficients was analyzed using a sliding window of 200 ms duration moving in steps of 100 ms. Dynamics of these coefficients for an example neuron (Fig. 3) is shown in panel C (*). pu, probe up; ku, key up; pb, push-button.

Fig. 4A shows the numbers of neurons with significant a1 or a2 coefficients during the relevant task components and Fig. 4B as functions of time. According to the graph of Fig. 4B, some VPC neurons encode only f1 (green trace) starting after the onset of the base stimulus (response latency, 63.2 ± 125.8 ms; mean ± SD.). Subpopulations of VPC neurons continued to encode only f1 during the delay period between f1 and f2, during the comparison period (while the f2 stimulus is presented), and most remarkably, during the postponed decision report period until the monkeys indicate the motor choice. As expected, some VPC neurons respond as a functions of f2 only (red trace), starting after the onset of the comparison stimulus (response latency, 102.6 ± 60.9 ms), but as with f1, many units encode information about f2 during the postponed decision report period (Fig. 4C). In addition, other neurons combine information about f1 and f2 to generate differential responses (d, blue trace; Fig. 4C)). For some of these cells, the response is purely differential (c, black trace; response latency, 104.7 ± 121.13 ms after f2 onset), whereas for others one of the two acoustic stimulus rates is represented more strongly (blue trace; response latency was 123.8 ± 138.2 ms after f2 onset). The plot in Fig. 4B hides some interesting dynamics of single neurons; for instance, we found that many neurons that initially encoded f1 and f2 during the stimulation periods, respectively, became partially differential (d) or categorical (c) during the postponed decision report, and could switch back and forth with remarkable flexibility across the postdiscrimination period (Fig. 4B). However, it does convey how strongly a quantity is represented by the VPC population at any moment. In particular, it shows that the number of differential cells decreases sharply as a function of time during the postdiscrimination delay. In fact, during this period, more VPC neurons directly reflect the rates of the two stimuli rather than the motor choice.

VPC Activity Correlates with the Decision Report.

We also investigated whether these four types of responses predicted the animals' motor choice. For this, we sorted the responses into hits and errors and calculated a choice probability index (13, 14, 16, 34). This quantified for each (f1, f2) pair whether responses during error trials were different from responses during correct trials. Choice probability indices were computed separately for neurons that encoded information about f1 only, about f2 only, that were partially differential (d) or fully differential (c) (Fig. 4B). The result is shown in Fig. 5, which plots the four choice probability indices as functions of time. The four traces are significantly above 0.5, indicating that, during the postdiscrimination delay, there are significant differences in activity between trials that result in hits versus errors. These differences are maintained by neurons that contribute at different times during this period. The crucial point, however, is that even those neurons that encode only f1 or f2 have choice probability indices well above the 0.5 chance level. They show that all types of neurons are correlated with the animals' motor behavior. This means that their activity contributes to the observed variations in performance, even though, in principle, after the end of f2 only the categorical signals are needed for generating the postponed decision report.

Fig. 5.

Correlation between neuronal and behavioral responses. Choice probability indices as a function of time. Green trace: neurons that encoded information about f1; red trace: neurons that carried information about f2; blue trace: partially differential neurons that carried information about f1 and f2 (d); black trace: fully differential neurons that carried information about f2 - f1 only (c). The orange trace corresponds to neurons that had large choice probability indices and were tested in a control task in which animals received the same stimulus pairs but had to follow a visual cue to produce the motor choice response.

The responses during the postponed decision report could be interpreted in two different ways. One interpretation of this result is that the perceptual discrimination is not entirely consolidated once the second stimulus ends, but rather keeps brewing until the motor report is actually initiated. Another possibility is that the choice probabilities of VPC neurons simply reflect a purely motor signal. To investigate this, in addition to the standard tests, some of the neurons that carried information about f1, f2, or f2 - f1 were tested in a variant of the task in which the same acoustic stimuli were presented but the monkeys were instructed to press one of the push-buttons according to a visual cue (Methods). In this case, the auditory information could be ignored. Under this condition, the choice probability indices of VPC neurons dropped considerably (Fig. 5, orange trace). This suggests that at least part of the association between neuronal activity and behavior quantified by the high choice probabilities is because of sensory or perceptual processing. If so, the VPC neurons may be participating in a delayed decision-making process that takes place up to the moment when the postponed decision is reported.

Discussion

These results contrast with those obtained in primary auditory cortex (A1) during the same task (9). In that study, we found that A1 neurons responded exclusively during the stimulation periods, but not during the delay periods of the task. Thus, while A1 neurons responded exclusively during the stimulation periods, the activity of subpopulations of VPC neurons correlated with the task components of the acoustic flutter discrimination task. These results suggest that these two cortical areas play different roles in this task.

One could argue that the neuronal events recorded during the delay period reflect other processes, such as preparation for a future action. This seems unlikely, however, because (a) delay responses between f1 and f2 depended on f1 regardless of subsequent movements; (b) responses during the postponed decision period often reflected f1 or f2 information; (c) differential responses depended exclusively on f2 and f1; (d) ROC indices depended on hits versus error trials; and (e) when the same movements were guided by visual cues the f1, f2 and differential activity disappeared.

In principle, the neuronal responses recorded in VPC during the acoustic discrimination task seem similar to those recorded in the same premotor area using the vibrotactile discrimination task (14). But there are some differences. One could argue that these differences are because of task design, but the task sequences are similar. For example, in the acoustic flutter discrimination task, monkeys are asked to report the result of the comparison between f2 and f1 after a fixed delay (9). The same sequence is in the vibrotactile discrimination task, but monkeys are asked to report the comparison between f2 and f1 immediately after the end of f2 (14). Therefore, we can compare the response properties of VPC neurons in the two tasks until the comparison period. The comparative analysis shows that, in the acoustic flutter discrimination task, few neurons had f1 responses during the stimulus presentation compared with the number of neurons that responded during the same period in the vibrotactile discrimination task. This was not the case during the delay period between f1 and f2 since many neurons encoded information about f1, as in the vibrotactile discrimination task. Importantly, the same types of positive and negative monotonic f1 graded responses were found in the two tasks, both during the stimulation periods and delay period between f1 and f2. Some other important differences were also detected during the comparison period in the two tasks. For example, few neurons responded to f2 presentation in the acoustic discrimination task, in contrast more neurons responded to f2 in the vibrotactile discrimination task (14). Also, in the acoustic discrimination task very few neurons reflect in their activity whether f2 > f1 or f2 < f1, whereas more neurons reflected this operation in the vibrotactile discrimination task. Despite these differences between the two tasks, VPC contains neurons whose discharge rates varied during the delay period between the acoustic stimuli, as a monotonic function of the f1 rate. This result confirms our original proposal that behavioral tasks that require ordinal comparisons between scalar analogue stimuli would give rise to monotonic responses (31).

As indicated above, in the acoustic flutter discrimination task monkeys are asked to postpone their decision report (9). During this period, we observed many neurons that encoded f1 and f2 at different times during the postponed decision period. The encoding was similar to that observed for f1 and f2 during the stimulus presentation and for f1 during the delay between f1 and f2. Also, few neurons had differential responses for f2 > f1 or f2 < f1 during the postponed decision report period. Again, these responses occurred at different times. These responses are very similar to those observed in medial premotor cortex during the postponed decision report period, while monkeys executed the vibrotactile discrimination task (16). Thus, during the postponed decision report period, the dynamics of the neuronal population of VPC holds in line all of the elements associated with task execution. In fact, most of the neuronal responses associated with the postponed decision report are entirely task dependent. These encodings disappeared during the same task sequence, when monkeys are visually instructed which button to press to get reward.

The neuronal activities during the acoustic and vibrotactile tasks indicate that VPC seems well suited for integrating current sensory and memory signals of a given sensory modality. Indeed, the responses recorded in VPC reflect the encoding of the stimulus parameters both during sensory and working memory, the result of the comparison and the decision report (current results and those reported in ref. 14). These processing seem at the service of perceptual action that requires sensory evaluation. This process epitomizes a neural operation already described at the level of single neuron's responses in VPC: the mirror cells (27). In essence what a mirror cell reflects is the result of the comparison between current sensory information and previous experience, when a sensory evaluation requires a decision report. Our experimental results show this operation, at least to the extent possible within the minimalist environment of the laboratory tasks.

Because similar processes are observed in the acoustic, vibrotactile (14) and visual (30) discrimination tasks, we are tempted to suggest that VPC is involved with perceptual decisions, regardless of the sensory modality. This would suggest that the VPC circuit has the capacity to reorganize itself depending on perceptual task demands. The acid test, however, would be to probe whether single VPC neurons have the capacity to convert more than one sensory modality event into a decision motor report (35). Future experiments are needed to probe this conjecture in monkeys trained to discriminate more than one sensory modality.

Methods

Acoustic Flutter Discrimination Task.

The acoustic flutter discrimination task has been described before (9). Monkeys were handled according to the standards of the National and the Society for Neuroscience.

Recordings.

Neuronal recordings were obtained with an array of seven independent, moveable microelectrodes (14) (2–3 MΩ) inserted in VPC (area F5) contralateral to the responding hand/arm. Standard histological procedures were used to construct surface maps of all penetrations.

Data Analysis.

We considered a neuron's response as task-related if during any of the relevant periods [f1, delay between f1 and f2, f2, delay between the end of f2 and pu, reaction time or movement time] its mean firing rate was significantly different from a control period preceding the initial probe indentation at the beginning of each trial (Wilcoxon test, P < 0.01) (36). By definition, f1 and f2 correspond to the base and comparison periods, respectively. The first delay was into consecutive intervals of 500 ms beginning at the end of f1 and up to the beginning of f2. Similar intervals were used for the second delay between f2 and pu (Fig. 1A). The reaction time was the period from the end of ku to the beginning of the push button press (pb; Fig. 1A).

The f1-dependent responses during the stimulus period (500 ms) and during the delay between f1 and f2 (at least 500 of the 3000 ms) were defined as those that had an acceptable linear fit (χ² goodness-of-fit probability, Q >0.05) (33) for the mean firing rate as a function of stimulus frequency, where the slope was significantly different from zero (permutation test, n = 1000, P < 0.01) (36).

The dependence on f1 and f2 was obtained through multivariate regression analysis (32, 33). Errors in fit coefficients a1 and a2 were derived from the variance in responses to the individual (f1, f2) stimulus pairs and resulted in a full 2-D covariance matrix of errors. Coefficients were considered significantly different from (0, 0) if they were >2 standard deviations away. Neuronal responses were defined unambiguously as dependent on (f1, f2) if the coefficients of the planar fit were within 2 standard deviations of one of the two lines a2 = 0 or a1 = 0; responses were considered dependent on f2 - f1 (labeled as ‘c’ in Fig. 3 C and D) if the coefficients were >2 standard deviations away from these two lines and within 2 standard deviations of the line a2 = −a1. Responses not satisfying this criterion were classified as “mixed” (labeled as ‘d’ in Fig. 3 C and D). The dynamics of these coefficients was analyzed using a sliding window of 200 ms duration moving in steps of 100 ms.

ROC index was calculated using methods from signal detection theory (13, 14, 16, 34). This quantity measures the overlap between two response distributions, in this case between hits and errors for each (f1, f2) pair. Notice that a value of 0.5 indicates full overlap and 1 indicates completely separate distributions. Thus, the ROC index quantifies selectivity for one or the other outcome of the discrimination process. To compute it at different times, we used a sliding window of 500 ms duration moving in 100 ms steps, beginning 1000 ms before f1 and ending 1000 ms after the pu.

The beginning of the f1 tuned response (latency) was estimated for each neuron by identifying the first of three consecutive 10-ms bins displaced in steps of 1 ms after f1 onset, in which a1 was significantly different from zero and a2 was not significantly different from zero (14). The beginning of the f2 tuned response (red trace) was similarly estimated for each neuron as for the f1 response, but a2 was significantly different from zero. The beginning of the comparison response was estimated for each neuron by identifying the first of three consecutive 10 bins after f2 onset, in which a1 and a2 were significantly different from zero. We also required that a1 and a2 was two standard deviations away from a2 = −a1 line; that the signs of a1 and a2 were opposite and only a1 was significantly different from zero between 500 ms before and 100 ms after f2 onset; that the response became differential (f2 – f1) during the last 300 ms of f2 (these responses fall between the a1 = 0 and a1 = −a2 lines in Fig. 4A; blue dots and blue trace). The beginning of the categorical response (black dots and black trace) was estimated for each neuron by identifying the first three consecutive 10-ms bins, in which the coefficients a1 and a2 were significantly different from zero and both coefficients were within two standard deviations of the a2 = −a1 line (these values fall close to the diagonal as shown in Fig. 3 and 4A).

Trials in the control visual task proceeded exactly as described in Fig. 1A, but at the probe down (pd) the correct target push-button was illuminated. Acoustic stimuli were delivered while the light was kept on and at the end of the period between f2 and pu, the light was turned off; the monkey was rewarded for pressing the previously illuminated push-button. Hand/arm movements in this situation were identical to those in the acoustic discrimination task, but were cued by visual stimuli.