Estimates of the contribution of single neurons to perception depend on timescale and noise correlation

Abstract

The sensitivity of a population of neurons, and therefore the amount of sensory information available to an animal, is limited by the sensitivity of single neurons in the population and by noise correlation between neurons. For decades, therefore, neurophysiologists have devised increasingly clever and rigorous ways to measure these critical variables (Parker and Newsome, 1998). Previous studies examining the relationship between the responses of single middle temporal (MT) neurons and direction discrimination performance uncovered an apparent paradox. Sensitivity measurements from single neurons suggested that small numbers of neurons may account for a monkey’s psychophysical performance (Britten et al., 1992); but trial-to-trial variability in activity of single MT neurons are only weakly correlated with the monkey’s behavior, suggesting that the monkey’s decision must be based on the responses of many neurons (Britten et al., 1996). We suggest that the resolution to this paradox lies: 1) in the long stimulus duration employed in the original studies, which led to an overestimate of neural sensitivity relative to psychophysical sensitivity, and 2) mistaken assumptions (since no data were available) about the level of noise correlation in MT columns with opposite preferred directions. We therefore made new physiological and psychophysical measurements in a reaction time version of the direction discrimination task that matches neural measurements to the actual decision time of the animals. These new data, considered together with our recent data on noise correlation in MT (Cohen and Newsome, 2008), provide a substantially improved account of psychometric performance in the direction discrimination task.

Introduction

Psychophysical performance depends upon several critical features of the underlying neural pathways, including the sensitivity of single neurons, the amount of noise correlation between sensory neurons, and the amounts of noise (if any) introduced at the pooling and decision stages. An early attempt to measure these factors directly in behaving animals involved comparing psychophysical and neural sensitivity to motion direction; electrophysiological recordings were obtained from the middle temporal area (MT) of extrastriate cortex while monkeys discriminated opposed directions of motion in a noisy motion stimulus (Newsome et al., 1989; Zohary et al., 1994). These authors reported that the sensitivity of MT neurons is, on average, equivalent to psychophysical sensitivity, raising the possibility that performance is based on the responses of small numbers of neurons (Barlow, 1972). However, trial-to-trial correlation between fluctuations in neural responses and the animal’s choices (choice probability) was weak, suggesting that performance is based on signals pooled from many neurons (Britten et al., 1996). These data can be reconciled if substantial noise is introduced at the decision stage (Shadlen et al., 1996).

While this set of studies has been very influential, it and many others like it (e.g. Celebrini and Newsome, 1994; Hernandez et al., 2000; Prince et al., 2000; Recanzone et al., 2000; Uka and DeAngelis, 2003, 2004; Barberini et al., 2005) suffers from the key weakness that the time course of the neural measurements was not well matched to the time course of perceptual decisions. Neural measurements were made during a two-second viewing interval, but recent experiments suggest that monkeys avail themselves of only a few hundred milliseconds of stimulus exposure in making decisions in this task (Roitman and Shadlen, 2002; Kiani et al, 2008; Cohen and Newsome, 2008). Experimental estimates of neural sensitivity may therefore be artificially inflated relative to the measured psychophysical sensitivity. Indeed, a recent reaction time study by Cook and Maunsell (2002) indicates that, in a motion detection task, MT neurons are substantially less sensitive relative to psychophysical sensitivity than previously thought.

A second weakness of the original MT studies is that their models assumed that neurons in columns encoding different motion directions were uncorrelated, which had not been tested directly (Zohary, et al, 1994; Shadlen, et al, 1996). We have recently found that the trial-to-trial noise in stimulus-evoked responses are in fact positively correlated for all MT neurons with a common spatial receptive field (Cohen and Newsome, 2008), which greatly affects estimates of population sensitivity.

Our study assesses the direction information available in MT by measuring the sensitivity of single neurons on the timescale of perceptual decisions using a reaction time task. To relate the physiological measurements to performance, we adapted the model of Shadlen and colleagues to incorporate our updated measurements of neural sensitivity as well as new data on noise correlation (Cohen and Newsome, 2008). We found that on average, even optimally tuned MT neurons are considerably less sensitive than the monkey, and the updated model now provides a better account of psychophysical performance.

Methods

The subjects in this experiment were two adult male rhesus monkeys (Macaca mulatta, weight 13–15 kg). Prior to electrophysiological recordings, we implanted each animal with a scleral search coil for measuring eye movements (Judge et al, 1980), a head holding device (Evarts, 1968), and a recording cylinder (Crist Instruments, Damascus, MD) that provided access to MT. The monkeys performed a discrimination task for liquid rewards while seated in a primate chair with their heads restrained. All surgical and behavioral procedures conformed to guidelines established by the National Institutes of Health and were approved by the Institutional Animal Care and Use Committee of Stanford University.

Behavioral task & visual stimuli

All visual stimuli were presented on a CRT monitor positioned 57 cm from the monkey’s eyes. For the primary experiments, we trained two monkeys to perform a reaction time version of a two-alternative, forced choice (2AFC) direction discrimination task in which monkeys discriminated opposed directions of motions in a stochastic random dot display (Roitman & Shadlen, 2002; see Figure 1a). The monkeys fixated a central spot of light for 200 msec (“fixation period”), and then two saccade targets appeared (“target period”), each corresponding spatially to one of the two opposed directions of motion. The duration of the target period was selected from a truncated exponential distribution (minimum 150 msec, mean 900 msec, maximum 1800 msec) in order to discourage anticipation of the stimulus onset (Roitman and Shadlen, 2002). The motion stimulus then appeared (“stimulus period”) in the receptive field of the MT neuron under study (see “Recording methods” below). Once the stimulus period began, the monkey was free to indicate his direction judgment at any time by making a saccadic eye movement to one of the two targets flanking the stimulus. When the monkey’s eye left the fixation point, the fixation point and motion stimulus disappeared leaving only the two saccade targets present on the screen. To discourage rapid guessing, on correctly completed trials, the reward was delivered a minimum of 800 msec following the onset of the stimulus (for reactions times shorter than 800 msec) or immediately following the saccade.

Figure 1.

Task and behavioral data. a. Schematic diagram of the behavioral task. In each experiment we recorded from a well-isolated single MT neuron whose receptive field is schematized by region enclosed by the blue dashed circle in the first panel. A trial begins when the monkey fixates a central spot of light (first panel, fixation period). After 200 msec, two saccade targets appear, the position of which indicates the axis of motion to be discriminated, which was typically the preferred-null axis of the neuron under study (second panel, target period). After a random interval (drawn from a truncated exponential with a mean of 1100 msec, minimum 150 msec, maximum 1800 msec), the stimulus appears in the receptive field of the neuron under study (third panel, stimulus period). The monkey is free to indicate his direction judgment with a saccade to the appropriate target at any point after the stimulus appears. The stimulus and fixation point disappear as soon as the eyes leave the fixation window (fourth panel), and reaction time is defined as the time the stimulus was on the screen. b. Mean psychometric performance as a function of coherence for Monkey T (black) and Monkey D (grey). Mean performance data are based on all trials that contributed neural data to subsequent analyses (see text for inclusion criteria). Error bars represent standard error of the mean. c. Mean reaction time as a function of coherence for Monkey T (black) and Monkey D (grey). Data are based on the same set of behavioral trials as in panel b. Error bars represent standard error of the mean.

We only included trials for analysis if they met all of the following criteria: 1) the monkey viewed the stimulus for at least 150 msec without his eyes leaving the fixation point, 2) the monkey initiated a saccade within three seconds of stimulus onset, and 3) the monkey made a single saccade to one of the targets immediately after breaking fixation. During the fixation, target, and stimulus periods, the monkey was required to maintain fixation within a circular window (1–1.5° radius). On successfully completed trials, we defined the monkey’s reaction time as the time from the onset of the stimulus to the time the monkey’s eye left the fixation window.

We used stochastic random-dot stimuli that were similar to those used in many previous studies (see for example Britten et al, 1992; Roitman & Shadlen 2002). We varied the strength of motion, and therefore the difficulty of the task, by changing the probability that on a given frame, a given dot was replotted in apparent motion in one of two opposite directions (motion coherence). In each experiment, we selected a range of motion coherences that spanned psychophysical threshold, and motion direction and coherence were randomly interleaved from trial to trial. In most experiments, there were twice as many 0% coherence trials as any other single coherence. The monkeys received liquid rewards for correct choices on trials with non-zero coherence. For 0% coherence (random motion noise) trials, the monkeys were rewarded randomly with a probability of 0.5.

To prevent the monkey from using low-level visual cues to solve the task (e.g. when a dot appears in the upper right-hand corner of the screen, the correct answer is left), we used a new seed for the random number generator on each trial. Therefore, the exact placement of random dots over space and time differed on each trial, resulting in small spatio-temporal fluctuations in motion energy that were unique to each trial. The net motion energy in these fluctuations is zero on average, but slight fluctuations can in fact influence psychophysical judgments on individual trials (Bair, 1995; Neri and Levi, 2006; Kiani et al, 2008) and thereby influence choice probability measurements. For a subset of neurons recorded from Monkey T, therefore, we employed the same random number seed on all 0% coherence trials to control for effects of stimulus variance.

Recording methods

We recorded extracellular action potentials from 265 well-isolated, direction-selective MT neurons on tungsten microelectrodes using standard techniques (for example, see Britten et al, 1992). The neurons were recorded from four hemispheres in two monkeys (185 neurons from Monkey T and 80 neurons from Monkey D). At the beginning of each recording session, we inserted a stainless steel guide tube 1–3 mm past the dura. We advanced either one or two microelectrodes through the guide tube into the brain using a hydraulic microdrive (Narishige, Tokyo, Japan). We identified MT by the pattern of gray and white matter transitions during descent, the topographic organization of MT, and the well-known electrophysiological properties of MT neurons. We isolated neurons through a spike waveform template matching algoritihm (EXPO, Peter Lennie) or a dual window discriminator (Bak Electronics, Mount Airy, MD).

After isolating a neuron, we qualitatively measured the neuron’s spatial receptive field, and then for most experiments, we quantitatively measured a direction-tuning curve by displaying fully coherent motion in eight different directions (500 msec exposure) while the monkey fixated a small spot of light. We fit a circular Gaussian function to the firing rate data to determine the neuron’s preferred direction. We selected strongly direction selective neurons for study, our criterion being that distributions of responses to 100% coherent motion in the preferred and opposite (null) direction were non-overlapping. We chose this criterion so that our results could be directly compared to earlier studies which used the same criterion (Britten et al, 1992; 1996).

The main results in Figures 2–6 are from experiments in which we recorded from a single neuron on a single microelectrode (85 experiments from Monkey T and 30 experiments from Monkey D). In these experiments, we chose stimulus parameters such that the neuron under study would be ideally suited to perform this direction discrimination. We placed the stimulus in the neuron’s receptive field, and the monkey discriminated motion in the neuron’s preferred direction from motion in its null direction. The coherent dots moved at the neuron’s preferred speed. We included for analysis all neurons for which the monkey completed at least 30 trials per direction and coherence (mean 51 trials for Monkey T, 46 trials for Monkey D).

Figure 2.

Neurometric (grey) and psychometric (black) curves for four example experiments. a. A rare experiment in which neurometric and psychometric thresholds were not significantly different. b.,c. More typical examples in which psychometric performance was significantly better than neurometric performance. d. In a few experiments, the neuron failed to completely differentiate preferred from null motion even at the highest coherence.

Figure 6.

Frequency histogram of the correlation between firing rate and reaction time for 0% coherence trials in which the monkey reported motion in the neuron’s preferred (upward bars) and null directions (downward bars). The shaded bars indicate individual experiments for which this correlation was significantly different from 0 (Pearson’s correlation, p<0.05). The mean of the distribution of correlation coefficients for preferred direction choices is significantly less than 0 (t-test, p<0.001). The mean of the distribution for null direction choices is not significantly different from 0 (t-test, p=0.56).

The “off-axis” results in Figure 7 come from data collected primarily for experiments reported elsewhere (Cohen and Newsome, 2008) in the same two monkeys as the data presented here. In these experiments, we clamped two tungsten microelectrodes (Fred Haer, Bowdoinham, ME; 0.8–3 MΩ) together such that their tips were separated slightly in depth (range approximately 200–800 microns). We recorded from pairs of single neurons — one on each electrode — with largely overlapping receptive fields, and we placed the stimulus in the approximate union of the two receptive fields. We picked the stimulus speed that best drove both neurons.

Figure 7.

Neurometric performance and choice probability for off-axis neurons. a. Mean neurometric performance is plotted as a function of the angle between the neuron’s preferred direction and the axis of motion being discriminated for four example coherences. The break in the x-axis after 0 degrees off-axis indicates that the data for on-axis neurons were collected in a separate set of experiments from the off-axis neurons (see Methods). Error bars represent standard error of the mean. b. Mean choice probability is plotted as a function of the angle between the neuron’s preferred direction and the axis of motion being discriminated (choice probability was measured for only 0% coherence trials). Conventions as in a.

The monkey performed the usual 2AFC motion discrimination task, but the axis of discrimination did not correspond to the preferred-null axis of either neuron. Instead, on randomly interleaved trials, the monkey discriminated motion about the either the axis that bisected the angle between the preferred directions of the neurons under study or the orthogonal axis. Therefore, for each neuron, we have data for two angles between the neuron’s preferred direction and the motion axis, so each neuron contributes twice to the results in Figure 7. The difference in the preferred directions of the two neurons ranged from 3° to 178° in this data set, corresponding to 1° to 89° off axis.

Data Analysis

Time period for calculating spike rates

A primary goal of this study was to calculate the sensitivity and choice probability of MT neurons during the reaction time direction discrimination task and compare these results with sensitivities calculated during the fixed duration version of the same task (Britten et al. 1992, 1996). We therefore adapted the methods used to calculate neural sensitivity and choice probability in the previous studies to the data from our reaction time task.

In the fixed duration task (Britten et al, 1992), each trial had the same length, so neural sensitivity and choice probability were calculated based on spike counts from the entire two-second stimulus presentation period. The fact that trials in a reaction time task have different durations presents some difficulties for data analysis. For example, the visual transient, which is a period of characteristically high firing rate near the beginning of the stimulus presentation, will contribute more to the firing rate for short trials than for long trials. Also, the “motor preparation time” – the interval after the decision has been made but before the saccade is executed – can comprise a non-trivial fraction of the entire reaction time (as much as 300 msec, see Mazurek et al., 2003). Ideally, one would like to exclude spikes occurring during this interval since these spikes are presumably irrelevant to the monkey’s choice.

We experimented with excluding portions of the initial transient response and portions of the motor preparation time from our analyses. Within reasonable limits, however, these manipulations did not qualitatively affect the distributions of neural sensitivity and choice probability computed from our data set (data not shown). We therefore selected the simplest option, calculating the firing rate on each trial (spike count/reaction time) for the entire stimulus duration prior to saccade onset.

Neural sensitivity

As in previous studies, we calculated the sensitivity of single neurons in our direction-discrimination task by constructing a ‘neurometric function’ (for detailed methods see Britten et al, 1992). For each coherence, we used distributions of responses to preferred and null direction motion to calculate the receiver operating characteristic (ROC). The area under the ROC curve represents the proportion of trials in which an ideal observer could correctly distinguish preferred from null direction motion at this coherence given only the firing rate of the neuron under study and that of a hypothetical “antineuron” with the opposite preferred direction but otherwise identical response properties (Green and Swets, 1966). Thus, coherences for which the distributions of firing rates overlap minimally will have ROC area near unity, meaning that the firing of this neuron is sufficient to perfectly discriminate preferred- from null-direction motion at this coherence. Coherences at which the two distributions overlap completely will have ROC area of 0.5, meaning that an ideal observer would achieve only chance performance at this coherence. The neurometric curve is the fraction of trials the ideal observer could correctly discriminate at each of the coherences we tested (grey lines in Fig. 2).

We then compared the monkey’s psychometric curve and the neurometric curve by fitting each with a Weibull function given by:

$$p=1-0.5e^{-{(c/\alpha )}^{\beta }}$$

where p is the proportion of correct responses and c is the coherence of the stimulus. The parameter α represents the coherence at which the proportion of correct responses is 82%; we used this value for the neurometric or psychometric threshold. The parameter β represents the slope of the curve.

For each experiment, we used a bootstrap technique similar to that presented in Uka and DeAngelis (2004) to test whether the measured psychophysical and neurometric thresholds differed significantly. We computed 2000 mock psychometric functions from the observed (binary) distribution of correct and incorrect choices at each coherence, and we computed 2000 mock neurometric functions from the observed response distributions to preferred and null direction motion at each coherence. We considered the psychometric and neurometric thresholds to differ if the 95% confidence intervals for the means of the two bootstrapped distributions did not overlap (p < 0.05).

To generate each mock psychometric function, we sampled randomly (with replacement) from the observed distribution of correct and incorrect choices, building up a total number of “trials” for each coherence equal to the number of trials obtained in the actual experiment. We then fitted a cumulative Weibull to these mock data and calculated a threshold as described above. To compute each mock neurometric function, we sampled randomly (with replacement) from the observed distributions of firing rates for preferred and null directions at each coherence, again building up a total number of “trials” equal to that obtained in the actual experiment at each coherence. We then computed ROC curves from these mock data, fitted a cumulative Weibull to the set of ROC values for each neurometric function, and calculated a neural threshold as described above.

Choice probability

We calculated the trial-to-trial covariation between fluctuations in single unit firing rates and behavioral choice using the choice probability metric (Britten et al 1996). For 0% coherence trials, we divided the complete distribution of responses of each neuron into sub-distributions of trials in which the monkey guessed motion in the neuron’s preferred direction and trials in which the monkey guessed null direction motion. We then calculated an ROC curve from these two distributions and measured the area under the ROC curve to estimate the proportion of trials on which an ideal observer could predict the monkey’s impending choice based on the firing rate of the neuron under study.

We tested whether a neuron’s choice probability differed significantly from chance (0.5) using a permutation test. From the distribution of firing rates measured in each experiment (for 0% coherence stimuli), we randomly assigned each trial as either a preferred-direction or null-direction choice with the same probability that the monkey actually chose each direction. We then recomputed choice probability for the new distribution of firing rates as described above. We repeated this process 2000 times for each neuron to construct a distribution of choice probabilities expected by chance. We defined a significant choice probability as one that fell outside the 95% confidence interval for the mean of this distribution.

To compute the timecourse of the choice probability effect (Fig. 6), we filtered the raw spike trains with an exponential with a time constant of 100 msec and computed an average firing rate over time. This filter is causal; that is, a spike affects the filtered firing rate after it occurs, but not before. We chose this time constant so that our results can be easily compared with those of a recent study of the dynamics of MT responses during a motion detection task (Cook & Maunsell, 2002).

Designation of off-axis preferred directions

For the “off-axis” neurons in Figure 7, we needed to designate one of the two directions along the axis of discrimination as the “preferred” direction for the purposes of calculating choice probability or neurometric performance. Prior to beginning each experiment, we measured responses to 500 msec presentations of fully coherent motion in each of the two directions to be discriminated. We designated the direction that elicited the higher response to be the “preferred” direction.

Pooling Model

We used a neuronal pooling model similar to one developed by Shadlen and colleagues (1996) to test whether the measurements of neural sensitivity, choice probability, and noise correlation between pairs of MT neurons (see Cohen and Newsome, 2008) observed in the reaction time task could account for the monkeys’ psychometric performance. The model has been described in detail previously (Shadlen et al, 1996), so here we will describe only the basic structure of the model and the aspects in which our model differs from the previously published model.

The model simulates the responses of populations of MT neurons to the same set of stimuli employed in our experiments, computes a decision variable from the pooled (summed) population responses, and renders a psychophysical choice based on the computed decision variable. We simulated the responses of a population of MT neurons whose preferred directions ranged from 0° to 350° in increments of 10°. In all of our simulated trials, the model discriminated upward from downward motion. On each trial, the model makes a decision by comparing the summed firing rates of two pools of neurons, the pool of neurons whose preferred directions have an upward component, and those whose preferred directions have a downward component. We varied motion coherence, direction (up or down), the number of neurons in each pool, and the breadth of each pool (the greatest angle between a neuron’s preferred direction and the axis of motion for a neuron to be considered part of a pool, termed θ). We simulated responses of each neuron in the population to 10,000 trials of each motion direction and coherence.

As in the original model (Shadlen et al, 1996), we simulated the responses of individual MT neurons to reflect both the mean and the variance of the responses of the neurons we recorded. In addition, we simulated noise correlation between pairs of MT neurons that depended on the difference in their preferred directions as well as whether they contributed to the same or opposite pools (Cohen and Newsome, 2008), and we simulated reaction times to reflect the distributions of reaction times observed in our experiments.

Mean responses

The response, r_i,ϕ(k,c), of the i^th MT neuron with preferred direction ϕ° from the up-down axis of motion on the k^th trial of coherence c could be written,

$$r_{i,\varphi }(k,c)=\langle s_{i,\varphi }(c)\rangle +\epsilon ,$$

where 〈r_i,ϕ(c)〉 is the average response of the neuron to a stimulus of coherence c and ε is a noise term.

The average response depends on both the coherence and on ϕ, the angle between a neuron’s preferred direction and the up-down axis of motion. (The maximum allowable ϕ for a neuron to be considered part of the pool was θ, which we varied.) For optimally tuned neurons (those whose preferred directions were either up or down, so ϕ=0°), the average firing rate (spikes/sec) we recorded was

$$\langle r_0(c)\rangle =0.265c+23.32$$

for motion in the neuron’s preferred direction and

$$\langle r_0(c)\rangle =-0.072c+23.32$$

for motion in the neuron’s null direction. These equations represent best-fit lines (Britten et al., 1993) of the average responses of the neurons in our main data set at each of the coherences we tested.

For neurons with non-zero ϕ, we modeled the mean response by assuming cosine tuning. Therefore, the mean response for off-axis neurons was

$$\langle r_{\varphi }(c)\rangle =\langle r_0(c)\rangle cos(c).$$

We tried other reasonable tuning functions, and we found that the assumption of cosine tuning did not qualitatively affect our results (data not shown).

Covariance of simulated responses

In the original pooling model (Shadlen et al, 1996), the correlation in the trial-to-trial fluctuations in response to a given stimulus (noise correlation) between pairs of MT neurons placed important constraints on the performance of the simulated population of MT neurons. Estimates of noise correlation in the original model were based on data from pairs of neurons that were recorded on the same electrode and therefore typically had very similar tuning (Zohary et al, 1994; Bair et al., 2001). Based on these results, the original model postulated that neurons within a pool have relatively high noise correlation and that noise between pools was independent.

In more recent experiments (Cohen and Newsome, 2008), we found that noise correlation depends both on the difference between the preferred directions (ΔPD) of two neurons and on whether they belonged to the same or different perceptual pools, and that noise correlation was, on average, positive for all pairs of neurons with overlapping receptive fields. We therefore simulated covariance between pairs of neurons in our model to match the noise correlation observed in our experiments. The correlation C, between neurons i and j when the two neurons are in the same or different pools was given by:

$$\begin{array}{c}{C}_{i,j,\mathit{same}\mathit{poolj}}=-0.0011({PD}_i-{PD}_j)+.22,\\ C_{i,j,\mathit{different}\mathit{pools}}=-0.0011({PD}_i-{PD}_j)+.19,\text{for}({PD}_i-{PD}_j)<135,\text{and}\\ C_{i,j,\mathit{different}\mathit{pools}}=0.00156({PD}_i-{PD}_j)-.17,\text{for}({PD}_i-{PD}_j)>135,\end{array}$$

where PD_i is the preferred direction of neuron i, and all directions are in degrees (see Cohen and Newsome, 2008, Figure 4).

Figure 4.

Frequency distributions of measured choice probabilities. a. Choice probabilities measured on 0% coherence trials for 44 neurons in monkey T; a different random number seed was used on each trial. The shaded bars indicate experiments for which the choice probability was significantly different from 0.5 (p<0.05, bootstrap test described in Methods). The black arrow indicates the mean of the distribution. The mean is significantly greater than 0.5 (t-test, p<0.05). b. Same, for 30 neurons in Monkey D. Again, the mean is significantly greater than 0.5 (t-test, p<0.05). c. Same, for 41 neurons in monkey T, but using the same random number seed for every trial in a given experiment. The mean remains significantly greater than 0.5 (p<0.05).

We used a method very similar to the one used in the original pooling model to impose noise correlation in our simulated MT responses. The variance of the distribution of responses of a particular neuron to a stimulus of a given direction and coherence was 1.5 times the mean. We chose the Fano factor of 1.5 because it is physiologically realistic and the same as that used in the original model (see Shadlen et al, 1996).

As in the original pooling model, we imposed noise correlation according to the desired correlation matrix, C, by calculating its matrix square root, Q, such that

$$C=QQ\text{'}.$$

Q is the unique matrix square root of C such that every eigenvalue has a non-negative real part.

We then formed a correlated matrix of standard deviates for the N neurons by generating a vector of independent, normal deviates, x, with zero mean and unit variance and multiplied it by Q. The resulting matrix,

$$y=Qx,$$

has covariance C (see Appendix 1 of Shadlen et al, 1996 for a complete derivation).

We then offset and scaled y to get our matrix of noisy responses. Therefore, the responses, r, of the population of MT neurons on the k^th trial with coherence c is a one by N vector and could be written

$$r(k,c)=\langle r(c)\rangle +y\sqrt{1.5\langle r(c)\rangle },$$

where 〈r(c)〉 is a one by N vector of average responses that depends on each neuron’s θ as above.

We generated simulated responses of the population of MT neurons while varying direction, coherence, pool size, and θ, which is the maximum ϕ to be considered part of a pool. For simulations in which θ was non-zero, we randomly selected the preferred directions of the neurons in one pool from a uniform distribution of allowable preferred directions and then set the preferred directions of the neurons in the opposite pool to be 180° opposite to the preferred directions in the first pool. Note that for θ=180°, neurons whose preferred directions are orthogonal to the axis of motion being discriminated might belong to either pool. In this case, we added orthogonal neurons to both pools, and their responses differed only in the pool-dependent noise correlation we imposed. We then calculated the model’s psychometric data and the choice probability for each simulated neuron.

Reaction time

We wanted our version of the pooling model to capture the short timescale of perceptual decisions revealed by the reaction time task as well as the trial-to-trial variability in reaction times. However, the decision rule or algorithm by which the reaction time on a given trial is determined (see for example Roitman and Shadlen, 2002; Wang, 2002), will likely affect the model’s predictions. Models incorporating decision rules rely on many parameters, any of which may affect predicted psychometric thresholds and choice probabilities. Detailed models of this sort are a subject of active investigation by several laboratories.

Because the goal of our modeling exercise was to show that simple modifications of the original pooling model (Shadlen et al, 1996) can account for our data in the reaction time task, we chose the simplest possible decision rule; reaction times were selected randomly on each trial. We simulated reaction time on a given trial by sampling, with replacement, the distribution of reaction times observed in the actual experiments for each coherence. For example, the reaction time for a given 12% coherence trial was picked from the distribution of reaction times on 12% coherence trials for both monkeys. Therefore, the distributions of reaction times in our simulated trials matched the combined distributions of reaction times of the two monkeys.

Results

Behavior

We trained both monkeys to perform the reaction time direction-discrimination task described in Figure 1a (see Methods). Figure 1b and c show average psychometric and chronometric curves, respectively, for Monkey T (black) and Monkey D (grey). Consistent with previous results in this task and many others (e.g. Roitman and Shadlen, 2002), psychophysical performance was more accurate (Fig. 1b) and reaction times were shorter (Fig. 1c) as coherence increased. These relationships held for both monkeys individually, although Monkey T generated significantly lower psychophysical thresholds (t-test, p<0.01; mean threshold = 18.1% coherence and 25.8% coherence for monkeys T and D, respectively) and had slower reaction times than Monkey D (t-test, p<0.001; mean reaction time = 674 msec and 585 msec on 0% coherence trials for monkeys T and D, respectively).

Sensitivity of single neurons

Figure 2 shows four examples of neurometric (grey) and psychometric curves (black) measured simultaneously. The curves in Figure 2a are largely overlapping and the corresponding neurometric and psychometric thresholds are statistically indistinguishable, meaning that the monkey, given perfect access to the firing rates of his MT neurons, could have achieved the observed psychometric performance by relying solely on the responses of the neuron we recorded. These neurons were the exception, however. More typically, neurometric thresholds were higher than psychometric thresholds (for example, Figs. 2b and c), meaning that the monkey’s psychometric performance was better than could be achieved based on the motion information available in the responses of a single neuron. In a few cases, such as the example in Figure 3d, neurometric performance was substantially below 100% correct, even at the highest coherence tested. We selected for study only neurons whose distributions of responses to 500 msec presentations of fully coherent motion in the preferred and null directions did not overlap (see Methods). At the highest coherences, however, the monkeys’ reaction times were usually shorter than 500 msec, so some neurons, like the one in Figure 2d, did not fully discriminate motion at the highest coherence.

Figure 3.

Frequency histograms of neurometric to psychometric ratios (neurometric threshold divided by psychometric threshold) for each monkey. The black arrow indicates the geometric mean of each distribution. The means of both distributions are significantly greater than 1 (p<0.01, t-test on the log-transformed ratios). a. Monkey T. b. Monkey D.

Figure 3 shows frequency histograms of the ratio between the neurometric and psychometric thresholds for each experiment and each monkey. Ratios less than one indicate that the neurometric threshold was lower than the corresponding psychometric threshold, so the monkey performed worse than the signaling of the single neuron. Ratios greater than one indicate that the monkey performed better than the single neuron. The geometric mean of both distributions is significantly greater than one, indicating that on average, the monkeys performed better than they could have based on a single neuron (geometric mean = 2.67 for Monkey T, p<0.001; geometric mean = 2.19 for Monkey D; t-test, p<0.005 in both cases).

Shaded bars on the histogram indicate experiments in which the neurometric and psychometric thresholds were significantly different (p<0.05, bootstrap test described in Methods). Of the 85 experiments in Monkey T, 67 (79%) yielded significantly different neurometric and psychometric thresholds. Of these, neurometric thresholds were higher than the corresponding psychometric thresholds in 65 out of 67 experiments (97%). In Monkey D, 24 out of 30 (80%) experiments yielded significantly different neurometric and psychometric thresholds, and of these, neurometric thresholds were higher in 23 (96%).

The mean neurometric threshold for the two monkeys was not significantly different (mean 51% coherence for Monkey T and 56% coherence for Monkey D; t-test, p=0.31), indicating that their neurons showed the same average sensitivity to motion direction. Because their psychometric thresholds were different, their mean neurometric to psychometric ratios were different as well. Even the monkey with poorer psychometric performance, however, had a mean neurometric threshold more than twice his mean psychometric threshold (Fig. 3b).

Choice probability

We quantified the correlation between firing rate and choice (choice probability) on 0% coherence trials by using ROC analysis as described in Methods. Figures 4a and b illustrate histograms of choice probability for Monkeys T and D, respectively. The means of both distributions are significantly greater than 0.5 (t-test, p<0.05; mean = 0.5403 and 0.5377 for monkeys T and D, respectively). Mean choice probability was not significantly different for the two monkeys (t-test, p = 0.56). The filled bars on the histograms indicate individual experiments for which the choice probability was significantly different from 0.5 (p<0.05, permutation test as described in Methods). For the two monkeys, 21 of 25 neurons (84%) with significant choice probabilities were in the intuitively appropriate direction, with higher firing rates for preferred direction choices. There was no correlation between psychometric threshold and choice probability for either monkey (r = 0.006, p = 0.96 for Monkey T; r = −0.09, p=0.33 for Monkey D). There was also no significant correlation between neurometric threshold and choice probability (p = 0.39, ANCOVA with monkey identity as a coregressor).

On most experiments, the random dot placement in each 0% coherence trial was created using a different seed for our random number generator. To determine whether small amounts of random motion energy accounted for choice probability, we used the same random seed for 41 experiments in Monkey T. Figure 4c shows the frequency histogram of choice probabilities for these experiments. Consistent with a prior report (Britten, et. al., 1996), the mean choice probability in these experiments (0.5301) was significantly greater than 0.5 (t-test, p<0.05) and statistically indistinguishable from the mean choice probability from experiments in Monkey T in which a new random seed was used for each trial (t-test, p=0.22).

Significant choice probabilities (and also the correlation between firing rate and reaction time—see below) show that the firing rate of MT neurons during the stimulus presentation interval is closely related to the monkey’s decision process. We examined the detailed time course of choice probability in an effort to gain insight into the dynamics of the decision process. We first filtered the spike trains (see Methods) to obtain a firing rate at each millisecond for each neuron for each 0% coherence trial. We then computed choice probability using the instantaneous firing rates at each millisecond for each neuron.

Figures 5a and b depict the average choice probability at each time point across all neurons for Monkeys T and D, respectively; the width of the line represents the mean ± standard error. The traces are aligned to stimulus onset in the left panels and to saccade initiation in the right panels. At the beginning of the trial, the mean choice probability is near 0.5 for both monkeys, indicating that neural activity before motion signals reach MT is not correlated with the monkey’s eventual perceptual judgment. Both monkeys’ choice probability rises after stimulus onset, peaking at different times for the two monkeys (roughly 200 msec after stimulus onset for Monkey T and roughly 120 msec for Monkey D). For both monkeys, mean choice probabilities fall as the saccade approaches; Monkey T’s reaches chance levels approximately 150 msec before the saccade, while Monkey D’s mean choice probability reaches chance roughly 70 msec before the saccade. The slow decay in choice probability may be in part due to the 100 msec exponential filter we used to determine firing rates on each trial (see Methods). However, the slow rise in choice probability cannot be due to the filter because the exponential filter is causal, and is consistent with previous results in MT (Cook and Maunsell, 2002) and V2 (Nienborg and Cumming, 2006).

Figure 5.

Time course of choice probability aligned to the onset of the stimulus (left panels) and the initiation of the saccade (right panels). The thickness of the line represents the mean ± standard error. a. Monkey T. b. Monkey D.

Correlation between MT firing rates and reaction time

Choice probability measures the correlation between the trial-to-trial fluctuations of MT responses and the monkey’s binary decision. The reaction time task gives us an additional handle on the monkey’s decision process – his response time. Figure 6 depicts a frequency histogram of the correlation coefficients between each neuron’s firing rate and the monkey’s reaction time on 0% coherence trials. We computed the correlation coefficients independently for trials on which the monkey chose the neuron’s preferred direction (upward bars) and trials on which he chose the null direction (downward bars). Shaded boxes indicate experiments for which the correlation coefficient was significantly different from zero.

For preferred direction choices, reaction times were significantly faster on trials in which the neuron’s firing rate was higher than average. The mean correlation coefficient was −0.161 for the two monkeys combined, and this was significantly less than 0 for the combined distribution (t-test, p<0.001) as well as for each monkey individually (t-test, p<0.01, data not shown). For the 41 neurons in Monkey T in which we used the same random number seed on each 0% coherence trial, the mean correlation coefficient was −0.176 which was also significantly different from 0 (t-test, p<0.01) and was not significantly different from the mean correlation coefficient for the rest of the neurons in our sample (t-test, p=0.30). For null direction choices, firing rate and reaction time were not significantly correlated (mean = −0.01 across both monkeys; t-test, p=0.56).

Off-axis neurons

In separate experiments, we obtained neurometric threshold and choice probability data for 150 MT neurons while the monkey preformed the direction-discrimination task along an axis different from the preferred-null axis of the neuron under study (first presented in Cohen and Newsome, 2008). Figure 7a shows average neurometric performance (area under the ROC curve) as a function of the angle between the neuron’s preferred direction and the axis of motion the monkey was discriminating (termed ϕ) for four different coherences. For each non-zero coherence, performance predictably decreased as a function of ϕ. Perhaps surprisingly, neurometric performance was nearly 70% correct on the highest coherence, even for the least sensitive, or farthest off-axis, group of neurons (ϕ between 67.5° and 90°).

Figure 7b depicts average choice probability as a function of ϕ (error bars represent the standard error of the mean). Again, choice probability predictably decreases as a function of ϕ, but notably, it is significantly higher than 0.5 for each bin (t-test, p<0.05). This result is consistent with other studies showing that even very off-axis neurons have choice probability greater than 0.5 in the fixed duration version of this task (Britten et al, 1996) and in other tasks involving moving stimuli (Purushothaman and Bradley, 2005; Bosking and Maunsell, unpublished observations). Together, these results suggest that even the least sensitive neurons carry useful motion information (Fig. 7a) and their responses are correlated with the monkey’s decisions (Fig. 7b).

Because the data for off-axis neurons were collected in a separate set of experiments from the on-axis neurons (in which task difficulty, similarity between a neuron’s tuning and the stimulus, and other factors were different), we analyzed the relationship between degrees off-axis and neurometric performance (Fig. 7a) and choice probability (Fig. 7b) without including the on-axis neurons. For the off-axis neurons (all data points to the right of the hash marks on the x-axis of each figure), there was a significant negative correlation between degrees off-axis and proportion correct for the higher two coherences shown (96% coherence: r = −0.38, p<0.0001 and 24% coherence: r = −0.24, p<0.01) and a non-significant trend toward a negative correlation for 6% coherence (r = −0.08, p = 0.11). There was also a weak but significant correlation between choice probability and degrees off-axis for these neurons (r = −0.13, p <0.05). These results indicate that the responses of neurons whose preferred directions are far from the relevant axis of motion are less informative and less correlated with behavior than neurons whose preferred directions are more closely aligned with the axis of motion being discriminated.

Measurements of neural sensitivity and correlation in the reaction time task better account for observed psychometric performance and choice probability

A central goal of this study was to reconcile the monkey’s psychophysical performance with the neurophysiological data in the reaction time task. Specifically, we instantiated a version of a pooling model (Shadlen et al, 1996; see Methods) to determine whether our measurements of neural sensitivity in the reaction time task, considered together with the task-dependent noise correlation we measured using simultaneous recordings of pairs of MT neurons in this task (Cohen and Newsome, 2008), could explain our monkeys’ psychophysical performance and the neuronal choice probability. We were interested in the model’s predictions as a function of pool size (the total number of neurons in each of the two pools) and θ, the width of each pool (the maximum angle between a neuron’s preferred direction and the axis of motion being discriminated for a neuron to be considered part of one of the pools).

Figure 8a plots the model’s predicted psychometric threshold as a function of pool size for three values of θ. As in the original pooling model based on data from the fixed duration task (Shadlen et al, 1996), performance improves (threshold decreases) as a function of pool size, becoming asymptotic at a pool size of 50–100 neurons because noise common to the entire pool cannot be averaged away by recruiting additional neurons (see Methods). Predicted psychometric threshold increases with θ; the model predicts a psychometric threshold of about 10% coherence at asymptote for θ=0°, 13% coherence for θ=45°, and 16% coherence for θ=90°. Monkey T’s average threshold was 18.1% coherence (lower dashed line in Figure 8a), and Monkey D’s threshold was 25.8% coherence (upper dashed line in Figure 8a). By comparison, the most similar model in the original study predicted an asymptotic threshold of 6% coherence (arrow in Figure 8a).

Figure 8.

Predictions of the updated pooling model. a. Predicted psychometric threshold as a function of pool size for three values of θ, which can be considered the “width” of the pool. Specifically, θ is the maximum angle between the preferred direction of each neuron in the pool and the axis of motion being discriminated. The dashed lines represent the mean psychophysical threshold observed in experiments for Monkey T (upper) and Monkey D (lower), and black arrow represents the predicted asymptotic performance in the original pooling model (Shadlen et al, 1996). b. The average choice probability predicted by the pooling model as a function of pool size. Predicted choice probability also depends weakly on θ, the maximum angle between the preferred direction of each neuron in the pool and the axis of motion being discriminated. The dashed line represents the average observed choice probability, which was nearly identical for the two monkeys (compare Fig. 5a and 5b). The black arrow represents the predicted asymptotic choice probability in the original simulations of Shadlen et al, 1996. c. Predicted choice probability is plotted as a function of the angle between the neuron’s preferred direction and the axis of motion being discriminated (compare to Fig. 7b).

Figure 8b depicts the model’s predicted choice probability as a function of pool size for the same three values of θ. The predicted choice probability is the average choice probability of all neurons involved in the decision (and therefore the average of all included values of θ.) Like psychometric threshold, predicted choice probability decreases with pool size and asymptotes around 50–100 neurons per pool. The initial fall in predicted choice probability occurs because the variance in the outcome of the decision process is spread over an increasing number of noisy neurons, and each individual neuron is therefore less correlated with the final decision as pool size grows. The choice probability remaining at asymptote is due again to common noise in the pool of sensory neurons which cannot be decreased by adding additional neurons. In addition, predicted choice probability decreases with θ. The model predicts asymptotic choice probabilities of about 0.585 for θ=0°, 0.582 for θ=45°, and 0.575 for θ=90°. These values are somewhat higher than our measured choice probability (approximately 0.54 for both monkeys, dashed line in Figure 8b) but are much closer to the observed values than the asymptotic prediction of the original pooling model (0.65, arrow in Figure 8b).

Our model predicts lower choice probability than the original model primarily because, having no data to the contrary, the original model assumed that the two pools were independent (had uncorrelated noise). In recent experiments, however, we found that the noise in the responses of the vast majority of MT neurons with largely overlapping spatial receptive fields, including those with very different preferred directions, is positively correlated to some degree (Cohen and Newsome, 2008), a finding which is consistent with data from motor cortex (Lee et al, 1998; Maynard et al, 1999; Averbeck and Lee, 2003) and primary visual cortex (Kohn and Smith, 2005; Smith and Kohn, 2008). Our current model reflects these widespread positive correlations (see Methods). Because the model makes decisions by comparing the firing rates of the two pools on each trial, correlated noise between pools decreases choice probability. This result can be appreciated intuitively by recalling that choice probability measures the correlation between a neuron’s noisy response and the monkey’s decision. If that noise is also correlated with evidence for the opposite decision, choice probability will necessarily go down. This reasoning in fact suggests a more general inference: choice probability magnitude depends substantially on the difference in correlation between the two pools of sensory neurons, irrespective of the absolute correlation levels present in the pools.

To test this inference, we conducted additional simulations, adding a constant amount of correlation to all pairs of MT neurons. Therefore, the correlation C_i,j between neurons i and j was exactly as described in Methods, plus a constant. We repeated this procedure for several different levels of added correlation, ranging from −0.5 to 0.5. Pleasingly, the resulting choice probabilities were exactly those plotted in Figure 8b as long as no individual pairwise correlations exceeded +/− 0.8. These results indicate that choice probability magnitude and correlation are intimately entwined; choice probability is determined by the relative correlations between the two pools of neurons.

Unlike in the original model, our predicted choice probability depends slightly on θ (Figure 8b) because we observed that noise correlation between neurons within a pool decreases as the difference between their preferred directions increases (Cohen and Newsome, 2008). Therefore, the model predicts that choice probability will decrease with the distance between a neuron’s preferred direction and the axis of motion being discriminated. Figure 8c plots predicted choice probability as a function of degrees off axis. This relationship is qualitatively similar to our physiological results (Fig. 7b), though the observed decline in choice probability as a function of degrees off-axis was noisy and not as steep as the one plotted in Fig 8c. Predicted choice probability as a function of degrees off axis does not depend on θ, so we plot only a single line here (as opposed to Figures 8a and 8b). These results indicate that in the model, the primary effect of broadening the pool width is incorporating more neurons with low choice probabilities, leading to a lower overall predicted choice probability for high θ (Fig. 8b).

Does choice probability indicate that a neuron is involved in the perceptual decision?

Choice probability was originally conceived as one way to establish that the responses of particular sensory neurons causally contribute to the animal’s perceptual decision. The reasoning was that a trial-to-trial covariation between the fluctuations in a neuron’s response and the monkey’s choices indicates that the responses of that neuron are likely to contribute to the pool of evidence that influences the monkey’s perceptual choices. Indeed, in motion processing tasks similar to ours, the role of optimally tuned MT neurons in perception has been confirmed not only by their significant choice probability (Britten et al., 1996; Dodd et al., 2001; Cook and Maunsell, 2002; Krug et al., 2004; Barberini et al., 2005; Liu and Newsome, 2006), but also from studies using causal techniques such as inactivation through lesions or pharmacology (Newsome et al., 1985; Newsome and Pare, 1988; Schiller, 1993; Schiller and Lee, 1994; Orban et al., 1995; Rudolph and Pasternak, 1999) or activation through microstimulation (Salzman et al., 1990; Salzman and Newsome, 1994; DeAngelis et al., 1998; Bisley et al., 2001; Nichols and Newsome, 2002; Ditterich et al., 2003; DeAngelis and Newsome, 2004). However, new evidence from this study as well as our previous study measuring the responses of pairs of MT neurons (Cohen and Newsome, 2008) leaves open the possibility that: 1) choice probability can occur even for neurons that are not causally involved in a task, and 2) these “spurious” choice probabilities can be just as large as those measured from neurons that are causally involved.

The first point is fairly obvious. Neurons that do not contribute causally to downstream decision mechanisms can nevertheless exhibit a significant choice probability if they are positively correlated with those that do. Thus positive noise correlation within a pool of sensory neurons can lead to “spurious” choice probability effects that are not indicative of causality. The second point, however, is more subtle. One might initially think that choice probabilities should be quantitatively larger for neurons that contribute causally to the decision mechanism, but simulations presented in this section (which are similar to those presented by B. Cumming, oral presentation) show that this is not the case for pool sizes exceeding roughly 20 neurons.

We simulated the responses of 1000 neurons with identical tuning and firing properties (rectangle in Figure 9a). These neurons can be thought of as on-axis neurons in a single perceptual pool. We simulated a choice on each trial based on the responses of a small, randomly selected subset of the population that comprised a decision pool (solid oval in Figure 9a) while ignoring the responses of the remaining neurons (dashed oval in Figure 9a). Each neuron in the population had a mean firing rate of 50 spikes per second, and we imposed zero-mean, correlated Gaussian noise such that each neuron had a Fano factor of 1.5 using methods similar to those we used for the pooling model in Figure 8. In each simulation, the noise correlation between all pairs in the entire population was the same (regardless of whether the neurons were involved in the decision), and we ran simulations with noise correlation coefficients of 0, 0.05, 0.1, or 0.2 (blue, green, red, and cyan lines in Figure 9b). We modeled the simplest possible 2AFC task; the monkey chose “yes” when the mean activity of the neurons in the decision pool (solid oval in Figure 9a) was greater than 50 spikes per second and “no” otherwise. To generate Figure 9b, we simulated responses to 10,000 trials per correlation and pool size; using this method, the simulated “monkey” made 50% “yes” choices (or approximately 5000 trials) and 50% “no” choices. We then computed choice probability for each neuron (both those neurons inside the decision pool (solid oval) and those outside the decision pool (dashed oval)).

Figure 9.

Simulations showing that choice probability is similar for neurons involved and not involved in a decision. a. Schematic of simulations. We simulated the responses of a population of 1000 identically tuned neurons with mean response 50 spikes/sec and constant pairwise noise correlation (rectangle). The model’s decision was based on the responses of a fraction of these neurons that comprised the decision pool (solid oval), and the responses of the rest of the neurons were ignored (dashed lines). The solid oval representing the decision pool is small compared to the dashed oval representing the non-decision pool to represent the relative number of neurons in each. The model’s decision in a hypothetical task was “yes” if the mean activity in the decision pool was greater than 50 spikes/sec and “no” otherwise. b. Predicted choice probability for neurons involved (solid lines) and not involved (dashed lines) in a perceptual decision for four uniform levels of noise correlation across the population. Choice probability is plotted as a function of the number of neurons in the decision pool. Choice probability for neurons not involved in the task approaches choice probability for involved neurons for medium to large pool sizes.

We found that when noise correlation was greater than 0, choice probabilities for neurons not involved in the decision (dashed lines in Figure 9b) were statistically indistinguishable from those for neurons in the decision pool (solid lines) for larger pool sizes. Using experimentally feasible numbers (sampling 50 neurons from the decision pool (from the solid oval in Figure 9a) and 50 neurons outside the pool (from the dashed oval) on 100 trials per condition), choice probability within and outside the pool of involved neurons is statistically indistinguishable for all three non-zero correlation levels we tested when pool size is greater than about 20 neurons (t-test, p<0.05). While choice probability may still enable us to identify large pools of neurons that include neurons that contribute to the decision process, choice probability alone does not permit distinctions between individual neurons that are or are not causally linked to downstream decision mechanisms.

Discussion

Our primary goal was to reassess the relationship between neural responses and psychophysical performance on a motion discrimination task, taking advantage of new noise correlation data (Cohen and Newsome, 2008) and new MT electrophysiological data obtained during a reaction time task, which limits the acquisition of neural data to the actual timescale of decision-making. We found that typical MT neurons are two to three times less sensitive to motion direction signals than is the monkey psychophysically. Our data contrast to those of Britten and colleagues (1992) who estimated average neural sensitivity to be equal to psychophysical sensitivity using a fixed duration stimulus, suggesting that the monkeys in their study may not have made use of the full viewing interval, but agree substantially with the conclusions of Cook and Maunsell (2002) who employed a reaction time detection task.

Somewhat surprisingly, our choice probability measurements are similar to those obtained by Britten and colleagues (1996) in the fixed duration task. If our monkeys used a higher proportion of the viewing interval than the monkeys in the original task and if the fluctuations in neural responses are independent across time (little autocorrelation in the responses), then we would expect higher choice probability in our task. If, however, choice probability is due in part to top-down inputs that are present regardless of whether the monkey is actively integrating motion information (Nienborg and Cumming, in review), choice probability should remain high throughout a long, fixed duration trial and should be similar in the fixed duration and reaction time tasks. The data from the original studies show a small amount of autocorrelation (Bair et al, 2001), but not enough to account for the absence of a difference in choice probability between the two studies. The absence of higher choice probability in our data should be a subject for future study.

The observation that most single MT neurons are not sensitive enough to account for the monkey’s behavior and the absence of neurons with the very large choice probabilities (which would be expected if choices were based on the responses of only a few neurons) suggest that the monkey must make use of many MT neurons to make his decisions. The reaction time task provides additional insight into the relationship between neural activity and the decision-making process—the temporal interval required to form perceptual judgment. We found that high firing rates in MT are associated with short response latencies when the monkey makes a choice in the preferred direction of the neuron under study.

Comparison of psychometric performance and neuronal sensitivity

The most substantial difference between our results and those from the fixed duration task is the measurement of neural sensitivity. The relatively low neural sensitivity in our study can be explained by the much shorter viewing interval in the reaction time task as compared to the two-second fixed duration task. The main limitation on the ability of MT neurons to encode stimulus direction is the noise in their responses; integrating neural activity over a longer period of time yields greater estimates of neural sensitivity because some of the noise is averaged out over time. Because trial durations are generally quite short in the reaction time task, estimates of neural sensitivity will necessarily be lower than for trials of longer duration.

The comparison between neural and psychophysical data relies on the assumption that the monkey performed as well as possible given the neural evidence available. Our monkeys generated somewhat worse psychophysical performance than the monkeys in the Britten et al. (1992) study (18% and 26% coherence thresholds for our monkeys; 5%, 12%, and 20% threshold for the monkeys in Britten et al., 1992), so one concern is that our monkeys may not have used as much of the available neural evidence as the monkeys in the original studies. Importantly, however, this difference actually predicts results of opposite sign. If our monkeys did not perform up to their full capabilities, the neurometric: psychometric ratio would be lower than in the original study, whereas we found a higher neurometric: psychometric ratio. If anything, then, neuronal sensitivity is even worse relative to psychophysical sensitivity than we report here.

Noise correlation affects choice probability predictions

Consistent with previous studies, we observed a weak correlation between the trial-to-trial fluctuations in MT responses and the monkey’s choices (Figs. 4, 5 and 7b) as well as the reaction time (Fig. 6). The original pooling model of Shadlen and colleagues (1996) predicted substantially higher choice probability values than were observed in either the current study or the previous study by Britten and colleagues (1996). The discrepancy between the original model and the choice probability data appears to be due, in part, to a faulty assumption about the level of correlated noise present in the population of MT neurons. Although noise correlation measurements had been obtained for adjacent neurons measured on a single electrode (Zohary et al., 1994), measurements were not available for spatially segregated neurons located, for example, in different direction columns. In the absence of data, Shadlen and colleagues (1996) assumed that the responses of neurons in different columns were independent.

We recently assessed this assumption by recording simultaneously from two electrodes in MT (Cohen and Newsome, 2008). For pairs of MT neurons with spatially overlapping receptive fields, we found that noise correlation is positive even if the neurons are located in entirely different direction columns. The positive noise correlation presumably derives from common inputs to neurons with similar receptive fields. Positive noise correlation across the entire pool of sensory neurons reduces the choice probability predicted by the pooling model (Fig. 8b) because it reduces the difference in correlation between the two pools. Intuitively, this is easiest to appreciate by considering two neurons with opposite preferred directions. If the choice probability of both neurons is sufficiently large, positive noise correlation becomes mathematically impossible. High choice probability means that on trials when the monkey chose up, an upward neuron must fire more than its average while a downward neuron fires less than its average; the converse is true when the monkey chooses down. Therefore, the trial-to-trial fluctuations in the two neurons’ responses must be anti-correlated, giving them negative noise correlation. Conversely, if the two neurons have high positive noise correlation, they cannot have high choice probability. We observed moderate but positive noise correlation among oppositely tuned neurons, which caused our model to predict significant but lower choice probabilities than the original pooling model.

A simple pooling model of the reaction time task better accounts for physiological results

Our updated pooling model more closely predicts observed psychometric thresholds and choice probabilities than the original model (Shadlen et al, 1996). The shorter stimulus durations in our task and better noise correlation data (Cohen and Newsome, 2008) improved the model’s predictions. Consistent with previous studies (Purushothaman and Bradley, 2005), our measurements from “off-axis” neurons indicate that neurons whose tuning was not matched to the stimulus under study carried useful information about motion direction (Fig. 7a), and that trial-to-trial fluctuations in the responses of these neurons were correlated with the monkey’s behavior (Fig. 7b). Including these “less sensitive” neurons in the pooling model reduced even further the discrepancy between model predictions and actual data (Fig. 8).

The remaining errors in the updated model are in the same direction as in the original model—lower psychometric threshold and higher choice probability than is actually observed (Fig. 8, compare dashed and colored lines). A possible reason for the remaining discrepancy, as suggested in the original modeling study, is that additional “pooling noise” may occur at the decision stage. By adding an arbitrary amount of pooling noise to our simulations, we could eliminate the remaining error between model and data. Other potential reasons for the remaining discrepancy exist, however, including effects of “motor preparation” time (Mazurek et al, 2003; Kiani et al, 2008) and imprecise estimates of choice probability due to very low firing rates for some neurons. While we have no way to estimate the relative contribution of these factors to our calculations, a combination of the three could certainly account for the remaining difference between model predictions and experimental data.

Concluding remarks

The direction discrimination task we employed is an extensively developed model system for analyzing the neural substrates of perceptual performance. In this report we have rectified the most serious shortcomings of previous studies using this task, most notably the mismatch between the temporal integration interval for psychophysical and neural data and incomplete data on noise correlation. We find that a simple pooling model using these measurements largely accounts for psychophysical performance. The relationship between psychophysical sensitivity and neural sensitivity may be very different in fundamentally different kinds of discrimination tasks (Snowden et al., 1992; Purushothaman and Bradley, 2005) and sensory modalities (Relkin and Pelli, 1987; Matsumora et al., 2008; Stuttgen and Schwarz, 2008). Our results suggest that a complete account of the neural mechanisms underlying psychophysical performance in other systems will require simultaneous recordings from multiple neurons and measurements of neural sensitivity on the timescale of behavior.