Perceptually Bistable Three-Dimensional Figures Evoke High Choice Probabilities in Cortical Area MT

Abstract

The role of the primate middle temporal area (MT) in depth perception was examined by considering the trial-to-trial correlations between neuronal activity and reported depth sensations. A set of moving random dots portrayed a cylinder rotating about its principal axis. In this structure-from-motion stimulus, the direction of rotation is ambiguous and the resulting percept undergoes spontaneous fluctuations. The stimulus can be rendered unambiguous by the addition of binocular disparities. We trained monkeys to report the direction of rotation in a set of these stimuli, one of which had zero disparity. Many disparity-selective neurons in area MT are selective for the direction of rotation defined by disparity. Across repeated presentations of the ambiguous (zero-disparity) stimulus, there was a correlation between neuronal firing and the reported direction of rotation, as found by Bradley et al. (1998). Quantification of this effect using choice probabilities (Britten et al., 1996) allowed us to demonstrate that the correlation cannot be explained by eye movements, behavioral biases, or attention to spatial location. MT neurons therefore appear to be involved in the perceptual decision process. The mean choice probability (0.67) was substantially larger than that reported for MT neurons in a direction discrimination task (Britten et al., 1996). This implies that MT neurons make a different contribution to the two tasks. For the depth task, either the pool of neurons used is smaller or the correlation between neurons in the pool is larger.

Cortical area MT (V5) in the macaque plays an important role in the perception of visual motion and recent evidence suggests a role in the perception of stereo depth. MT contains an ordered map of binocular disparity (DeAngelis and Newsome, 1999) and electrical microstimulation in MT influences the perceptual reports of monkeys in a stereo task (DeAngelis et al., 1998). These studies link stereoscopic depth perception to the activity of small regions of MT, rather than individual neurons.

Pursuing the link between activity and perception at the level of single neurons is more difficult. One approach is to measure simultaneous neuronal and perceptual responses to stimuli that support more than one interpretation. Trial-by-trial correlations between neuronal activity and perceptual reports provide critical evidence that neurons contribute to a visual percept (Parker and Newsome, 1998). Such correlations have been demonstrated for single MT neurons and motion perception (Newsome et al., 1989;Britten et al., 1996), when the sign of the correlation is systematically related to the preferred direction of motion of the neuron, and also in a binocular rivalry paradigm (Logothetis and Schall, 1989).

More recently, a qualitatively similar correlation was demonstrated in a depth order task (Bradley et al., 1998). Monkeys viewed a transparent rotating cylinder, defined by structure-from-motion. This produces a compelling sensation of depth, in which the dots moving in one direction are perceived in front of the dots moving in the opposite direction. Nothing inherent in the stimulus defines the depth order (which set of dots is in front), so the stimulus is ambiguous: the perception is bistable and undergoes spontaneous fluctuations in the perceived direction of rotation (Wallach and O'Connell, 1952; Ullman, 1979).

The stimulus can be rendered unambiguous by adding binocular disparities that define the depth order of the dots. Many MT neurons are selective for depth order in binocular stimuli (Bradley et al., 1995). Bradley et al. (1998) found trial-by-trial correlations of neuronal firing and perceptual reports for perceptually ambiguous cylinders. Although these correlations were related to the selectivity for disparity-defined depth order, neurons that preferred a clockwise (CW) direction of rotation with the unambiguous stimuli did not always increase their firing with the animal's reports of clockwise rotation in the ambiguous stimulus. Their data suggest that MT neurons contribute to the perceptual processing of stereoscopic depth for the cylinder stimuli, but the nature of their contribution remains unclear.

The exact magnitude of the correlation between neuronal firing and perceptual reports is of critical importance for several reasons. First, it allows comparison with the data of Britten et al. (1996), so that the relative contributions of MT neurons in motion and stereo may be assessed. Second, the magnitude of the correlation has important consequences for models of how a perceptual decision is derived from a population of neurons (Shadlen et al., 1996). Third, comparison of the magnitude of the correlation with other experimental observations (e.g., eye movements) allows the possible contribution of a number of potential artifacts to be evaluated. None of these issues can be addressed by examination of the data presented by Bradley et al. (1998)

Therefore, we recorded the activity of single MT neurons to binocular structure-from-motion stimuli and simultaneously gathered perceptual reports from the animals when working near psychophysical threshold. We quantified the trial-by-trial correlation between perceptual report and firing rate using the “choice probability” metric (Celebrini and Newsome, 1994; Britten et al., 1996).

MATERIALS AND METHODS

Subjects. Data were obtained from two male monkeys (Macaca mulatta). Each monkey was implanted (under general anesthesia) with a stainless steel head-restraining device and a scleral magnetic search coil (Judge et al., 1980) in each eye. The animals were trained initially to maintain attentive fixation on a binocularly presented marker and were subsequently trained to perform a psychophysical discrimination task. Behavior was controlled by operant conditioning techniques using fluid as a positive reward. When behavioral performance was satisfactory (see Results), a stainless steel recording chamber was implanted over the occipital cortex and neuronal recording experiments began. Additional behavioral training on the psychophysical task was undertaken throughout the experimental phase if necessary. All of the procedures complied with the United Kingdom Home Office regulations on animal experimentation.

Stimuli. A graphics workstation (Indigo2; Silicon Graphics, Mountain View, CA) provided video signals to two cathode-ray tube monitors [Tektronix GMA 201 in the first half of the experiments (Tektronix, Wilsonville, OR), Eizo FlexScan 78 in the second half (Eizo, Woking, UK)] in a Wheatstone stereoscope configuration. Mean luminance was 188 cd/m² for the Tektronix monitors and 42 cd/m² for the Eizo monitors. The maximum available contrast was 99%, and the frame rate was 72 Hz. The screens were positioned at a distance of 89 cm from the observer. Each screen covered ∼21 × 17° of the visual field. To investigate more eccentric receptive fields (RFs), the fixation marker (subtending 5.88 arcmin) could be moved within this area. Each pixel subtended 0.98 arcmin. Subpixel resolution was achieved by using the built-in hardware anti-aliasing of the graphics workstation. Stereo separation was produced by splitting a three-channel color video signal; the “blue” signal drove the left monitor, and the “red” signal drove the right monitor (although the image on each monitor was black and white). The experimenter viewed an anaglyphic version of the stimulus on a color monitor.

The stimulus used in the experiments was the orthographic projection of dots placed at random locations on the surface of a transparent cylinder rotating about its principal axis (Treue et al., 1991). The dot size was usually 0.25 × 0.25° and the density was usually 25%, although these were occasionally adjusted, depending on the size of the receptive field. The stimulus typically contained ∼80 dots. For each stimulus, the dots were assigned random locations, and the direction of motion of each dot was also randomly chosen (left or right with equal probability). The velocity profile was sinusoidal with the peak velocity at the midline of image. Thus, the velocity of dots increased as they approached the midline and then decreased as they moved toward the lateral edges. When a dot reached the edge, its direction of motion reversed. On each video frame, 2% of the dots were replaced with new dots at random locations and a velocity appropriate to the new location. This resulted in a mean dot lifetime of 615 msec.

The stimulus produces a striking impression of a three-dimensional rotating cylinder. When no disparities are added to the dots, the direction of rotation is ambiguous. For long durations of viewing (several seconds or more), the stimulus is perceptually bistable and spontaneously flips its perceived configuration. For shorter durations (including the 2 sec presentations used here), the stimulus is stable during the presentation but changes from one presentation to the next. We undertook human psychophysical experiments with this stimulus. Observers were asked after the experiment whether they had perceived any changes in percept during the 2 sec trials. None were reported. Horizontal disparity was used to separate the two surfaces in depth and so specify the direction of rotation. Disparity was added so that each moving surface received equal but opposite disparity (Fig.1). Hence, the center of the three-dimensional cylinder corresponded to the fixation plane. The disparity of each individual dot was scaled according to its distance from the midline, in a manner similar to the velocity scaling described above. Thus, the maximum disparity occurred at the midline of the stimulus and decreased toward the edges. This largest (midline) disparity was used to characterize the disparity of the stimulus such that a positive disparity applied a crossed disparity (near depth) to rightward moving dots and an uncrossed disparity (far depth) to leftward moving dots. This was termed counterclockwise (CCW) rotation (as viewed from above). A negative disparity applied a crossed disparity to leftward moving dots and an uncrossed disparity to rightward moving dots. This was termed CW rotation (Fig. 1). The meaning of “disparity” here is somewhat different from that used in many studies. Its magnitude describes the range of disparities present and its sign describes the relationship between disparity and motion. Thus, equal and opposite disparities in this context have identical motion and disparity signals, but the relationship between motion direction and disparity is reversed.

Fig. 1.

The cylinder stimulus consists of two oppositely moving transparent surfaces made up of random dots. At zero disparity, the dots are all imaged at the fixation plane. The velocity of the dots is a sinusoidal function of spatial position, which gives rise to a sensation of depth-from-motion. Because no depth order is specified, the direction of rotation of the cylinder is ambiguous. Adding a binocular disparity to the dots removes this ambiguity. Disparities are scaled sinusoidally according to the position of each dot on the surface of the cylinder. Positive disparities place the rightward moving dots at crossed disparities and the leftward moving dots at uncrossed disparities. This was defined as CCW rotation. The depth order is reversed for negative disparities, which correspond to CW rotation.

The amplitude of the disparity signal was used to control the degree of ambiguity. For large disparities, the direction of rotation is unambiguously defined, and the percept is stable. For both human and monkey observers, large disparities produce nearly perfect performance. As the disparities are reduced, the discrimination between CW and CCW rotation becomes increasingly difficult, and observers perceive the rotation opposite to that defined by the sign of disparity with increasing frequency.

Psychophysical task. Once the monkey fixated on the fixation marker, presentation of the stimulus began. If fixation moved outside a window ±0.5–1.0° from the fixation mark at any time during the stimulus presentation period (usually 2 sec), there was a brief pause before the next trial could start. If fixation was maintained for the duration of the stimulus presentation, both the fixation marker and the stimulus were extinguished and two choice targets were presented, one to the left and one to the right of the former position of the fixation marker. Regardless of the stimulus, the choice targets were always placed in the same location relative to the fixation marker. The monkey was required to indicate his choice of rotation direction by making a saccade to one of the choice targets. If the choice was correct, a fluid reward was given and then the next trial commenced. If the choice was incorrect, a checkered pattern was presented for a brief time before the next trial could start. A correct choice was defined as a saccade consistent with the direction of rotation specified by the disparity added to the cylinder stimulus (Fig. 1): left target for CW rotation and right target for CCW rotation. For the zero-disparity stimulus, the monkey was rewarded, at random, on half of the trials.

Single-unit and eye movement recording. The location of the implanted recording chamber permitted a posterior approach for penetration toward area MT. A stainless steel guide tube was used to penetrate the dura at the start of each recording session. Parylene-coated tungsten microelectrodes (0.3–2.0 MΩ impedance at 1 kHz; Microprobe Inc.) were passed through the guide tubes. The electrodes were advanced manually to the tip of the guide tube and then manipulated with a hydraulic microdrive (Narishige, Tokyo, Japan).

Electrode signals were amplified (Bak Electronics) and filtered (200 Hz to 5 kHz) before being digitized (32 kHz) and stored to disk on a personal computer using the Datawave Discovery system (DataWave Technologies, Minneapolis, MN). This provided a system for on line classification of the spikes. The stored electrode signals were subsequently classified off line using software developed in the laboratory.

The horizontal and vertical positions of both eyes were measured using a magnetic scleral search coil system (CNC Engineering, Camarillo, CA). These data were digitized and sampled at 587 Hz before being stored to disk. In one monkey (Bi), the implanted coil in one of the eyes became unseated after approximately three-quarters of the experiments had been completed. The experiments continued because the quality of fixation and the direction of the choice saccade could still be estimated from the signal coming from the remaining coil, although inevitably vergence data were unavailable for these experiments.

Experimental procedure. After isolating a single unit, the preferred direction of motion was determined qualitatively using a circular patch of moving random dots. The minimum response field was then mapped using a rectangular patch of dots moving in the preferred direction. Quantitative direction tuning functions were then obtained using a circular patch of dots covering the RF. If necessary, the speed of dot motion was adjusted to ensure a vigorous response. Disparity selectivity was then measured, varying the disparity of the dots covering the RF. These were surrounded by an annulus of dots (also moving in the preferred direction) that were always at zero disparity. Finally, the responses to cylinder stimuli with different disparities were measured. The size of the cylinder matched the RF, and the cylinder orientation was set so that the opposing motions within the stimulus ran along the preferred–null motion axis of the neuron. On a small number of occasions (nine) when part of the receptive field lay outside the boundaries of the monitors, the stimulus covered the receptive field as fully as possible.

If the neuron was selective for the depth order specified by disparity, the psychophysical task began. The stimulus parameters were matched to the properties of each neuron under study, so the signals carried by each neuron were potentially relevant to the psychophysical task performed. Psychophysical trials were presented in blocks that consisted of repeated presentations of cylinder stimuli at several different disparity levels (usually five or seven). The range of disparities was centered on zero, with the other levels being simple multiples of a disparity increment (e.g., 0.02, 0.01, 0, −0.01, and −0.02°). Hence, each block contained equal numbers of CW and CCW stimuli and a proportion of fully ambiguous (zero-disparity) stimuli in pseudorandom order. The magnitude of the disparities was chosen to ensure that the animals were working near threshold. This range of disparities was therefore substantially smaller than that used for the initial characterization of disparity selectivity. Over this narrow range of disparities, it is unlikely that the monkey was able to distinguish the zero-disparity trials from the near-threshold disparity trials. In human subjects, these stimuli are not discriminable from those with no disparity (Nawrot and Blake, 1993). As many trials as possible were collected over this range, and data collection typically ceased only when the isolation of the neuron was lost or when the behavioral performance of the animal declined because of satiation. In a few cases in which >50 repetitions of each stimulus had been completed and the animal was not satiated, the electrode was advanced in search of another neuron.

These experiments were usually performed only if the preferred direction of the neuron was within ±45° of horizontal. This ensured that there was no uncertainty in the interpretation of the choice targets (which were always horizontally displaced from the fixation target). In a small number of later experiments (n = 12), the criterion was widened to include neurons with direction preferences within ±75° of horizontal, and no decline in behavioral performance was evident.

Data were only included for analysis from blocks of trials in which the animal's behavioral performance was accurate and the responses to the zero-disparity stimulus were reasonably unbiased. The psychometric function was examined visually for the typical sigmoidal shape (Fig. 2,a, c). The smooth progression in choice proportions suggests that the animals are treating ambiguous stimuli in the same way as those near threshold. Any experiments in which the animal selected one choice on >75% of the zero-disparity trials were also discarded. A lower limit of 15 repetitions of the zero-disparity stimulus was set for the data from a neuron to be included in the final data set.

Fig. 2.

Examples of behavioral and neuronal responses obtained simultaneously. At the top (a, c), there are two psychometric functions, one from each monkey. The corresponding neuronal tuning functions obtained from the same set of trials are at the bottom (b, d). a and bshow data from monkey Bi collected from a total of 191 trials.c and d show data from monkey Mr collected from a total of 470 trials. For the behavioral data, the percentage of choices in the CW direction is plotted as a function of added disparity, and the points are fitted with cumulative Gaussian curves. The threshold was taken as the SD of the fitted curve. Error bars show the SD of the binomial distribution. The neuronal data show the mean firing rate (error bars show the SD) over the 2 sec stimulus presentation plotted as a function of added disparity. The response of each neuron changes monotonically with added disparity. In b, the larger responses occur for negative disparities (CW rotation) as opposed to positive disparities (CCW rotation). Thus, the preferred (PREF) direction of rotation for this neuron was CW. The neuron whose data are illustrated in d showed the opposite preference (CCW).

Confirmation of locations of recorded neurons. Physiological criteria were used to identify area MT. We relied on a characteristic pattern of transitions between white and gray matter for this angle of approach, the fact that a high percentage of neurons were direction selective, nearby neurons and multiunit recordings all showed similar direction preferences and disparity tuning, and on the fact that the pattern of receptive field locations changed with the position of the electrode (within penetrations and between penetrations) in a way that concurred with the known topography (van Essen et al., 1981; Maunsell and van Essen, 1983;Desimone and Ungerleider, 1986; Albright and Desimone, 1987; DeAngelis et al., 1998).

In one of the two animals (Bi), we confirmed that we had been recording in area MT by placing an electrolytic lesion (4 μA for 4 sec) at one of these recording sites. After the animal was given a lethal dose of barbiturate anesthetic (60 mg/kg) and an anticoagulant, it was perfused transcardially with PBS until exsanguination and then fixed with 4% paraformaldehyde. Parasagittal sections (50 μm thick) of the occipital cortex were prepared, and a series of one in five sections was stained for myelin by the Gallyas method to delineate the extent of cortical area MT (van Essen et al., 1981). Other sections were stained with cresyl violet to aid in the identification of the lesion. A lesion was identified along an electrode track, passing through an area on the posterior bank of the superior temporal sulcus in a zone in which the myelination in the lower cortical laminas was noticeably more dense, indicating that the lesion was in cortical area MT.

RESULTS

A total of 301 penetrations were made in three hemispheres from two monkeys. The selectivity of 322 neurons for depth order in the cylinder stimulus was tested. The measurement of choice probability was attempted in 190 of 322 neurons that showed clear disparity selectivity. Of these, 93 (53 from monkey Mr and 40 from monkey Bi) yielded sufficient data to provide at least 15 ambiguous (zero-disparity) trials with satisfactory psychophysical behavior. (Note that this required many more than 15 behavioral trials as a whole because, at most, 20% of the trials were fully ambiguous. See the next section for a discussion of what is meant by satisfactory behavior). The majority of these neurons (84 of 93) had receptive fields that were wholly contained within the visible area of the display monitors, with their centers at eccentricities between 2.8 and 15° (mean ± SD of 7.2 ± 2.2°). The average RF size was 5.8° (calculated as the square root of the area of RF). The need to test the animals' behavior with cylinders whose axis was within the neighborhood of vertical (see Materials and Methods) created a selection bias for direction of motion. The preferred/nonpreferred axis for direction of motion for 81 of 93 neurons was within 45° of horizontal and was distributed evenly between ±45°. For the calculation of choice probabilities, the mean ± SD number of repetitions of the zero-disparity stimulus was 40 ± 16 (range of 16–93).

Behavioral performance

It is critical for studies of this type that the animal's response at the end of each trial represents a reliable indication of what the animal perceived. Good psychophysical performance on those trials in which the direction of rotation is specified by disparity indicates that this is the case, so psychophysical performance is examined first. Figure 2, a and c, shows examples from each monkey of behavioral performance during unit recording. The proportion of CW choices changes smoothly as a function of disparity, indicating that the animals were performing the task reliably. Across all of the data sets from which the zero-disparity trials were used to calculate choice probabilities, the mean percentage of correct responses (across all trials with some disparity) was 82%. This excellent performance was achieved, even though a narrow range of disparities was used. For 79 of 93 experiments, the largest disparity used was ≤0.1°. Considering only this largest disparity in each experiment, the mean percentage of correct responses was 97%. These values indicate that the animals' psychophysical reports were reliable during the neuronal recording.

The behavioral data from each experiment were fitted with cumulative Gaussian curves, using a maximum-likelihood estimator (Watson and Pelli, 1983), and threshold was taken as the SD of the fitted Gaussian. In the examples shown, the thresholds were 0.019 and 0.020° (Fig. 2, a, c, respectively). Across all experiments, the mean ± SD threshold was 0.031 ± 0.026°, and there was a systematic increase in thresholds with stimulus eccentricity. When tested with similar stimuli, thresholds were comparable for both monkey and human observers.

This psychophysical performance appears to be substantially better than that obtained by Bradley et al. (1998). Over a comparable range of eccentricities (<8.3°), the mean threshold in the present study is 0.023°. Although Bradley et al. (1998) do not report threshold values, their data summary shows 75% correct responses overall for a disparity ∼0.2°, implying much larger thresholds than we have found. Even for the largest disparity (0.4°), the overall performance of their animals was only 85% correct, again much poorer than that of our animals.

Neuronal tuning

The range of disparities used was typically small to make the animals work near threshold. The majority of neurons (87 of 93) were so well tuned to disparity that they showed significant modulation of firing over this narrow range (p < 0.05 with a one-way ANOVA). These data gave an immediate, definitive assignment of the preferred direction of rotation of each neuron. For the remaining six neurons, the preferred direction of rotation was determined from the neuronal responses to coarser disparities during the initial measurement of selectivity for depth order. In these cases, the tuning to coarse disparities was sufficiently strong that response distributions for opposite rotation directions at the largest disparity were nonoverlapping. Again, this gives a definitive assignment of the preferred direction of rotation of each neuron.

Within the set of 93 neurons, 45 (26 from monkey Mr and 19 from monkey Bi) were selective for CW rotation, like the example in Figure2b. A total of 48 neurons (27 Mr and 21 Bi) were selective for CCW rotation, like the example in Figure 2d. The majority of neurons, like these two examples, had firing rates that were monotonic functions of disparity (over the narrow range of disparities that were used) and appeared antisymmetric about zero disparity.

Thus, these neurons all consistently signaled the direction of rotation of the cylinder stimulus when its rotation was defined unambiguously by disparity. This allows a simple prediction to be made about the activity in the zero-disparity trials. If these neurons are involved in perception of the cylinder, we would predict that, for neurons with CW preference, there should be a greater mean response on trials when CW rotation is reported by the monkey compared with trials when CCW rotation is reported. For a neuron with CCW preference, the opposite prediction holds.

Analysis of trial-by-trial correlation between neuronal and behavioral response

The main analysis examined only those trials that presented the zero-disparity stimulus. Figure 3 shows, for the experiment illustrated in Figure 2a, b, the neuronal response on each zero-disparity trial. The trials in which the monkey chose CW [the preferred (PREF) rotation of the neuron] are shown by the filled symbols, and the trials in which the monkey chose CCW (the NULL rotation of the neuron) are shown as open symbols. For this experiment, the monkey chose CW and CCW with about equal frequency. Overall, the neuron fired more spikes on the CW choice trials compared with the CCW choice trials, although the disparity of the visual stimulus was identical and zero for all these trials.

Fig. 3.

 Analysis of the trial-by-trial correlation between neuronal response and behavioral choice for the experiment illustrated in the left panels of Figure 2. The scatterplot in the middle shows the neuronal response plotted against trial number for each zero-disparity (ambiguous) trial. Thefilled symbols represent the trials on which the monkey chose CW (PREF) rotation, and the open symbols represent those trials on which the monkey chose CCW (NULL) rotation. These data are summarized by the histograms on the left, which show the distribution of neuronal firing rates by choice. The separation of the two distributions can be quantified by the application of a signal detection theory to produce a measure of choice probability. The plot at the right shows, for a range of criterion response levels, the proportion of the PREF trials that exceeded the criterion (ordinate) against the proportion of NULL trials that exceeded the same criterion (abscissa). The area under this curve gives a nonparametric, criterion-free, measure of the separation of the distributions, the choice probability. For this example, the choice probability was 0.79; from the spike counts of this neuron alone, the probability of correctly predicting the animal's response on each zero-disparity trial is 0.79. This choice probability is significantly different from 0.5, which is the value that would be expected if the association between neuronal response and behavioral choice were random (permutation test; p < 0.05). CP, Choice probability.

The histogram shows the extent of overlap between the distributions of firing rates. The separation between these distributions indicates the degree of covariation between behavioral choice and neuronal response: the greater the separation, the greater the covariation. The degree of separation was assessed quantitatively by a nonparametric analysis based on signal-detection methods to produce a metric termed the choice probability [described by Britten et al. (1996)]. The choice probability is the area under the curve produced by plotting, for a range of criterion response levels, the proportion of the PREF trials that exceed the criterion (ordinate) against the proportion of NULL trials that exceed the same criterion (abscissa). This is shown for the example experiment in the right panel of Figure 3, for which the choice probability was 0.79.

The choice probability can range from 0.0 to 1.0. It indicates the probability with which an ideal observer can correctly predict the animal's choice based on the firing rate of the neuron. Values >0.5 indicate a positive correlation between firing rate and choice; values <0.5 indicate a negative correlation. Because the experimental paradigm requires the animal to make one of two choices, a value of 0.5 indicates no correlation.

The statistical significance of each experimental estimate of choice probability was assessed using a permutation test (Britten et al., 1996). The range of choice probabilities expected by chance was determined by calculating new choice probability values from the data. The actual spike counts observed and the number of CW and CCW choices were maintained, but these values were paired randomly for each calculation of a permuted choice probability. A distribution of 4000 permuted choice probability values was generated, representing the distribution of choice probabilities that would have been expected to occur by a chance association between choice and neuronal firing. If an observed choice probability from an experiment lay outside of the central 95% of the values in its own permutation distribution, then it was considered to be significantly different from 0.5 (i.e., two-tailed test; p <0.05). For example in Figure 3, the value of the choice probability was significantly different from 0.5.

Figure 4 shows the calculation of choice probabilities for two additional examples. Figure 4a shows a choice probability that was close to the population mean but not significantly >0.5. Figure 4b shows a choice probability close to 0.5.

Fig. 4.

Choice probabilities for two more neurons. For each experiment, the two smaller graphs on theleft show the psychophysical performance (top) and the neuronal responses (bottom) as functions of cylinder disparity; the scatterplot shows the trial-by-trial neuronal response for the subset of zero-disparity trials, labeled according to behavioral choice (filled, CW; open, CCW); the graphs on the right show the curve constructed for calculation of the choice probability. The neuron illustrated at the top (a) preferred CW rotation. Its choice probability was 0.66, close to the average for the whole population but not significantly different from 0.5. The neuron at thebottom (b) also preferred CW rotation but had a choice probability close to that expected by chance. In this case, there appears to have been no link between variations in neuronal firing and the monkey's perceptual choice.

The choice probabilities for all 93 neurons are summarized in Figure5. The distribution is clearly biased toward values >0.5 (positive correlation). The mean choice probability was 0.67 (different from 0.5; t test; p < 0.001), and 77 of 93 values were >0.5. This choice probability is considerably greater than the average choice probability found for direction discrimination tasks in MT (0.555) (Britten et al., 1996) or medial superior temporal area (MST) (0.594) (Celebrini and Newsome, 1994). Possible reasons for this are considered in Discussion.

Fig. 5.

Distribution of choice probabilities. This histogram of choice probabilities summarizes the results from 93 MT neurons, all of which were selective for the disparity-defined direction of rotation of the cylinder stimulus. The mean choice probability was 0.67, with a range of 0.35–0.98. Filled bars delineate the neurons with a choice probability significantly different from 0.5 (40 of 93 neurons; permutation test;p < 0.05). Every choice probability that is statistically significant is >0.5 (i.e., the correlation is in accordance with the stimulus preference of the neuron for direction of rotation).

This large mean choice probability also resulted in a large number of choice probabilities that were significant for individual neurons. A total of 40 of 93 neurons had choice probabilities that were significantly different from 0.5. All of these values were >0.5; there were no cases in which the choice probability was significantly less than 0.5 (negative correlation). This is in contrast to all previous studies of this type, in which significant negative correlations have also been found in addition to significant positive correlations (Logothetis and Schall, 1989; Celebrini and Newsome, 1994; Britten et al., 1996;Leopold and Logothetis, 1996; Bradley et al., 1998).

The most startling difference is between our data and those ofBradley et al. (1998), who found a substantial number of negative correlations. This discrepancy is puzzling because the stimulus and task that they used was extremely similar to the one used here. One difference between these studies is that our analysis was limited only to the zero-disparity stimulus, whereas their analysis pooled responses over several disparity conditions. Therefore, we examined the choice probability on trials in which a non-zero disparity was present. Of course, the presence of a non-zero disparity defines an unambiguous direction of rotation. Choice probability can only be calculated in these circumstances if the animal's responses to a given stimulus are not all in one direction. Furthermore, in cases in which the majority of responses are in one direction, the confidence interval for the choice probability is very wide. (If the animal only makes one “mistake,” then the value of the choice probability is effectively determined by the firing rate on that single trial.). Nonetheless, we calculated choice probabilities for every stimulus condition in which the animal made at least one incorrect response. These values of choice probability are shown on the abscissa of Figure6a, and the ordinate shows the percentage of trials for which the monkey chose the preferred direction of rotation of the neuron. The filled symbols indicate choice probabilities that were significantly different from 0.5 (p < 0.05). The solid linesuperimposed on the scatterplot shows the running mean (sliding boxcar average) of 20 adjacent values of choice probability.

Fig. 6.

Choice probability across all stimulus disparity conditions for which the monkey made at least one mistake (n = 489). The scatterplot at thetop (a) shows the choice probability on the abscissa and the percentage of choices made in the preferred direction of each neuron on the ordinate. The jagged line indicates the running mean of 20 adjacent values of choice probability. The data are summarized in the histogram at the bottom(b). The mean of the distribution was 0.613. Ina, the choices were mostly of one type at the extremes, so this produces less reliable and more scattered estimates of choice probability, which biases the mean choice probability toward 0.5 in these regions. If the analysis of mean choice probability is restricted to conditions for which the animals' choices contained at least 20% of each type of response, then the mean choice probability was 0.643, close to the value obtained for zero-disparity trials. Thefilled symbols in a and the filled histogram bars in b show choice probability values that were significantly different from 0.5 (103 of 489 cases; permutation test; p < 0.05). A total of 98 of these 103 significant choice probabilities were >0.5, and only 5 of 103 were <0.5.

The data from these unambiguous trials are very similar to those from the zero-disparity trials alone: the majority of choice probabilities are >0.5 regardless of the stimulus condition. Furthermore, the mean value of the choice probability for the unambiguous stimuli that produced CW and CCW responses with nearly equal frequency was 0.613, similar to the value of choice probability for the zero-disparity case.

At the top and bottom of the ordinate of Figure 6a, the psychophysical responses are nearly all in one direction and the measurement of choice probability becomes increasingly unreliable, as reflected in the increasing spread of values. Apart from this statistical inevitability, there does not appear to be any systematic relationship between the performance of the monkey and the choice probability, in agreement with the findings of Britten et al. (1996). When the relative proportion of the CW and CCW choices was smaller than 4:1, the choice probability measures were reasonably reliable. Within this more restricted range, the distribution of choice probabilities is now very similar to that for the zero-disparity case, with a mean ± SD choice probability of 0.64 ± 0.15. Thus, the results obtained by including all stimulus conditions for which it was possible to measure a choice probability are in agreement with the results estimated solely from the zero-disparity trials. Although there are a few (5 of 489) significant negative correlations in Figure 6, this number is smaller than that expected by chance (24 of 489, 5%). These correlations all occurred with stimulus conditions that produced only a small number of mistakes, so it is extremely unlikely that the significant negative correlation would remain if such data were combined with other stimulus conditions for the same neuron.

This analysis indicates that the inclusion of nonambiguous stimuli cannot explain why Bradley et al. (1998) found so many significant negative correlations. Two likely explanations remain. One is that the animals' reports in their study may not have been reliable, as indicated by the relatively poor psychophysical performance of those animals (discussed above). The second arises from the fact that Bradley et al. (1998) did not calculate a choice probability, but they performed t tests on the firing rate distributions. They performed multiple t tests for each neuron (one for each stimulus condition), but made no adjustment to the significance criterion to reflect these multiple comparisons. So the negative correlations reported as significant in that study may not in fact have been significant when considered in the context of the complete data set.

The large choice probability for zero-disparity stimuli shows that the activity of these neurons was strongly linked to the monkey's choice, even though the physical stimulus was equivalent in all the trials. The most appealing interpretation of these data is that these neurons play an intimate role in the perceptual decision process. Before drawing this conclusion, it is necessary to consider other factors that might have separately influenced both the neuronal activity and the animals' choice. These factors might explain the covariation between the neuronal response and choice through their joint association with a third parameter. We consider the possible influences of (1) subtle changes in the visual stimulus, (2) eye movements, and (3) the influence of stimulation history.

Stimulus-induced covariation

The positions of the dots making up the cylinder stimulus were set at random and, for the majority of experiments, were completely different on each trial. It is possible that the arrangement of certain patterns of dots could have caused both a change in the firing of the neuron and a particular percept to be reported. This would have led to a correlation between neuronal response and behavioral choice.

For the cylinder stimulus, this seems very unlikely. No noise was added to either the motion signals or the disparity signals, so there is no reason why one zero-disparity stimulus should produce a stronger response than any other. The only difference between trials was in the locations of the dots, which were randomly assigned new starting positions on each trial. The stimulus preferences of MT neurons are not affected by dot position (Albright, 1992). Even if some neurons were sensitive to dot location, it is hard to see why a particular dot configuration should also influence the perceptual choice only in the appropriate direction for that particular neuron. Thus, the possibility that a stimulus-related variation might be responsible for contributing to the choice probability seems much more remote than for the direction discrimination experiments ofBritten et al. (1996), in which random fluctuations in the motion energy within the stimulus might conceivably cause both firing and percept to associate appropriately. Nonetheless, to confirm that the stimulus-induced variation was unimportant, six experiments were conducted in which the pattern of dots forming the stimulus was identical on each zero-disparity trial. The mean choice probability for this subset of experiments was 0.67, which is in excellent agreement with the result over the whole population.

As an additional control, we also performed measurements of choice probability on seven neurons that were not selective for the direction of rotation of the cylinder (when defined by disparity). Their responses therefore did not differentiate the two directions of cylinder rotation. Because there was no preferred direction of rotation, only the magnitude of the difference from the choice probability expected by chance (0.5) is meaningful. Across the seven neurons, the maximum difference from 0.5 was 0.088, the minimum was 0.002, and none of these differences was significant. Although the sample is small, it is relevant to note that the average choice probability for this set of seven neurons was very close to 0.5. Thus, the activity of these seven neurons appears unrelated to the perceptual choice of the monkey and they may be part of a distinct set of MT neurons not involved in the perceptual decisions studied here.

Eye movements

Eye movements are a potential source of variability in neuronal response measurements when recording from the visual cortex of alert animals. If the eye movement behavior depends systematically on the perceived stimulus configuration, it is possible that the eye movements might have different effect on the neuronal response for the two configurations. If this occurred, the choice probabilities measured could simply reflect the choice-related eye movement and not the percept itself. Therefore, we examined both the direction of microsaccades and vergence eye movements as a function of behavioral choice.

Microsaccades

Small fixational eye movements (microsaccades) have been shown to modulate the activity of MT neurons (Bair and O'Keefe, 1998), so the distributions of these movements were examined to assess whether they were related to choice. Saccades within trials were detected by measuring the speed at which conjugate eye position changed. Speed was defined as $\sqrt{{\dot{\upsilon }}^2+{\dot{\text{h}}}^2}$, whereυ˙ and h˙ are the magnitudes of the vertical and horizontal conjugate eye velocity components, respectively. If this exceeded 10°/sec, a saccade was deemed to have occurred, and its magnitude and direction of the movement were calculated. These microsaccades were than examined separately for CW and CCW choice trials.

Microsaccades were found to occur in all directions, with a very slight bias toward the location of the stimulus. However, there was no consistent difference between the distributions of microsaccades occurring on the different choice trials. This as confirmed by calculating the vector average of all the saccades for each condition. In each case, the vector average was small compared with the mean of the magnitudes of the individual microsaccades (mean ratio across all conditions = 0.1), reflecting the lack of consistent direction for the microsaccades. For each experiment, the difference in the direction of the vector average between the two different choice trials as also small (mean of 26 °), reflecting the lack of choice-related differences. This indicates that microsaccades could not have had a substantial influence on the choice probability measures.

Vergence

All of the neurons studied were selective for binocular disparity; therefore, their responses might be affected by changes in the depth of fixation (vergence eye movement). It is clear that the effect of vergence is more serious for neurons that are primarily response to absolute, rather than relative, disparity (Cumming and Parker, 1999), although there is no firm evidence either way on this point for the population of MT neurons. If vergence eye movements were linked to the behavioral choice, this might trivially explain the choice probabilities measured. For each experiment, the mean vergence angle for each zero-disparity trial was calculated. An overall mean vergence angle was then calculated separately for the trials followed by CW responses and for those followed by CCW responses. The results are displayed in Figure 7, which shows the difference between the mean vergence angle for the two choices (CW minus CCW) plotted against choice probability. A separate plot is shown for each monkey, and the filled symbols show vergence differences that were found to be significant (t test; p < 0.05). Positive angles corresponded to divergent eye movements for CW choices.

Fig. 7.

Difference of mean vergence angle between the two choices (CW minus CCW) plotted against choice probability. These data are for the zero-disparity condition only. Filled symbolsshow vergence differences that were found to be significant (t test; p < 0.05). Positive values indicate that the mean vergence position was more divergent on CW compared with CCW choice trials. Vergence data were unavailable for 12 of the experiments involving Bi (therefore, n = 28 for this animal). For animal Mr (left), the difference in mean vergence angle for CW and CCW choices was close to zero in the majority of experiments. Only animal Bi (right) showed differences in vergence behavior for the two choices. However, no correlation was observed between vergence behavior and choice probability for either animal.

Monkey Mr showed no consistent difference in vergence with choice, whereas monkey Bi tended to be more converged on the trials for which he made a CCW response. It is as if he converged at the apparent depth of the rightward moving dots. These changes in vergence could have influenced the measured choice probability. Note however that there is no relationship between the size and direction of the vergence eye movement and the magnitude of the choice probability.

An additional analysis also strongly suggests that vergence eye movements contributed little to the estimate of mean choice probability. Whether a vergence eye movement causes an increase or a decrease in firing rate depends solely on the disparity selectivity of the neuron. However, the preference of the neuron for direction of cylinder rotation depends on both the disparity selectivity and its preferred motion direction. This is illustrated in Figure8 by considering two neurons that are selective for near disparities. If the two neurons have opposite direction preferences, they will have opposite preferences for the direction of cylinder rotation. A divergence eye movement will produce an increase in firing for both neurons (because it places the stimulus at near disparity). If the divergence is associated with a CW choice, then an increase in firing after the eye movement will produce a choice probability of >0.5 for the neuron that signals CW rotation. However, the same eye movement will produce a choice probability of <0.5 for the neuron that signals CCW rotation. Thus, a vergence eye movement that is related to choice would induce a spurious choice probability that would be artificially >0.5 for neurons with one direction preference and artificially <0.5 for neurons with the opposite direction preference.

Fig. 8.

Possible effect of a vergence behavior that is associated with behavioral choice. The arrows indicate both the disparity and direction preferences of two hypothetical neurons.Left, A neuron preferring motion to the left and near disparities, hence preferring CW rotation. Right, A neuron preferring motion to the right and near disparities, hence preferring CCW rotation. The middle of the figure describes a vergence behavior associated with behavioral choice for zero-disparity trials. In this example, the animal diverges before CW choices and converges before CCW choices (the behavior exhibited by monkey Bi). Therefore, the stimulus is placed at near disparities on trials when a CW choice is made, increasing the firing rate of both neurons. For the neuron plotted at the left, this increase is associated with choices in the preferred direction of the neuron. Hence, the vergence movement leads to a choice probability of >0.5. For the neuron plotted at the right, which prefers CCW rotation, this increase in firing for CW choices leads to a choice probability of <0.5.CP, Choice probability.

Our sample contained approximately equal numbers of neurons with direction preferences to the left (46 neurons) or right (47 neurons), so vergence movements could not explain the overall mean choice probability in this population. Figure 9shows the choice probabilities separately by the preferred direction of motion of the neuron for each monkey. The distributions are similar, indicating that the vergence behavior had very little influence on the measured choice probability.

Fig. 9.

Distribution of choice probability according to the preferred direction of motion, shown separately for each monkey.Filled bars indicate neurons that preferred motion to theleft; open bars indicate neurons that preferred motion to the right. For monkey Bi, the mean values of choice probabilities were 0.648 ± 0.122 (±1 SD; n = 16) for neurons preferring leftward movement and 0.696 ± 0.135 (±1 SD; n = 24) for neurons preferring rightward movement. For monkey Mr, the mean values of choice probabilities were 0.684 ± 0.129 (±1 SD; n = 30) for neurons preferring leftward movement and 0.632 ± 0.148 (±1 SD; n = 23) for neurons preferring rightward movement. There was no significant difference between the distribution of choice probability values between the two motion preferences or between monkeys (χ²test; p > 0.05). Thus, the choice probabilities measured in monkey Bi cannot be explained simply by its vergence behavior.

The same analysis is also useful in excluding any other artifacts that derive from increased responses to any one depth plane. Suppose that the animals attended to whichever surface appeared in front and that this focus of attention induced greater response rates in those neurons selective for the attended depth plane. Following the logic of Figure8, this should produce choice probabilities of >0.5 for neurons preferring one direction and of <0.5 for neurons preferring the opposite direction. The fact that our sample contained similar numbers for each motion direction, combined with the fact that the choice probability was similar for both groups, argues against any explanation based on attention to just one of the depth planes of the stimulus. A similar argument can be made to show that the data cannot be explained by selective attention to any one direction of dot motion (Treue and Maunsell, 1996).

To examine the relationship between the vergence changes and behavioral choice in monkey Bi, an analysis of time course was performed. Figure10 shows, for monkey Bi, the average time course of the vergence behavior associated with each of the choices: the movement appears to start ≈200–300 msec after stimulus onset. Inspection of individual trials showed substantially faster vergence changes, but the time at which these occurred was variable, giving rise to that gradual change seen in the average. The timing of the vergence eye movement suggest that the eye movement is the result of the perceptual interpretation rather than its cause. This interpretation is supported by examination of vergence movements to unambiguous stimuli. We constructed averages such as those in Figure 10for unambiguous stimuli with non-zero disparity and opposite directions of rotation. For these stimuli, we can be confident that vergence changes do not influence the perceptual interpretation (because the perceptual response is determined by the stimulus). The time course of the vergence movement was very similar to that shown in Figure 10.

Fig. 10.

Time course of the vergence movements made by monkey Bi in zero-disparity trials. The dotted anddashed lines show the averages for trials associated with CCW and CW choices, respectively. The solid line shows the difference between these two. Monkey Bi tended to become more converged during trials at the end of which a CCW choice was made.

Previous history of stimulation

During the psychophysical task, the trials containing the zero-disparity stimulus were randomly interleaved with trials containing added disparities. Thus, although the visual stimulation on each zero-disparity trial was identical, the recent history of visual stimulation, response, and reward was not. If the preceding trials had independent effects on both the neuronal and behavioral responses, it might explain a covariation between choice and neuronal activity. Therefore, we examined the relationship between choice on ambiguous trials and events on the preceding trial. Because the animals performed with high accuracy, the great majority of ambiguous trials were preceded by an unambiguous trial in which the animal's response was correct. Therefore, we simplified the analysis by considering only those zero-disparity trials that were immediately preceded by a correct response to an unambiguous stimulus. Only data from neurons that had at least five repetitions for each previous trial condition were included. Figure 11shows that one monkey (Bi) had a tendency to choose the direction of rotation that had been presented on the preceding trial and for which he had been rewarded (a “win-stay” strategy). Note that this is the opposite of the result expected if neuronal activity on the preceding stimulus were to result in sensory adaptation and suppress the response on the current ambiguous trial [of the type shown for similar stimuli by Nawrot and Blake (1989)]. Monkey Mr shows a much weaker tendency in the opposite direction (more points above than below the identity line).

Fig. 11.

Behavioral bias induced by the stimulus in the preceding trial. Only preceding trials in which the animal made correct responses are included. The abscissa shows the percentage of choices in the preferred direction of rotation (PREF) of the neuron following stimuli that were rotating in the PREF direction of the neuron (PREF_n‖PREF_n−1). The ordinate shows the percentage of choices in the PREF direction following of the neuron following stimuli rotating in the nonpreferred (NULL) direction of the neuron (PREF_n‖NULL_n−1). Thediagonal line in each plot shows the identity line. Points farther away from the identity line indicate a stronger behavioral bias that was associated with the direction of rotation presented on the previous correct trial. The graphon the left shows the data from monkey Mr; there is a slight preponderance of data points above the identity line, indicating a weak tendency to choose the direction of rotation opposite from that of the preceding correct trial. Conversely, Bi (right) more often repeated the choice of the preceding correct trial. In both cases,filled symbols indicate cases in which the choice probability was statistically significant for the individual neuron, and open symbols show cases in which it was not.

To estimate a value for the choice probability that is unaffected by these behavioral biases, we split the data into two groups for each neuron based on the stimulus used in the preceding trial. The choice probability was then calculated separately for each group. If the effect of the preceding trial made an important contribution to the choice probability, then the choice probability within these groups should be smaller than the choice probability calculated from all the trials together. Figure 12 plots the weighted mean of the two within-group choice probabilities against the overall choice probability; a good agreement is evident, particularly for neurons with larger overall choice probabilities. Thus, the behavioral bias does not seem to contribute significantly to the measure of choice probability. A similar conclusion was reached concerning the activity of neurons in lateral intraparietal cortex (LIP) (Seideman, 1998; Seideman et al., 1998).

Fig. 12.

The influence of stimulation on the preceding trial on the choice probability. For all experiments in which the choice probability was significantly >0.5, the choice probability was calculated separately for trials that followed a preferred stimulus (pCP) and for trials that followed a null stimulus (nCP). If the difference between these groups was largely responsible for the choice probability, then the choice probability calculated within the groups should be substantially smaller than the overall choice probability. The choice probability calculated across all trials is plotted on the abscissa, and the weighted mean of the choice probabilities calculated from the two groups is shown on the ordinate. Most of the data points can be found along the identity line, so there is a good agreement between the two measures.

Time course of the neuronal response

The major finding of this study is a choice probability for the rotating cylinder that is substantially larger than reported previously in the same brain area using a direction discrimination task (Britten et al., 1996). This large choice probability cannot be attributed to stimulus-induced changes in firing, eye movements (conjugate or disconjugate), attention to a particular three-dimensional spatial location, or biases induced by preceding stimulation. It seems that the choice probability reflects an involvement of the MT neurons in the decision process. To explore this involvement in more detail, we examined the time course of the response.

For this analysis, we examined only those individual neurons for which a significant choice probability had been measured. Recall that, for all of these cases, the measured choice probability was >0.5. For each neuron, a peristimulus time histogram was constructed separately for the choices corresponding to the preferred direction of rotation (PREF) of the neuron and the choices corresponding to the nonpreferred direction of rotation (NULL) of the neuron. A variety of bin widths and/or filtering kernels was applied to the neuronal firing, all producing equivalent results. The results presented in Figure13 are for simple 20 msec bin widths, which offer a balance between temporal resolution and noise. To combine data across neurons with different firing rates, each pair of histograms was normalized by the peak of the PREF histogram; next, the average PREF and NULL histograms across all neurons were computed.

Fig. 13.

Time course of the neuronal responses to zero-disparity stimuli across all neurons with a significant choice probability. Top, Averaged normalized histograms (bin width of 20 msec) for choices corresponding to the preferred (PREF; solid line) and nonpreferred (NULL; dotted line) direction of rotation of the neurons. Bottom, Difference between the two histograms (PREF−NULL). The straight line shows the result of a linear least-squares regression for the period between 0.2 and 2.0 sec. This line has a slope of 0.02 normalized response units, which is significantly different from a line of zero slope (t test; p < 0.01).CP, Choice probability.

Figure 13 shows the average time course across 36 neurons that showed a significant choice probability (mean of 0.78). (The data from another four neurons with significant choice probabilities were excluded because these were collected using a stimulus duration of only 1.7 sec.) The bottom panel of Figure 13 shows the difference (PREF − NULL) between the two histograms. The difference in neuronal response between the two sets of trials appears after ∼100 msec and then remains present throughout the rest of the stimulus presentation period. Linear regression analysis, applied to data from 200 msec onward (to avoid biasing toward the lower differences occurring early on) produced a line of slope of 0.020, which was significantly different from a slope of zero (t test;p < 0.05). This indicates that the difference increases as the trial progresses. When the data were separated by monkey and reanalyzed, the results both resembled Figure 13, although the increase of the difference in firing rate with time was more pronounced for the data from Bi. Because monkey Bi showed choice-related vergence (Fig. 7) with a similar time course (Fig. 10), we examined whether this might explain the more pronounced increase with time in this animal. As before, the increase with time should be seen only for neurons with preferred directions to the left of vertical. For neurons preferring rightward motion, the vergence movement should reduce the difference. The time course for the two groups was similar (data not shown), and both showed significant positive slopes of the firing rate difference with time. This indicates that changes in vergence are not responsible for the increase in response difference over time shown in Figure 13.

Britten et al. (1996) and Celebrini and Newsome (1994) performed a similar analysis on the data from their direction discrimination experiments. In both cases, the onset of the response had a similar time course to that shown above. The difference in the firing rate with time did not increase for the data from MT neurons but did show a modest increase for the data from MST neurons.Britten et al. (1996) argued that the early onset of the response difference and its steady value throughout the trial were consistent with a purely feedforward sensory relationship. Suppose the response difference was the result of feedback reflecting the animal's intended response. On any trial for which the animal's intended responses changed during the course of the trial, this feedback signal would initially have been in the opposite direction, reducing the choice probability for the early part of the trial. It is unclear whether the same argument applies to our stimulus, however, because the perceived direction of rotation tends to remain constant for the duration of a 2 sec trial. On the other hand, there is some evidence that the integration of surface information improves over durations of this length (Treue et al., 1991). Note that a response difference that increases with time could be consistent with either feedforward or feedback processes (Celebrini and Newsome, 1994). If the most recent activity of the neuron contributes more to the decision than earlier activity, even a feedforward process will show an increasing response separation with time. These analyses of time course do not unequivocally demonstrate the causal direction of the link between firing rate and choice.

DISCUSSION

There is a substantial trial-by-trial correlation between the rate of firing of single neurons in MT and perceptual judgements of binocular depth. The judgement required the assignment of the direction of rotation of a cylinder defined by structure-from-motion. When the disparity of the dots in the cylinder was zero, the stimulus had two perceptual interpretations. Both gave rise to a compelling sensation of rotation, but the directions of perceived rotation varied from one trial to the next.

Our results are qualitatively similar to those of Bradley et al. (1998), but we extend those observations in several important ways. First, the correlation was present in an analysis restricted to trials involving a completely ambiguous (zero-disparity) stimulus. Second, the correlation was found when the animals were working close to psychophysical threshold. Third, the correlation was not attributable to stimulus-induced variation, eye movements, behavioral biases, sensory adaptation, or attention to a spatial location. Finally, and importantly, we measured the magnitude of the correlation by calculating choice probabilities (Britten et al., 1996) and found that these were substantially larger than those reported previously in area MT for a different stimulus and task (Britten et al., 1996). All choice probabilities that were significantly different from chance were in the appropriate direction (determined by the selectivity of the neuron for unambiguous, disparity-defined rotation). In contrast, Bradley et al. (1998) found that 7 of 34 significant correlations were in the inappropriate direction. This difference probably reflects a flaw in their statistical analysis (no correction for the use of multiplet tests).

The choice probabilities here are larger than those found byBritten et al. (1996), probably because of the stimulus or the animals' task rather than the recording methods or the analytical approach, which were almost identical. (By itself, a larger choice probability does not necessarily imply that a neuron is more directly involved in perception; it could still be the by-product of the activation of other neurons that do take part in the decision.) A major difference of the two studies is the nature of the stimulus ambiguity. In the studies by Britten et al. (1996), the stimulus consisted of random dots moving in all directions. Although that stimulus has no net motion signal, the psychophysical task requires the subjects to choose between only two alternatives for the perceived direction of motion. Subjects are able to make such decisions, but for human observers, the stimulus actually appears as dots moving in all directions. Thus, in the forced-choice task, stimuli that give rise to one reported direction of motion appear very similar to the stimuli that give rise to exactly the opposite report. Conversely, with the cylinder stimulus, the percept is unambiguously one direction of rotation or the other, corresponding to very different stimulus appearances. Some form of “winner-take-all” network could both deliver the perceptual decision and induce the larger choice probabilities measured here, either directly in MT or indirectly through the activity at another cortical site.

It is possible, of course, that other factors contribute to the differences between the studies. Individual dots maintained a fixed trajectory for much longer periods in our stimulus than in that ofBritten et al. (1996). Perhaps the persistence of individual dots is able to activate a positive feedback loop that magnifies small differences. However, if this were the only explanation, then one would expect the difference in firing rate to grow more dramatically over time than shown in Figure 13.

Another important difference between the two studies is the nature of the animals' task, Britten et al. (1996) used a direction judgement task, whereas our task is a stereoscopic depth task, performed on moving dots. It may simply be that choice probabilities are larger for stereopsis than for motion in MT neurons. This might occur if these neurons are more closely linked to perceptual decisions about binocular depth than direction of motion. Classically, of course, the response of MT neurons has been associated with the perception of motion rather than binocular depth. These large choice probabilities in a binocular depth task, as well as the effects of electrical microstimulation (De Angelis et al., 1998), suggest that the perceptual role of MT is broader than previously assumed.

Regardless of what gives rise to the large choice probabilities, they have important implications for our understanding of how the information from these neurons is used in forming a perceptual decision. If one assumes that the pooled activity of a population of independent neurons determines the perceptual choice, the number of neurons in that population determines the size of the measured choice probabilities (Shadlen et al., 1996); the smaller the pool of neurons, the higher the choice probabilities. Thus, one interpretation of the larger choice probabilities that we observed, is that the pool of neurons for making judgements about the direction of cylinder rotation is smaller than the one for direction of motion.

If the neurons in the pool are not independent, a different possibility arises. If the trial-by-trial fluctuations in spike count are correlated between neurons, then the extent of this correlation is closely related to the choice probability. Greater correlation between neurons allows a large pool size to be reconciled with a given choice probability. The interneuronal correlation was measured for MT neurons by Zohary et al. (1994) with their motion stimulus. They found a mean correlation coefficient of 0.18 between neurons with similar direction preferences. If the interneuronal correlations were similar for the population of neurons that we have studied, this would imply that a pool of approximately six neurons [Shadlen et al. (1996), their Fig. 3] is determining the animals' responses to the cylinder stimuli. This number seems implausibly small. If so few neurons determine the animals' choices, then they should be encountered only rarely during recording, which was not the case.

The data could be reconciled with a larger pool size if the interneuronal correlation is higher in the neurons studied here than the correlation reported by Zohary et al. (1994). Because we only measured the activity of single neurons, the extent of correlation between neurons is not known. However, the population of neurons is probably quite similar to that studied by Zohary et al. (1994), in that both studies used random dot stimuli at zero disparity. This would imply that the size of the interneuronal correlation may differ depending on the external stimulus.

A change in interneuronal correlation could be induced by certain types of stimuli, if a stimulus-related signal from outside area MT simultaneously affects all of the MT neurons in a pool. For example, MST shows disparity-dependent direction selectivity (Roy et al., 1992), so it might send signals that are related to the perception of rotation, with the effect of increasing the activity of MT neurons that would normally signal the same rotation. Thus, it is possible that the large choice probability represents a top-down effect, perhaps influencing all of the neurons within a column and increasing interneuronal correlation. The primary evidence that has been used to argue that choice probabilities do not represent a top-down effect is the time course of the response. If the top-down signal represents an animal's intended response then this should grow on average as the end of the trial approaches. The observed time course shows no such increase for the direction discrimination task (Britten et al., 1996) and only a gradual trend in our data. However, for the cylinder stimuli used here, the percept is stable over periods of a few seconds, so the animals' intended response may be stable over the course of a single trial. Under such circumstances, the time course cannot be used to argue against a top-down influence.

If the large choice probabilities do reflect a top-down process, this process is more than just the direction of attention to a locus in three-dimensional space. If the animal attended to the surface that appeared in front and the effect of directed attention was to increase firing rates (Treue and Maunsell, 1996), then this should produce choice probabilities that were in the inappropriate direction for approximately half of the neurons studied. Because our sample contained approximately equal numbers of neurons preferring leftward or rightward motion (and also equal proportions preferring the two directions of rotation), then attentional phenomena like this would not increase the mean choice probability. The choice probability was similar in these groups (Fig. 9), so if there were any such attentional phenomena, the effects were small. We conclude that, if an attentional process is involved, then it is most likely feature-based and related to the perceived direction of rotation of the cylinder. It is not necessary to invoke top-down processes to explain these data. If some other brain area signaled the direction of rotation and sent appropriate projections to area MT, the effects would be the same. Whether or not this would be considered a top-down process would depend on the site from which the projection originated.

Correlations between an animal's reports and neuronal activity have been noted in several areas of the primate brain other than MT. In one comparative study (Thiele et al., 1999) using visual motion, these correlations were of a similar size in several visual areas, whereas a series of studies using binocular rivalry suggests that they are greater in some areas than others (Logothetis and Schall, 1989; Leopold and Logothetis, 1996;Sheinberg and Logothetis, 1997). These studies have not used choice probabilities to summarize the population data, so quantitative comparisons are difficult. Binocular rivalry is potentially a more complex perceptual effect than the bistable stimulus investigated here, because even when a specific percept (such as a horizontal rather than a vertical orientation) is associated with one of rivalrous conditions, the observer is in general still able to state that they are inspecting the rivalrous stimulus rather than its nonrivalrous equivalent.

Additional experimental data, especially measurements of interneuronal correlation, are required to understand how and why the magnitude of the choice probability depends on the stimulus, task, and cortical area. Clarifying the factors that are responsible for the size of the choice probability will improve our understanding of the way in which the activity of single neurons is linked to perceptual decisions.