Psychophysically measured task strategy for disparity detection is reflected in V2 neurons

Abstract

In perceptual tasks, subjects attempt to rely on their most informative cues. Such strategic choices should be reflected in the types of sensory neurons used. We investigated this in a binocular disparity discrimination task. Using psychophysical reverse-correlation, also known as image classification, we identified the perceptual strategy in two macaques. Correlation between reported disparity sign and disparity noise samples on each trial yielded detection ‘filters’. Filter amplitude at near disparities exceeded that at far disparities, indicating that the subjects relied more on near disparities. Recordings from both macaques’ disparity-selective V2 neurons showed correlation between neuronal responses and perceptual judgment in near-preferring, not far-preferring units, mirroring the psychophysically measured strategy. After one monkey learned to weight near and far disparities equally, activity in its far-preferring neurons correlated with choice. Thus, the pattern of correlations between neuronal activity and perceptual reports reveals how subjects use their neuronal signals.

Introduction

Visual perception is an active process, and subjects usually learn to exploit their most useful signals for a given perceptual task. Perceptual signals (carried by neurons) should therefore be used differently in different tasks, depending on the subject’s strategy. One method that has been widely used to suggest that signals from particular neurons contribute to a given perceptual decision combines threshold psychophysics with single-unit recordings. A trial-to-trial correlation between neuronal activity and perceptual judgment, which cannot be explained by the visual stimulus, is taken as evidence that neuronal signals contribute to a decision^1–7. Such correlation, often quantified by ‘choice probability’, should then also depend on the task strategy employed by the subject⁶. Only those neurons providing signals that a subject chooses to exploit in a particular strategy should show substantial choice probabilities. It is natural that subjects usually rely on those signals that are most suitable statistically for the task at hand. Several studies have found that neurons carrying higher precision signals for a given task show stronger correlation with perceptual judgment^2,3,5,8,9.

Note that this sort of relationship would be expected of almost any system that makes optimal use of a set of variable input signals, and so does not clearly identify a relationship with a subject’s strategy. Consequently, these relationships would be observed even if considerable ‘hard-wired’ processing modified the signals before they were used for decision making. Situations where subjects adopt sub-optimal strategies are potentially more informative: suppose that a subject ignores some useful signals but not others, and that substantial choice probabilities are seen only for those neurons whose signal is included by this subject’s strategy. This would indicate a significantly more specific connection between neuronal activity and the individual subject’s perceptual decision than previously shown.

Here we explore this possibility by combining measures of choice probability (in area V2) with an objective measure of the two macaque monkeys’ strategy: image classification, also termed psychophysical reverse-correlation^10–12. In our disparity discrimination task this method produces psychophysical kernels that quantify how disparities in the stimulus contribute to the subjects’ decisions. A similarity emerges. The psychophysical kernels show that the animals give considerably less weight to far disparity signals. The neurophysiological data reveal that neurons carrying far disparity signals do not have significant choice probability, unlike neurons with near disparity signals (and despite having similar statistical reliability for the task). Subsequently, one of the two animals learned to give equal weight to near and far disparity signals. Once its strategy changed, highly significant choice probabilities were measured both in this monkey’s far-preferring and near-preferring V2 neurons. Combining these techniques allows us therefore to demonstrate for the first time a neurophysiological signature of the strategy that subjects apply to perceptual tasks.

Results

Two macaque monkeys were trained on a disparity discrimination task in weakly correlated random-dot-stereograms (RDS, Fig. 1). They reported in a forced choice task whether they perceived a central circular patch in front of (near) or behind (far) a surrounding annulus. The central patch contained dots at one of two ‘signal’ disparities (one disparity was always near, the other always far) masked by ‘noise dots’, whose disparity was chosen at random from a discrete distribution (−1.2°, −1.0°…1.0°, 1.2°). The surrounding annulus was presented at 0° disparity, and was 100% correlated. We controlled task difficulty with the percentage of signal dots. When there were no signal dots subjects were rewarded randomly. Within each trial, the distribution of noise dots was fixed and recorded, although the position of every dot was reassigned on each video frame. A different distribution of noise dots was chosen randomly for every trial. If a subject’s strategy places more weight on some disparities than others, these disparities will dominate the perceptual decision. We measured this by calculating the average number of dots at each disparity for all stimuli that elicited a near choice, and by calculating a separate average for stimuli eliciting a far choice. The resulting averages constitute ‘detection kernels’ (supplementary Fig. 1) and can be summarized by their difference (see Methods, Fig. 1c), which we term the psychophysical kernel. These kernels were built for all trials on which the stimulus only included noise dots. The psychophysical kernel for monkey 1 is shown in Fig. 2a. The signal disparities of the stimulus in this experiment were −0.2° (near) and 0.2° (far). The kernel amplitude shows a clear peak at −0.2°. Kernel modulation is much weaker for all other disparities. In particular, note that the amplitude at 0.2° is much smaller than that at −0.2°. This pattern is similar for monkey 2 (Fig. 2b). For both monkeys the kernels are more strongly modulated at −0.2° than at 0.2° even though both disparity values were equally useful in the task. The amplitude at the near disparities is significantly larger than that at the far disparities in all conditions for monkey 1, and in three out of the four conditions for monkey 2 (P < 0.05, by resampling). This suggests that the monkeys relied more strongly on the near disparities than on the far disparities. The psychophysical kernels were similar in all task conditions (Fig. 3a,b), despite the changes in signal disparities. The similarity between kernels for different signal disparities allows us to summarize all the data with a single kernel for each monkey (Fig. 3c,d), confirming that the subjects’ perceptual decision was most strongly influenced by near disparity dots at about −0.2°. (The peak kernel amplitude at the near disparities in this average is significantly larger than that at the far disparities, P < 0.0001 for monkey 1, P < 0.0001 for monkey 2.)

Figure 1.

Methods. Panel a schematically depicts the sequence of events in the task for the monkeys. Once the animal fixates, the stimulus is presented on the screen for 2 s. After that two choice targets appear above and below the fixation marker. Only saccades made within 500 ms to the correct choice target were rewarded. Panel b, shows schematic views of the stimulus at different levels of inter-ocular correlation (100%, 50% and 0%). Upper row represents a front view, where red and yellow dots are shown to the left and right eye only, respectively. The bottom row shows a schematic view from above: the subject fixates, and the stimulus appears as a disc at the signal disparity (orange dots, 100% correlation), as a ‘cloud’ extending in depth (cyan dots, 0% correlation) or as a mixture of both (50% correlation). Panel c shows schematically how the psychophysical kernels are calculated. The distributions of noise dots (number of dots at each disparity) on each trial were sorted according to the subject’s choices. For both kinds of choices, the detection kernel is the average dot distribution minus the average number of dots per disparity across all trials (μ). N_near and N_far are the number of near choice-trials and far choice-trials. Because of the obligatory symmetry up to a scaling factor between the near and far choice detection kernels, they can be summarized by their difference (near choice detection kernel – far choice detection kernel).

Figure 2.

The psychophysical kernels for one experimental condition. Panels a, b show the psychophysical kernels for monkey 1 (obtained from N = 1602 trials, N_nearchoice = 567, N_farchoice = 1035), monkey 2 (N = 5296 trials, N_nearchoice = 2432, N_farchoice = 2864), respectively, for the experimental condition in which the signal disparities corresponded to −0.2° and 0.2° (red lines). (Across all correlation levels, monkey 1 made N = 4974 near choices and N = 6070 far choices; monkey 2 made N = 11785 near choices and N = 13252 far choices). Errorbars show s.d. of the resampled distributions. In both monkeys the peak amplitude for near disparities significantly larger than that for far disparities (by resampling, P < 0.02). On average, the stimuli contained 16 dots at each disparity.

Figure 3.

Psychophysical kernels for all experimental conditions. Each column displays data for one monkey. Top row panels show the kernels for all conditions. Each row corresponds to one experimental condition (c1: [−0.2°,0.2°]; c2: [−0.2°,0.4°]; c3: [−0.4°,0.4°] ; c4: [−0.4°,0.2°]). The abscissa plots disparity in degrees; color represents kernel amplitude (number of dots). The total number of trials from which the kernels were calculated for each monkey are N = 7428 (monkey 1), N = 14751 (monkey 2). (Across all correlation levels and conditions, monkey 1 made N = 24468 near choices and N = 26405 far choices; monkey 2 made N = 30237 near choices and N = 33702 far choices). Bottom row panels plot the mean of the kernels in the top row across experimental conditions. Errorbars are s.d. (indicating the degree of similarity across conditions – statistical testing is based on resampling the raw data, see methods).

These results demonstrate that both monkeys based their decisions primarily on the presence or absence of dots with near disparities and place less weight on dots with far disparities. But the results do not explain why subjects chose this strategy, which is suboptimal in terms of stimulus content, although not necessarily relative to the information content of the set of disparity-tuned neurons on which the monkeys base their decision. We speculate that this sub-optimal strategy was related to the appearance of the uncorrelated stimulus, when surrounded by a zero disparity annulus. The segmentation of the uncorrelated stimulus and its correlated surround at zero disparity yields a clear sensation of depth in some subjects (see supplementary material for additional results in humans (supplementary Fig. 2–4) and discussion). Initial responses during training suggest that this was also the case for the animals used here. Both monkeys had previously been extensively trained on a stereoacuity task with similar RDS. When they first attempted a task in which binocular correlation was varied, they readily detected high correlations, but showed a bias for far judgments at low correlation. This suggests that these two animals perceived uncorrelated dots as far. The monkeys subsequently underwent extensive training to eliminate this bias in responses using a staircase-procedure¹³. It seems that learning not to give a far response at low correlation led to a strategy in which dots with far disparities were weighted less than dots with near disparities (see supplementary discussion).

Why the monkeys relied upon dots with disparities close to −0.2°, even when the signal disparity has a different value, is less clear. Note however, that during the recording experiments, the signal disparities were always chosen to approximately match the preferred and the null disparity of the recorded neuron. When recording successively from more than one neuron in a session, the signal disparities could change within a single recording session. The monkeys’ reliance on disparities close to −0.2° may reflect a strategy they adopted to deal with changing signal disparities.

These psychophysical kernels demonstrate a sub-optimal strategy employed by the monkeys: far choices largely reflected an absence of near dots, rather than the presence of far dots. Although we can only speculate about why the monkeys adopted this strategy, the important point is that the psychophysical kernels document the fact that they were using this strategy. This then allowed us to investigate whether this perceptual strategy was reflected in the neuronal response in disparity selective V2 neurons. For this, we reexamined previously reported data¹⁴.

Choice related neuronal response reflects task strategy

Responses from 71 disparity selective V2 units were recorded (significant modulation of mean response with disparity in a one-way-ANOVA, P < 0.01) in the same two macaque monkeys while they performed an almost identical disparity discrimination task in weakly correlated RDS. The only difference was that the positions of the noise dots in the two eyes images were completely uncorrelated. To appreciate the similarity of the two stimuli, note that in the psychophysical stimulus, if the total range of disparities used equals the stimulus width, the horizontal dot locations are uncorrelated in the two eyes’ images. The values for the two signal disparities corresponded approximately to the preferred and null disparity of the neuron. We quantified the trial-to-trial correlation of the neuronal response with the perceptual judgment as choice probabilities using receiver-operating-characteristic analysis³. This metric describes the probability with which an ideal observer would be able to predict the subject’s response on a given trial, knowing only the firing rate on this trial and the distributions of firing rates for the choices to both targets. We recently reported that disparity selective V2 neurons show significant choice probabilities in this task, with the mean choice probability in the population being 0.57, significantly larger than chance. We now separated the neurons according to the sign of their peak disparity (near-preferring neurons have a peak disparity < 0° and far-preferring are those whose peak disparity > 0°) for both monkeys (Fig. 4). This reveals that the mean choice probability was only significantly larger than chance for near-preferring neurons (mean choice probability was 0.62, N = 37, above 0.5, P < 0.001, by resampling), but not for far-preferring neurons (mean choice probability was 0.51, N = 34, not significantly different from 0.5). The same pattern holds in each monkey individually (Fig. 5): the choice probabilities are significant for the near-preferring units (0.58, N = 20 and 0.67, N = 17, for monkey 1 and monkey 2; significantly different from 0.5, P < 0.001 for both, by resampling, respectively; Fig. 5a,b), and at chance for the far-preferring neurons (0.52, N = 17, P = 0.17 and 0.5, N = 17, P = 0.98, both by resampling), for monkey 1 and monkey 2, respectively; Fig. 5c,d). For both monkeys, the difference in mean choice probability between the two groups was significant (P < 0.05 for monkey 1, P < 0.0001, monkey 2, t-test).

Figure 4.

Distribution of choice probabilities in V2, separated for near-preferring and far-preferring neurons. Filled and open bars depict cells with significant (P < 0.05) and non-significant choice probabilities, respectively, (N = 71, overall mean choice probability = 0.57). The distribution of choice probabilities for near and far-preferring neurons are plotted in the top (blue) and bottom (green) half of the panel, respectively. Note that the distribution for near-preferring neurons is shifted towards higher values (mean choice probability for near-preferring units is 0.62, blue triangle, and for far-preferring units mean choice probability = 0.51, green triangle).

Figure 5.

Distribution of choice probabilities separated by monkeys and preferred disparity sign. In Panels a–d the near preferring neurons are summarized in the top row (N = 20, mean = 0.58 and N = 17, mean = 0.67 for monkeys 1 and 2, respectively) and far preferring neurons are shown in the second row (N = 17, mean = 0.52 and N = 17, mean 0.50 for monkeys 1 and 2, respectively). Panel f shows the psychophysical kernels for monkey 2 after re-training (symbols as in Figure 2). The peak amplitudes for near and far disparities are no longer statistically different (P = 0.38). The changed strategy in monkey 2 is mirrored by significant choice probabilities in far-preferring neurons after re-training (Panel e, N = 18, mean choice probability = 0.61, significantly larger than 0.5, P < 0.001, and different from the mean choice-probability for far-preferring neurons prior to re-training, P < 0.002). The mean choice probabilities for near-preferring remains unchanged (g, N = 18, mean choice probability = 0.65, larger than 0.5, P < 0.001, not significantly different from the mean choice probability preceding pre-training, P = 0.62). Vertical dashed lines in (a–e,g) show distribution means (blue: near-preferring; green: far-preferring; light green: far-preferring after re-training; cyan: near-preferring after re-training), to facilitate comparison.

A previous study of choice probability in disparity selective MT neurons in an almost identical task reported that two neuronal properties were associated with choice probabilities⁵: 1) neurons whose disparity tuning curve was more asymmetrical about 0° tended to show higher choice probabilities; 2) a neuron’s choice probability was negatively correlated with its disparity correlation threshold (‘neuronal threshold’) for the task. If one of these properties were more prominent for the near-preferring than for the far-preferring neurons in our V2 data, the discrepancy in choice probability might simply be a reflection of this. We therefore investigated the distribution of these two properties in our V2 neurons. In the MT-study, disparity tuning symmetry was quantified as the phase of the Gabor fit to the disparity tuning curve whose mean was constrained to 0°. For many disparity tuning functions in V2, this constraint on the mean leads to poor fits. We therefore used a slight modification of the symmetry-phase¹⁵ to quantify symmetry of disparity tuning: the disparity tuning curves were fit by Gabor functions (required to explain 65% of the variance; N = 69.). The extent to which this fitted curve was even or odd symmetric (about zero disparity) was then expressed as a phase angle between −90° and 90°. The distributions of |symmetry-phase| for near and far-preferring neurons were not significantly different (P = 0.49, Kolmogorov-Smirnov test). We quantified the neuronal thresholds as described previously^5,16, see14. The mean neuronal threshold for the task was slightly lower for the far-preferring neurons (mean threshold 36% ±17%) than the near-preferring neurons (mean 44% ±19%), although this difference was not statistically significant (P = 0.14, t-test). Differences in disparity symmetry or neuronal thresholds between near- and far-preferring V2 neurons are therefore unlikely to explain the lack of significant choice probabilities in far-preferring neurons.

Rather, choice probability depended on the preferred disparity of the neuron, producing the negative correlation in Fig. 6a, (Spearman’s rank correlation, r_s = −0.43 P < 0.001). This correlation was significant in both monkeys individually (r_s = −0.34, P < 0.05, and r_s = −0.49, P < 0.005 for monkey 1 and monkey 2, respectively). The effect on responses over a range of disparities can be seen from a population mean disparity tuning curve (Fig. 6b), where the contribution from each neuron is weighted by [choice probability − 0.5]. (Note that this difference ensures that for neurons whose firing is unrelated to choice, random deviations of choice probability from 0.5 average to a weight of zero). This average weighted tuning curve shows a peak at about −0.5°, and a trough for disparities > 0° (far disparities). To control for any sampling bias, we also show the unweighted average disparity tuning curve (Fig. 6b), which is more balanced, with one peak for near disparities and one for far disparities. The single peak at near disparities for the weighted average is therefore a reflection of the fact that the monkeys relied more strongly on near disparities. That this average peaks at disparities of −0.5°, even though the psychophysical kernels suggest that the animal most strongly relied on disparities around −0.2° is surprising, if one supposes that neurons that carry the most weight in the decision have the highest choice probabilities. However, this assumption ignores the effect of inter-neuronal noise correlation. If noise was equally correlated between all near preferring neurons, then choice probability should be similar in all near preferring neurons, regardless of which neurons carry most weight for the decision¹⁷. This would produce a weighted tuning curve which would reflect the unweighted mean across all near neurons, as can be seen in Fig. 6b. This is similar to a previous observation¹⁷, in which the correlation between choice probability and neuronal thresholds could not be achieved by weighting more precise neurons more strongly, but could be explained by supposing that more precise neurons showed stronger inter-neuronal noise correlations than less precise neurons.

Figure 6.

Choice probability depends on preferred disparity. Squares correspond to data from monkey 1, circles to data from monkey 2. Panel a compares preferred disparity [degrees] with choice probability, showing a negative correlation (r_s = −0.43, P < 0.001, N = 71). Dashed lines mark the disparities of −0.2° and 0.2°, solid black lines the disparity of 0°, and choice probability of 0.5. In Panel b the average disparity tuning curve weighted by (choice probability − 0.5) is shown in red. Superimposed is the unweighted average disparity tuning curve in gray. In contrast to the unweighted average, the weighted average has a peak at disparities < 0°, and a trough at disparities > 0° (The trough arises because the weighted curve is dominated by near preferring neurons, which often show a trough in their response at small far disparities). Black lines as in Panel a.

A different explanation for the discrepancy in Fig. 6 would be to suggest that choice probability has a top-down origin. If the top-down signal is sent back to all near preferring neurons then choice probability would be present in all these neurons giving rise to a weighted average, like that shown in Fig. 6. Nonetheless, we do not claim that these explanations are correct – we merely point out that the shape of the weighted tuning curve and the psychophysical kernel do not necessarily have to match.

A changed strategy yields choice probabilities in far neurons

Both monkeys subsequently performed the discrimination task extensively (several months) with a modified stimulus. In this RDS, all the dots had the same disparity on any given frame, but the disparity value was changed randomly from frame to frame (96Hz). The appearance of the RDS was that of a central disc whose depth changed from frame to frame, surrounded by a zero-disparity annulus. Because the frame rate exceeded the temporal resolution for disparity modulation^18,19, successive disparities ‘blurred’ together, producing a transparent percept, with near and far components visible. This stimulus therefore did not have a near or far appearance at 0% correlation. As a consequence of training with this stimulus, monkey 2 learned to rely equally on near and far disparities, even when re-tested with the stimulus used to measure psychophysical kernels above. The kernel measured after this re-training at signal disparities (−0.2°, 0.2°) is shown in Fig. 5f. Unlike before (compare Fig. 2b), the amplitudes at near and far disparities were not significantly different from each other (P = 0.38). (Note that despite the same amount of re-training, monkey 1 persisted to rely more strongly on near disparities as indicated by significantly different kernel amplitudes, P < 0.05, N = 520, supplementary Fig. 5.) We subsequently recorded additional data from 36 units (18 near-preferring, 18 far-preferring) in monkey 2. The mean choice probability in the population of far-preferring neurons was 0.61, significantly larger than 0.5 (P < 0.001, by resampling, Fig. 5e), and significantly different from the mean choice probability in the far neurons before retraining (P < 0.002, t-test). After re-training, mean choice probability in far-preferring neurons no longer differed from the mean choice probability in near-preferring units (P = 0.36, t-test), while the mean choice probability in near-preferring neurons remained unchanged with this training (P = 0.62, t-test, pre- versus post-re-training; mean choice probability after re-training was 0.65, larger than 0.5, P < 0.001, by resampling, Fig. 5g). This change in choice probability for far-preferring units following the change in perceptual strategy strengthens our conclusion that choice probability reflects task strategy.

Discussion

Here we combine neurophysiological measures of choice probability with the technique of psychophysical reverse-correlation, which quantifies how information is used by observers when performing a task, in the same subjects. The results manifest a compelling similarity: in our task the monkeys appear to rely predominantly upon dots with near disparities, and choice probabilities are substantially larger in neurons signaling near disparities (0.62) than those signaling far disparities (0.51). This occurs despite the fact that in purely informational terms, the near-preferring and far-preferring neurons provided equally reliable signals. After one monkey’s strategy changed to giving similar weight to near and far disparities, its far-preferring V2 neurons showed significant choice probabilities (0.61). This demonstrates that for neurons in a given brain area to show substantial choice probability requires not only that they provide information that is relevant to the task— they must also provide information that the subject’s strategy exploits. This is the first demonstration that a measured task strategy is reflected in activity at the level of individual neurons, and that changing this strategy changes choice probability in the relevant neuronal population. Although it has been suggested before that patterns in observed choice probability reflect subjects’ strategies⁵, such proposals have been interpretations of neurophysiological data, without behavioral evidence that any particular strategy was employed.

To appreciate why the monkeys may have adopted this non-optimal strategy it is helpful to consider their initial bias: reporting far in response to the uncorrelated stimulus. From the reports of human observers, it seems likely that this bias is perceptual, i.e. that an uncorrelated stimulus looks far to the monkeys. We do not know the origin of this perceptual bias, and its direction and strength differ between human subjects (see supplementary Results and Discussion). In response to an uncorrelated stimulus the rewards are randomized and the monkeys therefore see the stimulus as far but are rewarded on only 50% of the trials. Conversely, the monkeys only perceive the stimulus as near when the stimulus contains a sufficient proportion of signal dots at the near disparity and therefore always receives a reward. Consequently, the sensation of near is a more reliable guide than the far sensation. The optimal way to compensate for this perceptual bias would be to integrate all of the sensory information, and simply apply a decision criterion (incorporate a response bias) that maximizes rewards. The monkeys’ failure to do this is demonstrated by our psychophysical kernels. It is important to note that this demonstration that the animals are using the signals in this way holds regardless of our speculation as to why the monkeys developed their sub-optimal strategy. This suggests that adjusting the perceptual decision criterion interacts with the processing of the perceptual signals themselves. That these changes are also reflected in the choice probabilities of V2 neurons indicates that this interaction occurs surprisingly early in visual processing.

If this interpretation is correct, the interaction between response criterion and sensory signals could be important in many situations. It may be that experimental manipulations of the response rates, e.g. by changing reward regimens or target salience, produce similar changes in the representation of the sensory inputs. Combining psychophysical reverse-correlation with measures of choice probability allows this possibility to be examined directly.

The above interpretation also supports ‘early selection theories’ of attention, e.g.20. These theories suggest that selection for further cognitive processing occurs early in and affects sensory processing, as opposed to ‘late selection theories’, e.g. 21,22 which in their extreme position propose that sensory processing is unaffected by attention and that selective processing only occurs at a later stage. A number of physiological studies that manipulate attention e.g.23–27 have likewise supported ‘early selection’.

Two general schemes in which task strategy could be reflected in the choice related activity of sensory neurons have been put forward: bottom-up or top-down²⁸. In the bottom-up scheme, it has been suggested that during training the monkey’s strategy causes relevant neurons to become more strongly connected to the decision process^5,29. Detailed computational simulations¹⁷ of the bottom-up scheme have quantitatively explained empirical findings of choice probabilities and psychophysical performance for motion discrimination in MT by assuming two pools of sensory neurons each supporting one alternative perceptual decision. Adapted to our disparity discrimination task, one pool of neurons would consist of near-preferring neurons, the other of far-preferring neurons. The stronger connectivity developed to support task strategy could be implemented in this scheme as a higher weight given to the activity of the near-preferring pool, generating higher choice probabilities for this group. However, all of our results are also readily explained in a top-down scheme. If the perceptual decision is formed elsewhere in the brain (compare³⁰), it would be unsurprising that this signal be fed back more strongly into those neurons used in the task, as has been shown for neurons at higher stages of processing, such as VIP³¹. Feature based attention^23,25,32 is an alternative top-down mechanism which could give rise to significant choice probabilities. Findings of task-dependent changes of inter-neuronal noise correlations, which are closely coupled to choice probability¹⁷, in MT (M.R. Cohen & W.T. Newsome, Soc. Neurosci. Abstr. 16.6, 2005) are difficult to reconcile with a purely bottom-up explanation. It therefore seems likely that a component of choice probability is top-down.

Combining psychophysical reverse-correlation with measurements of choice probabilities in the same animals, refines our understanding of how the activity of single neurons is linked to perceptual decisions. These links depend not only on the information carried by single neurons, but also upon just how that information is exploited in forming a decision.

Methods

Animals

Two male macaque monkeys (Macaca mulatta) were implanted with scleral search coils in both eyes³³, head fixation posts and a recording chamber under general anesthesia. All procedures were in agreement with the Public Health Service policy on the humane care and use of laboratory animals and all protocols were approved by the Institute Animal Care and Use Committee.

Task

The subjects performed a disparity discrimination task (near vs far). Stimuli were circular dynamic random-dot-stereograms (RDS), consisting of a circular central region and a surrounding annulus. On each trial, the central patch contained one of two disparities. For the neurophysiological experiments (average psychophysical functions in supplementary Fig. 6), the disparities were chosen to be close to the preferred and null disparities of the unit. The disparity values were not necessarily symmetrical about zero, but one disparity always had to be negative and the other always positive. The percentage of the dots correlated between the two monocular images varied from trial to trial (typical percentages were 0%, 6.25%, 12.5%, 25%, 50%). For the 0% correlation condition the disparity became undefined and subjects were rewarded randomly. The dots in the surrounding annulus were always 100% correlated, at zero disparity. The monkeys had to maintain fixation during the 2 s stimulus presentation and indicate their choice by making a vertical saccade (3° above or below the fixation marker) to one of the choice targets within 500 ms after the end of the stimulus presentation (Fig. 1a).

Stimulus for psychophysical reverse-correlation

The way in which uncorrelated dots were added to the display differed slightly for the two parts of the study. For physiological recordings, uncorrelated dots (‘noise dots’) were given completely independent locations in the two eyes’ images (and this was done independently on each video frame). For the psychophysical work, in order to produce sufficient disparity variation to permit reverse-correlation analysis, noise dots were given disparity drawn at random from a discrete distribution of 13 disparity values (−1.2°, −1.0°, …, 1.0°, 1.2°). Dot density was 20%, stimulus diameter (center) 3°, the width of the surrounding annulus 1°, eccentricity 4.2°. The total number of dots was the same on each trial. For the duration of a single trial, the distribution of dot disparities was the same on every frame and these values were stored, but the dot locations were randomly assigned anew on each frame (i.e the stimulus was a dynamic RDS). Although one frame contains many false matches, averaging these over many frames of the dynamic stimulus reduces their impact relative to the signal dots. Note also that the difference between this stimulus and a truly uncorrelated stimulus is small. If the disparity range had been −1.5° to 1.5° (range = stimulus diameter), then the horizontal dot locations would have been uncorrelated in the two eyes.

Psychophysical reverse-correlation analysis

Only trials on which the stimuli consisted entirely of noise dots (defined as 0% correlation) were used to calculate the psychophysical kernels. To obtain the ‘detection kernels’ for the near and far choices, we calculated the mean disparity distribution for near and far choices respectively. The average number of dots per disparity for all trials was subtracted from each mean distribution, so that kernel amplitude was represented in units of numbers of dots (above or below average). The average number of dots at each disparity in the 0% correlated stimulus was 16. There is an obligatory symmetry between the near detection kernel and the far detection kernel up to a scaling factor, because they were obtained by splitting a single distribution into two. For this reason, the two kernels are summarized by their difference ( k_i ). This vector k_i corresponds to the amplitude of the psychophysical kernel as a function of disparity (i), s̄_near,i and s̄_far,i are the mean stimulus vectors preceding near and far choices, N_near and N_far the numbers of near and far choices, and s_{near, j,i} , s_{far, j,i} the stimulus vectors preceding the near and far choices on each trial (j), respectively:

$$k_i={\overline{s}}_{\text{near},i}-{\overline{s}}_{\text{far},i}=\frac{1}{N_{\text{near}}}\sum _{j=1}^{N_{\text{near}}}{s}_{\text{near},j,i}-\frac{1}{N_{\text{far}}}\sum _{j=1}^{N_{\text{far}}}{s}_{\text{far},j,i}.$$

We tested whether the monkeys relied predominantly on near or far disparities by resampling³⁴. For each resampled kernel we compared the peak amplitude for all near disparities against that for all far disparities, and p-values were obtained from these comparisons. This procedure was performed for each signal-disparity condition individually and on the combined dataset for each monkey.

Recording and stimulus presentation for electrophysiology

Most of the physiological data presented here are further analyses of data for which the recording procedure was fully described in¹⁴. Briefly, we recorded extracellular activity from disparity selective single-units in V2. The horizontal and vertical positions of both eyes were measured with a magnetic scleral search system and digitized at 800Hz. The monkeys viewed the stimuli on CRT monitors in a Wheatstone stereoscope configuration. All stimuli were dynamic RDS (2–5° center-diameter, 1–2° width of surround, 50% black and 50% white dots of 99% contrast, dot-density generally 40%, dot size generally 0.09×0.09°).

Analysis of choice probabilities

Trial-to-trial correlations between neuronal firing and perceptual choice were quantified as ‘choice probabilities’³ based on signal detection theory. Data for correlation-levels of 25% and below and for which the monkey had made at least five near choices and five far choices were included. For each trial, the mean firing rate was obtained. The firing rates at each disparity and correlation-level were z-scored to remove the stimulus evoked mean response and variance. The z-scores were then divided into two groups based on the monkeys’ choices. From the distributions of the z-scores for all near and far choices, we calculated a receiver-operating-characteristic (ROC) curve. The choice probability is defined as the area under the ROC curve. In a permutation test^3,13 (1000 permutations) we determined whether the choice probability was significantly different from chance. choice probabilities that lay outside the 95% interval of the distribution of permuted choice probabilities were considered significant.

Stimulus which altered the perceptual strategy

When trying to change the monkeys’ strategy, the RDS were modified such that on each frame, all the dots of the central region had the same disparity, but this disparity value changed randomly from frame to frame (the frame rate was 96Hz). For the condition defined as 0% correlation, the disparity value on each frame was drawn from a uniform distribution of discrete disparity values (usually −0.6°, −0.5°,… 0.5°, 0.6°). Disparity signal was introduced by increasing the probability for this disparity value on a given frame. This stimulus looked different from an uncorrelated stimulus, and yielded a more transparent percept with near and far disparities visible.