Neural correlate of subjective sensory experience gradually builds up across cortical areas

Abstract

When a sensory stimulus is presented, many cortical areas are activated, but how does the representation of a sensory stimulus evolve in time and across cortical areas during a perceptual judgment? We investigated this question by analyzing the responses from single neurons, recorded in several cortical areas of parietal and frontal lobes, while trained monkeys reported the presence or absence of a mechanical vibration of varying amplitude applied to the skin of one fingertip. Here we show that the strength of the covariations between neuronal activity and perceptual judgments progressively increases across cortical areas as the activity is transmitted from the primary somatosensory cortex to the premotor areas of the frontal lobe. This finding suggests that the neuronal correlates of subjective sensory experience gradually build up across somatosensory areas of the parietal lobe and premotor cortices of the frontal lobe.

Recent studies combining psychophysical and neurophysiological experiments in behaving monkeys have provided insights into which attributes of the neuronal responses evoked by a stimulus are related to sensory discrimination (1–3). In particular, these studies have addressed how neural codes are related to perception (4–8), working memory (9–13), and decision making (14–20). There remains, however, a fundamental problem posed by sensory-detection tasks: repeated presentations of a near-threshold stimulus might unpredictably fail or succeed in producing a sensory percept (21–22). Where in the brain are the neuronal correlates of these varying perceptual judgments? One possibility is that they are mediated by neurons of early sensory cortices (23–24) or by neurons of more central areas downstream in the processing hierarchy (22, 25–27). Previous studies sought support for these conjectures. In particular, studies found that the responses of neurons of the primary somatosensory cortex (S1), recorded while monkeys judged the presence or absence of near-threshold stimuli, did not covary with the monkeys' perceptual reports (22). In contrast, the activity of medial premotor cortex (MPc) neurons closely covaried with the perceptual reports (22). An important question posed by these results is whether the neuronal correlates of the perceptual judgments arise abruptly in a given cortical area or whether they gradually build as sensory information is transmitted across areas between S1 and MPc.

We addressed these questions by recording from single neurons in somatosensory cortices of the parietal lobe and in premotor cortices of the frontal lobe while trained monkeys reported the presence or absence of a mechanical vibration of varying amplitude applied to the skin of one fingertip. The results indicate that covariations between the neuronal activities and perceptual judgments increase gradually, from lower covariations in the somatosensory cortices of the parietal lobe to higher covariations in premotor areas of the frontal lobe.

Results

General.

We trained two monkeys (Macaca mulatta) to perform a detection task in which they had to report whether the tip of a mechanical stimulator probe vibrated or not by pressing one of two push buttons with the free hand (Fig. 1a). Stimuli were sinusoidal of varied amplitude across trials, had a fixed frequency of 20 Hz, and were delivered to the glabrous skin of one fingertip of the restrained hand. Stimulus-present trials were interleaved with an equal number of stimulus-absent trials in which no mechanical vibrations were delivered. Because of task design, the monkeys' responses could be classified into four types: hits and misses in the stimulus-present condition, and correct rejections and false alarms in the stimulus-absent condition (Fig. 1b). Detection performance was calculated from the behavioral responses (Fig. 1c). We recorded from single neurons in several cortical areas of the parietal and frontal lobes while monkeys performed the sensory detection task (Fig. 1d).

Fig. 1.

Detection task. (a) Trials began when the stimulator probe indented the skin of one fingertip of the restrained right hand (probe down, PD). The monkey then placed its left hand on an immovable key (key down, KD). After a variable prestimulus period (uniformly distributed from 1.5 to 3.5 s), on half of the randomly selected trials, a vibratory stimulus (20 Hz, 0.5 s) was presented. Then, after a fixed delay period (3 s), the stimulator probe moved up (probe up, PU), indicating to the monkey that it could make the response movement (MT) to one of two response push buttons. The button pressed indicated whether or not the monkey felt the stimulus (“yes” and “no” responses, respectively). (b) Depending on whether the stimulus was present or absent and on the behavioral response, the trial outcome was classified as a hit, miss, correct rejection (CR), or false alarm (FA). Trials were pseudorandomly chosen; 90 trials were stimulus-absent (amplitude 0), and 90 trials were stimulus-present with varying amplitudes (nine amplitudes with 10 repetitions each; 2.3–34.6 μm). (c) Psychometric detection curve obtained by plotting the proportion of “yes” responses as a function of stimulus amplitude in logarithmic abscissa (n = number of runs; a run consists of 180 trials, 90 stimulus-absent and 90 stimulus-present trials). (d) Recorded cortical areas include 1/3b, 2, 5, secondary somatosensory cortex (S2), and ventral premotor cortex (VPc) on the left hemisphere; dorsal premotor cortex (DPc) and MPc bilaterally; and primary motor cortex (M1) on the right hemisphere.

Neural Responses Across Cortical Areas During the Detection Task.

We found that the activity evoked by the vibrotactile stimulus is distributed from early somatosensory cortices to a large number of areas, including association and motor areas. Fig. 2a shows that the majority of the recorded neurons across cortical areas showed transient increases in their firing rates in response to a suprathreshold stimulus of the stimulus set. We also found a small number of neurons (<10%) that had transient decreases in their firing rate during the stimulus presentation. These responses were observed in the S2 and in the premotor areas (VPc, DPc, and MPc only). In addition, we also recorded neurons in the frontal lobe that had sustained increased or decreased activity beginning during the stimulus onset and ending during the probe up, which triggered the initiation of the decision motor report.

Fig. 2.

Mean firing rate in stimulus-present trials across the recorded cortical areas. (a) Each row plots mean firing rates to a suprathreshold stimulus in a given cortical area, and each column groups the neuronal responses with similar dynamics across cortical areas (n = number of neurons). Neurons from each cortical area were sorted into three possible categories (ordered into three columns). (Left) Neurons with transient responses to the stimulus (sensory neurons). The continuous line indicates rapidly adapting responses (area 3b and area 1 panels). Dashed lines indicate slowly adapting responses (area 3b and area 1 panels). Solid red lines in the remaining panels show neurons that transiently decreased their firing rate in response to the stimulus. Red dashed line in the area M1 panel shows mean activity of neurons that responded only during movement time. (Center) Activity of neurons that responded during the stimulus period and continued during the delay period (delay neurons). (Right) Mean activity of neurons with ramping changes in firing rate during the delay period. (b) Mean normalized firing rates as a function of stimulus amplitude. Colored lines are linear fits to the firing rate as a function of the logarithm of the amplitude (see Materials and Methods).

The responses in different cortical areas might play distinct roles in the processing of sensory stimuli. To test this possibility, we measured to what extent the neuronal firing rate was modulated by the stimulus strength. We carried out linear-regression analysis on the normalized firing rates as a function of stimulus amplitude across the recorded areas. Results show that the slopes of the fitted lines progressively approached zero in neurons downstream from the primary somatosensory areas (Fig. 2b). The decreasing slope values imply that neuronal responses of higher-order areas do not encode stimulus amplitude with the same fidelity as those in the early somatosensory cortex. Thus, the representation of the stimulus gradually transforms from a parametric one to a more abstract representation, an all-or-none response that does not depend on the amplitude but only on whether the subject felt or missed the stimulus (22).

Covariations Between Neuronal Responses and Perceptual Judgments.

The mere fact that neurons respond during the detection task does not imply that they participate in the construction of a sensory percept. One way to estimate the relationship between the neuronal activities and the sensory reports is by means of the choice-probability index (18, 28, 29), which quantifies the proportion of behavioral responses that can be predicted from single neuronal responses. By analyzing the neuronal responses to repeated presentations of the same near-threshold stimuli, we estimated the proportion of behavioral responses that could be predicted as a function of time and across cortical areas (Fig. 3; see Materials and Methods). As reported before, S1 showed little predictive capacity regarding the behavioral outcomes in response to near-threshold stimulus presentations (22). This finding was also the case for somatosensory areas 2 and 5, which showed choice-probability indices close to 0.5. However, variations in the activity of S2 neurons onward were correlated with the behavioral outcomes significantly above chance (Fig. 3). It must be noted that the predictive activity of S2 neurons was restricted to the stimulus period, whereas neurons from VPc, DPc, and MPc showed predictive activity also during the delay period between stimulus offset and the initiation of the decision motor report. Note also that M1 neurons showed no significant predictive activity. This result suggests that the activity in premotor cortices does not constitute a motor signal alone.

Fig. 3.

Proportion of behavioral responses that were predicted from the neuronal activity. Mean choice-probability indices across all neuronal types are plotted as a function of time for each of the recorded cortical areas during stimulus-present trials (Left) and stimulus-absent trials (Right). Note how choice-probability values increase from the primary sensory areas to the premotor areas (black lines, mean value; red area, ±SEM). Black lines at the top of each panel mark the times where choice-probability values significantly depart from 0.5 (t test, P < 0.01).

In addition to predicting whether subjects would fail (miss) or succeed (hit) in perceiving the vibration in stimulus-present trials, premotor activity also predicted the behavioral outcome on stimulus-absent trials, made by calculating the choice-probability index between correct-reject and false-alarm responses (see Materials and Methods). In the majority of stimulus-absent trials, the monkeys correctly answered “no” (correct reject), but on ≈10% of trials, monkeys erroneously answered “yes,” producing a false-alarm response. Remarkably, the neuronal activity from premotor areas predicted a significant fraction of these false-alarm responses (Fig. 3 Right).

Timing of Perceptual Decision Signals Across Cortical Areas.

The time it takes for a given area to start responding to the stimulus presentation can be related to the location of this area within the sensory-processing hierarchy. To address quantitatively the relationship between the predictive capacity of neurons and the hierarchy of the recorded areas, we plotted the choice-probability indices as a function of the response latency (Fig. 4a; see Materials and Methods). As evidenced by the 1-σ contours of two-dimensional Gaussian fits (17), neurons located in areas with longer mean latencies (i.e., downstream in the processing stages) covaried with the subjects' perceptual reports. This increase in the predictive capacity of neurons can also be appreciated by plotting the mean choice-probability index as a function of the mean response latency for each cortical area (Fig. 4b). M1 was excluded from the regression analysis because the majority of neurons showed responses during the movement period, and only a small fraction of them weakly responded to the stimulus (Fig. 2a).

Fig. 4.

Timing and strength of perceptual decision signals across cortical areas. (a) Choice-probability indices for individual neurons (mean value: hits vs. misses and correct rejections vs. false alarms) plotted as a function of the response latency for each cortical area (colors are as in Fig. 1d). Neurons from each area were fitted with two-dimensional Gaussians. Color markings at the abscissa indicate the mean response latency for each cortical area. (b) Mean choice-probability index for each area plotted as a function of the mean response latency. A linear fit shows how the choice-probability index increasingly grows as a function of latency (M1 neurons were excluded from the fit; red dot and dotted circle). (c) Recorded areas grouped into five processing stages by analysis of variance of response latencies. Each rectangle groups the areas with latencies that were statistically indistinguishable from each other.

We also tested whether the choice-probability values were correlated with the response latency in neurons within each cortical area. As evidenced by the positive slope of the major axis of the ellipses, the choice-probability indices for neurons within areas VPc, DPc, and MPc were positively correlated with the response latencies (Pearson's correlation coefficients: VPc, r = 0.34, P = 0.004; DPc, r = 0.35, P < 0.001; MPc, r = 0.18, P = 0 .009; ref. 30), which means that even neurons within the same processing stage tended to be more correlated with the subjects' perceptual reports if their responses to the stimulus appeared later.

To analyze further the information flow between the recorded cortical areas, we performed an analysis of variance on the response latencies followed by a Tukey's multiple-comparison test (31). This analysis showed that the cortical areas could be arranged according to the response latency into five possible processing stages (Fig. 4c). Neuronal responses appear first in areas 1/3b, then in area 2, then simultaneously in S2 and area 5 (response latencies of these last two areas are statistically indistinguishable). The fact that the responses appear simultaneously in these last two areas is consistent with the idea of parallel ventral and dorsal streams of somatosensory information processing (32). However, the fact that S2 neurons showed larger choice-probability indices suggests that this area is more related to the detection task than area 5.

Responses to the vibratory stimulus then appear in the DPc and the VPc, which correspondingly show more predictive capacity about the subjects' perceptual reports than early sensory areas (Fig. 4b). Finally, the statistical tests showed that the last responses to the stimulus appear in neurons from M1 and MPc. It is important also to note that the choice-probability index reaches its maximum value in area MPc, and it drops to chance levels in the neurons from M1.

Sensory vs. Motor Responses.

The results show that neurons from the frontal lobe relate to the subjects' perceptual reports. However, an alternative interpretation of these results is that, instead of relating to a sensory percept, responses of frontal-lobe neurons are more likely associated with the motor plan implemented during the detection task. To evaluate the influence of movement direction on the neuronal responses, we carried out control experiments in which the correct response button was illuminated at the beginning of the trial. In this condition, the monkeys were not required to attend the vibratory stimuli but just to press the illuminated button at the end of each trial to get a reward. Importantly, in the control task, the response buttons were reversed so that monkeys had to push the “no” button in stimulus-present trials and the “yes” button in stimulus-absent trials. The control task provided a condition in which both the decision to make a movement and the movement direction were given at the beginning of the trial. If the predictive activity we observed in the frontal lobe were the result of the decision process or the onset of a motor plan, we would have expected this activity to be modified by the onset of the light cue and the reversed movements.

Comparison of activity in detection and control trials revealed that, although some areas showed significant changes in basal firing rates (evident in no-stimulus trials; Fig. 5), responses to the stimulus were neither abolished nor changed in sign. In other words, increases or decreases in the firing rates observed during the normal detection task were also observed during control trials. These results do not support the view that the predictive activity recorded in the frontal lobe is the result of motor planning.

Fig. 5.

Sensory vs. motor responses during the detection task. Neurons for each area were tested in a control condition in which detection trials were presented as usual, but a light cue indicated to the monkeys which button to press at the beginning of each trial (n = number of neurons). In the control task, however, the response buttons were reversed relative to the detection task. (Left) Responses of the neurons during the stimulus-present trials (black continuous lines) and during the stimulus-present and reversed lights (red dashed lines). (Right) Stimulus-absent trials (black continuous lines) and stimulus-absent trials and reversed lights (red dashed lines).

Discussion

Analysis of the relationship between the neuronal responses and stimulus amplitude revealed that the activity of early somatosensory areas encodes stimulus strength. This sensory representation gradually transforms, starting in somatosensory areas S2 and 5, into an all-or-none response in the premotor areas of the frontal lobe that highly covary with the monkeys' reports about the presence or absence of the vibratory stimulus.

Choice-probability analysis revealed that neuronal activity in response to the stimulus spreads from the somatosensory cortex to the premotor areas within a 180-ms period, and it progressively correlates more with the subjects' perceptual reports. This gradual increase in choice probability across the cortical hierarchy is consistent with the hypothesis that sensory perception develops through time and across cortical areas, which might also suggest that no particular area plays a preponderant role in the sensory-to-motor transformation leading from a stimulus representation to a perceptual report.

The hierarchy suggested by the statistical analysis of neuronal-response latencies across cortical areas seems to favor a serial processing. However, it must be noted that there is high variability in response latencies and a high amount of overlap across the recorded areas (Fig. 4a). This overlap reveals that there are some neurons from higher cortical areas that show shorter latencies than some neurons from lower cortical areas, leaving room for feedback inputs from higher to lower areas, a process that has recently been acknowledged to play important roles in sensory processing (33).

Because the neuronal activity with the highest correlation with perceptual reports was recorded in the premotor areas, a trivial explanation of the results would be that this activity is related to the motor component of the detection task. We do not think that this explanation is satisfactory because the control task of reversed movements guided by visual cues shows that basal neuronal activity was modulated only weakly by the direction of movement. In addition, given that the light cues indicated the correct response button at the beginning of the trial, we consider unlikely the possibility that the responses to the stimulus could be caused by motor processes. The fact that neurons from M1 do not show large predictive capacities renders the motor explanation of our data further unlikely.

The picture of somatosensory processing that has emerged from the experiments is far from complete. For instance, there are many areas from which we did not record, which are nonetheless known to show somatosensory responses (34). However, we think that the results are complete enough to show that the activity arising from the somatosensory cortex gradually relates more to the subjects' perceptual reports as it reaches the premotor areas. Our results are consistent with observations in the frontal eye field of monkeys performing a masking task (26). In this task, the monkeys reported the location of a briefly flashed visual cue that was rapidly substituted by a masking stimulus. Responses of frontal eye field neurons were highly correlated with the subjects' behavioral reports, whereas neurons in areas closer to the periphery were not.

To conclude, we think that the gradual transformation of a sensory representation might be a fundamental process by which the cortex builds up a sensory percept, evidenced by the fact that covariations between the neuronal responses and perceptual reports grow across cortical areas, from lower covariations detected in early sensory cortices to higher covariations detected in frontal-lobe neurons. Thus, the subjective sensory experience might be consolidated in the frontal lobe (22) after a gradual transformation of the sensory representation. This interpretation is also supported by the fact that in the stimulus-absent trials, the activity of frontal-lobe neurons predicted the false alarms, whereas the somatosensory areas of the parietal lobe did not. One limitation of the present data set, however, is that neurons in different areas were recorded separately, so it is very likely that we have missed important functional relations between neurons within and across the processing stages. In the future, more information about the neuronal correlates of sensory and perceptual transformations and about the dynamics of real-time neuronal interactions should be obtained by using multiple-site simultaneous recordings.

Materials and Methods

Detection Task.

Stimuli were delivered to the skin of the distal segment of digits 2, 3, or 4 of the restrained right hand by a computer-controlled stimulator with a 2-mm round tip (BME Systems, Baltimore, MD). The initial indentation was 500 μm. Vibrotactile stimuli consisted of trains of 20-Hz mechanical sinusoids lasting 500 ms with amplitudes of 2.3–34.6 μm (Fig. 1a). Stimulus-present trials were interleaved with an equal number of trials where no mechanical vibrations were delivered to the skin (amplitude 0). Monkeys pressed one of two buttons to indicate stimulus presence (left button) or stimulus absence (right button). Correct responses (stimulus-present and stimulus-absent trials) were rewarded with a drop of liquid. Ten repetitions of each of the nine stimulus-amplitude classes combined with 90 stimulus-absent trials resulted typically in 180 trials in each experimental run. Detection curves were constructed by plotting the probability of “yes” answers as a function of stimulus amplitude (Fig. 1c). Animals were handled according to institutional standards that met or exceeded those of the National Institutes of Health and the Society for Neuroscience.

Neuronal Recordings and Sites.

Neuronal recordings were obtained with an array of seven independent, movable microelectrodes (2–3 MΩ; ref. 4) inserted in areas 3b, 1, 2, S2, 5, VPc, DPc, and MPc in the hemisphere contralateral to the stimulated hand (Fig. 1d) and in VPc, DPc, MPc, and M1, in the hemisphere ipsilateral to the stimulated hand. Neurons from areas 3b and 1 had small cutaneous receptive fields with either slowly adapting properties or quickly adapting properties, whereas those from areas 2, 5, and S2 had large cutaneous receptive fields with no obvious submodality properties. Neurons of the frontal cortex had no obvious cutaneous or deep receptive fields; they were selected if they responded to any of the different components of the detection task. The locations of the electrode penetrations were confirmed with standard histological techniques. Cortical areas were identified based on cortical landmarks.

Data Analysis.

For each neuron studied during the detection task, we calculated the firing rate as a function of time by using a 50-ms window displaced every 10 ms. The activity of neurons from each cortical area was pooled and grouped according to response dynamics (Fig. 2a). Normalized activity was calculated for each neuron by dividing the mean response to each stimulus amplitude by the mean response to the largest stimulus amplitude. For each neuron, responses to the stimulus were defined as the mean number of spikes within a 500-ms window centered at the response peak. Linear regressions were performed on the normalized firing rates as a function of the stimulus amplitude (Fig. 2b; ref. 30). Only hit trials from neurons with increments in activity were used for this analysis.

The proportion of predicted behavioral responses was calculated by means of the choice-probability index (18, 28, 29). This quantity measures the overlap between two response distributions; in this case, distributions between hit and miss trials at near-threshold amplitudes of 12.6, 9.0, and 6.4 μm, and between correct-reject and false-alarm trials (stimulus-absent). This analysis was a function of time, with a 50-ms bin displaced every 10 ms beginning from probe down and ending 500 ms after the motor response. Choice-probability values that significantly departed from 0.5 were identified by means of a one-tailed t test (P < 0.01).

The neuronal response latency in each trial was calculated by using an algorithm developed by Schall and colleagues (35). Briefly, this algorithm identifies periods of activity with significantly higher or lower firing rates compared with basal activity (P < 0.01). The time from the stimulus onset to the first spike of a significant burst within the stimulus period was defined as the stimulus-response latency. The latency for neurons that decreased activity in response to the stimulus was the mean interspike interval of the prestimulus period to the time of the spike that marked the beginning of the decreased response (generally a 200- to 500-ms window with no spikes). The time resulting from this sum marks the location where a spike would have been expected to appear if no stimulus was presented.

Abbreviations

DPc

dorsal premotor cortex

M1

primary motor cortex

MPc

medial premotor cortex

S1

primary somatosensory cortex

S2

secondary somatosensory cortex

VPc

ventral premotor cortex.