Encoding goals but not abstract magnitude in the primate prefrontal cortex

Summary

Functional neuroimaging studies show that perceptual judgments about time and space activate similar prefrontal and parietal areas, and it is known that perceptions in these two cognitive domains interfere with each other. These findings have led to the theory that temporal and spatial perceptions, among other metrics, draw on a common representation of magnitude. Our results indicate that an alternative principle applies to the prefrontal cortex. Analysis at the single-cell level shows that separate, domain-specific populations of neurons encode relative magnitude in time and space. These neurons are intermixed with each other in the prefrontal cortex, along with a separate intermixed population that encodes the goal chosen on the basis of these perceptual decisions. As a result, domain-specific neural processing at the single-cell level seems to underlie domain generality as observed at the regional level, with a common representation of a prospective goals rather than a common representation of magnitude.

Introduction

Time and space have such a close relationship in human perception that, according to Piaget, “time and space form an inseparable whole” (Piaget, 1927, p. 1). Temporal and spatial perceptions interfere with each other in both humans and monkeys (Casasanto and Boroditsky, 2008; Merritt et al., 2010; Xuan et al., 2007), saccadic eye movements compress magnitude judgments of both space and time (Morrone et al., 2005), and spatial manipulations such as prism adaptation cause misperceptions of time intervals (Magnani et al., 2011).

These and other psychophysical interactions have led to the idea that the brain encodes magnitude in domain-general representations that include space and time, as well as number, size and speed (Gallistel and Gelman, 2000; Walsh, 2003). According to this theory, some neural networks encode a greater or lesser quantity in the abstract, independent of metrics such as distance, duration, speed, numerosity etc. Although some findings support an abstract neural representation of magnitude, such as the effect of cortical damage on both space and time perception (Basso et al., 1996; Mitchell and Davis, 1987; Zorzi et al., 2002), other results seem to contradict this idea. For example, some patients with lesions of different frontal and parietal areas have deficits in either perceiving numbers or durations, but not both (Cappelletti et al., 2009; Cappelletti et al., 2011). Likewise, an asymmetry in the interference between temporal and spatial perceptions indicates separate, domain-specific mechanisms that interact with each other, rather than a common representation of magnitude (Casasanto et al., 2010).

Neurons in the prefrontal (PF) and parietal cortex encode space, time and number (Nieder and Miller, 2004a, b; Tudusciuc and Nieder, 2007, 2009), including categories of these metrics (Merchant et al., 2011), and several contemporary theories of the PF cortex have stressed domain generality and cross-domain information processing (Baars et al., 2003; Duncan, 2010; Wilson et al., 2010). In two previous reports, we have described PF activity during the discrimination of relative durations (Genovesio et al., 2009) and relative distances (Genovesio et al., 2011), recorded in the same PF areas. These reports do not, however, address whether individual neurons in these areas encode relative magnitude in both cognitive domains. In order to search for a representation of common magnitude, we analyzed the activity of cells that were recorded in both of these discrimination tasks, along with a control task that we used to identify goal coding.

Results

Figure 1A–C illustrates the three tasks. The monkeys sat within reach of three switches, and on each trial they viewed two visual stimuli, presented sequentially on a video screen. Each stimulus was either a red square or a blue circle. The first stimulus (S1) was followed by a delay period (D1), the second stimulus (S2) and another delay period (D2), after which both stimuli reappeared, one to the left and the other to the right. This event served as the “go” signal, and to receive a reward the monkeys had to touch the switch below the stimulus that had lasted longer (Fig. 1A), appeared farther from a central reference point (Fig. 1B), or appeared previously on that trial (Fig. 1C), depending on the task. By design, the monkeys could not select their response during the S2 or D2 periods because the correct choice could be left or right. The matching-to-sample task (Fig. 1C) allowed us to identify cells that encoded nonspatial goals, independent of spatial or temporal judgments; it did not require a decision about stimulus magnitude, although the two (identical) stimuli appeared for intervals matching the duration task.

Figure 1. Tasks and electrode penetration sites.

A–C. Example sequences of stimuli and choices. A. Duration task. B. Distance task. C. Matching task. D. Entry points for electrode penetrations for all the neurons recorded in the duration task, distance task or both. Abbreviations: AS, arcuate sulcus; PS, principal sulcus. Note that because of the angle of penetration for the caudal entry sites, the recordings were taken from the prearcuate cortex (area 8), not from the postarcuate cortex through which the electrodes passed.

Fig. 1D shows the location of recorded neurons. Although many penetration sites were caudal to the arcuate sulcus, the electrodes were inserted at such a steep angle that they recorded activity in cortex rostral to the arcuate sulcus (area 8). Few if any cells were located medial to the superior limb of the arcuate sulcus and, by comparison with published maps, they were outside the frontal eye field, as well. Thus the recordings were concentrated in two regions: caudal PF cortex (area 8) and dorsolateral PF cortex (area 46) in both banks of the principal sulcus, extending to adjacent cortex ventrally. Fig. S1 shows penetration sites for cells with specific properties.

Of 1209 cells recorded in the duration task and 1671 cells in the distance task, we collected adequate data on 621 cells for both tasks (161 from dorsolateral PF and 460 from caudal PF; 80 from Monkey 1 and 541 from Monkey 2). Of these cells, we also obtained adequate data from 261 cells in the matching task. The data reported here come from the same monkeys and the same recordings used for our previous reports (Genovesio et al., 2009, 2011). No new cells were added for the present analyses. Fig. S2 presents the monkeys’ performance accuracy and reaction times. The tasks alternated in interleaved blocks with means of 192, 151 and 92 trials for the duration, distance and matching tasks, respectively.

The present analysis concentrated on an interval 80–400 ms after the ideal decision point, which refers to the instant that an ideal observer could have made a decision, not when the monkeys did so. We concentrated on this interval, called the decision period, in order to search for domain-general coding when the key perceptual decision could first be made. In the distance task, the ideal decision point corresponded to S2 onset. In principle, an observer could decide whether the red or blue stimulus had appeared farther from the reference point as soon as S2 appeared. The duration task had two decision points, depending on whether S1 was longer than S2. If so, then an observer could decide whether the red or blue stimulus had lasted longer at S2 offset. Otherwise, a decision could be made once the duration of S2 surpassed that of S1. In the matching-to-sample task, the monkeys could decide about the sample as soon as S1 appeared. Nevertheless, to compare activity among tasks we analyzed activity for the matching task in the same way as for the duration task. We also analyzed activity during the reaction and movement time (RMT) period, the interval between the “go” cue and the report.

For the distance task, a two-way ANOVA identified cells encoding order–distance conjunctions, feature–distance conjunctions, or both. One factor was whether, on any given trial, S2 had been farther or closer to the reference point than S1; the other factor was whether the blue stimulus had been farther or closer than the red stimulus. An analogous analysis was performed for the duration task, mutatis mutandis. Our previous reports have validated these statistical tests by confirming their principal conclusions with an independent method: multiple regression analysis (Genovesio et al., 2009, 2011). For the matching-to-sample task, a one-way ANOVA identified goal-selective cells (red or blue).

Fig. 2A compares order-based magnitude coding for the two main tasks. On the abscissa, it plots the difference in decision-period activity for the duration-discrimination task, reflecting a preference for trials with a longer S2 (positive values) or those with a longer S1 (negative values). On the ordinate, it plots the analogous difference for the distance-discrimination task: a farther S2 (positive) or a farther S1 (negative). Note that these cells did not encode the order of the stimuli per se, although many other cells in the same areas did so. Fig. S3A shows an example neuron of this type with opposite preferences in the two tasks.

Figure 2. Task contrasts.

A. Scatter plot for order-based relative-magnitude coding in the distance and duration tasks. A1. Red, cells with significant coding only in the distance task; green, cells with significant coding only in the duration task; A2. blue, cells with significant coding in both tasks. Positive values indicate a preference for S2s of greater magnitude. Inset: bar plot counting cells specific to one of the two tasks (red or green) or with activity in both main tasks (blue), with cells having the same preference summed in the dark-colored bar and cells with different preferences summed in the light-colored bar. B. Feature-based coding in the format of A, with positive values indicating cells with a preference for a red stimulus of greater magnitude. B1 and B2 Duration versus distance task; B3 duration versus matching task. The coordinates in parentheses represent one off-scale point in B1 and B3.

Cells with the same preference for relative magnitude in the two tasks, e.g., S1–farther and S1–longer, fell into either the lower left or upper right quadrant of the scatter plots in Fig. 2A. Fig. 2A1 shows data for cells with significant effects in either the duration task (green) or the distance task (red), but not both. Fig. 2A2 shows the results for cells that encoded relative magnitude in both tasks (blue). For all three groups together the preference in one task was independent of that in the other. For the present purposes, the cells with significant magnitude encoding in both main tasks are the most important group, and they showed no correlation in coding preference between the two tasks (r = –0.06, p = 0.606). The bar plot (inset of Fig. 2A) shows that these cells composed 26% of the population (blue bars), which was nearly equally divided between cells preferring the same order–magnitude conjunction in the two tasks (49%, dark blue) and those with the opposite preference (51%, light blue).

Fig. 2B shows corresponding results for feature-based coding. These cells encoded the conjunction of relative magnitude with color and/or shape, although for convenience we refer to them by color. The scatter plot shows each cell's preference for higher-magnitude red stimuli (positive values) or higher-magnitude blue stimuli (negative values). As with order-based magnitude coding, only a minority of cells (31%) encoded relative magnitude in both tasks, but of those 76 cells, 73 (96%) had the same preference in both tasks (inset of Fig. 2B, dark blue bar). Fig. 2B2 shows that among cells with significant coding in both tasks, there was a strong correlation in preferences (r = 0.81, p < 0.001).

Fig. 2B3 shows an analogous comparison for the duration and matching tasks. Of the 76 cells with significant feature-based magnitude coding in both tasks, 51 were also tested in the matching task. Of these 51 cells, 47 (92%) shared the same feature preference in the matching task as in both discrimination tasks. Fig. S4B2 shows the same data as a normalized index.

Because the matching task did not require any decisions about magnitude, we conclude that these cells encoded the nonspatial goal chosen by the monkey on each trial: red or blue. Cells with significant relative-magnitude coding in both main tasks showed a strong correlation between the duration and matching tasks (r = 0.95, p < 0.001), as well as between the distance and matching tasks (r = 0.81, p < 0.001). For the 37 neurons with significant effects in all three tasks, these correlations were r = 0.85, r = 0.97, and r = 0.86, respectively, for duration vs. distance, duration vs. matching, and distance vs. matching (p < 0.001).

Because the monkeys could not know which response to make until the two stimuli reappeared at the end of the D2 delay period (target on, “go”), the goal representation during the decision period specified the object that served as the target of a response and not the motor response per se or the spatial goal. Thus, of the cells showing feature-based coding (Fig. 2B), we found three separate populations of neurons: cells that encoded conjunctions of features with relative distance (e.g., red-farther), cells that encoded conjunctions of features with relative duration (e.g., red-longer), and cells that encoded the chosen goal (e.g., a red target stimulus in all three tasks). Fig. S3B shows a neuron with magnitude coding specific to the duration task; Fig. S3C shows one for the distance task; and Fig. S3D shows a cell that encoded its preferred goal in all three tasks. Fig. S4 confirms these results for normalized indices.

Fig. 3 examines whether the properties just described for the decision period persisted through the S2 and D2 periods. It shows the time course of population activity for order-based (Fig. 3A, black) and feature-based (Fig. 3B1, 2, red and green) relative-magnitude coding. These signals probably encoded perceptual decisions about the sensory inputs, and all of them dissipated as the “go” cue approached (arrows). For the distance task, for example, the signal dissipated during the S2 period and was virtually absent during the D2 period. In contrast, the neuronal population that encoded the goal (Fig. 3B, blue) showed a sustained signal for the distance (Fig. 3B3), duration (Fig. 3B4), and matching (Fig. 3B5) tasks. In all three tasks this signal remained robust throughout the D2 delay period, which ended with the “go” cue. The percentages in the Venn diagram (Fig. 3C) are for cells showing the same preference in a given combination of tasks.

Figure 3.

Population activity averages for cells showing significant effects in the decision period. A (shaded background). Order-based relative-magnitude encoding. Abbreviation: IDP, ideal decision point. Solid line: preferred stimulus–magnitude conjunction; dashed line: anti-preferred conjunction. A1. Distance task. A2. Duration task. All trials are included. Arrows, weakening of the magnitude coding signal. Shading represents SEM. B. Feature-based relative-magnitude coding in the format of A. Color code as in Fig. 2. B1 and B3 Distance task; B2 and B4 duration task; B5 matching task. B2 and B4 show trials with shorter S2s. C. Inset (shaded background): subpopulation of cells in B with significant feature-based effects by ANOVA, recorded in all three tasks. The Venn diagram shows the number of cells with significant effects in various combinations of the tasks. Percentages: cells showing the same preference (red or blue) in overlapping tasks.

After the D2 period, the red and blue stimuli reappeared and the monkeys could then convert their nonspatial choice (a red or blue target stimulus) into a choice between the two possible responses (left or right). Fig. 4 shows the population activity for cells that encoded the nonspatial features of the goal during the RMT period. Note that, averaging backward over time, these cells also carried a robust goal signal during the D2 delay period, prior to the “go” cue. The Venn diagram (Fig. 4C) shows that these cells, like those selected for magnitude encoding during the decision period (Fig. 3C), have the same preferences in all three tasks—with one exception. Of the 75 domain-general cells recorded in the RMT period, only a minority (11 cells for distance, 13 for duration) had domain-specific activity in the earlier, decision period.

Figure 4.

Population activity averages for cells showing significant effects in the RMT period. A. Distance task. B. Duration task. C. Numbers of these cells with significant effects in the format of Fig. 3C. D. Matching task for the 75 cells with significant effects in both the duration and distance tasks. Color code as in Fig. 2.

Discussion

Functional imaging studies have suggested the existence of a domain-general representation of magnitude in a prefrontal–parietal network (Dehaene et al., 2003; Fias et al., 2003; Pinel et al., 2004; Rao et al., 2001; Walsh, 2003). In support of this idea, psychophysical studies have revealed many perceptual interactions between the spatial and temporal domains (Casasanto and Boroditsky, 2008; Gallistel and Gelman, 2000; Magnani et al., 2011; Merritt et al., 2010; Morrone et al., 2005; Walsh, 2003; Xuan et al., 2007). For example, Srinivasan and Carey (2010) found that both adults and 9 month old infants were better able to bind visible lines with the duration of tones when they were relationally equivalent. The interference effects often show an asymmetry. In studies of both adults (Casasanto and Boroditsky, 2008) and children (Casasanto et al., 2010), judgments about the duration of a visual stimulus were influenced by its spatial length, but not the reverse. Language displays the same asymmetry; words that describe time in terms of space are far more common than those that describe space in terms of time (Lakoff and Johnson, 1999). Merchant et al. (2011) likewise found, in monkeys, that previous experience with categorizing distances could affect the categorization of stimulus duration, but not vice versa. However, Merrit (2010) found symmetrical interactions between temporal and spatial judgments in monkeys, so clearly more work is needed on this issue. Symmetrical or not, the spatial and temporal domains clearly interact in perceptual decisions in both humans and monkeys.

Although both imaging and psychophysical studies have suggested a domain-general representation of magnitude, a different organizational principle has emerged from the present findings. We found domain-specific perceptual processing at the single-cell level, with the intermixing of these neurons leading to domain-generality at the regional level (Genovesio et al., 2011). The finding of cell-level domain specificity in the caudal and dorsolateral PF cortex does not rule out the possibility of domain-general mechanisms elsewhere in the brain or in tasks that require magnitude judgments across domains. However, in the parts of PF sampled and in the present tasks, we found no coding of abstract magnitude in individual neurons. The finding of domain specificity at the single-cell level is consistent with the imaging findings, which describe activations in voxels comprising thousands of synapses and neurons. Nearby domain-specific cells would likely create a domain-general signal at the voxel level, and domain-general coding of goals could also contribute to the imaging results. So our findings do not conflict with imaging results, but they seem to clash with the psychophysical findings showing perceptual interactions between the spatial and temporal domains.

Perhaps the cells that encode nonspatial goals can help resolve this apparent discrepancy. These domain-general neurons are intermixed with cells that encode relative magnitude in each domain: spatial and temporal. Goal representations have been reported previously in the PF cortex, where they have been linked to the concept of prospective coding (Kusunoki et al., 2009; Rainer et al., 1999; Saito et al., 2005). The terminology of Schall (2001) might prove helpful here. He distinguished between decisions, which involve the analysis of sensory inputs for perception, and choices among goals or actions. Our findings suggest that the psychophysical interaction across cognitive domains occurs at the level of goal choices, not at the level of perceptual decisions. The cell population that encodes response goals could serve as a shared resource that generates interference. Domain specificity at the level of perceptual decisions and domain generality at the level of goal choices could account for the neuronal, imaging and behavioral data.

Our results also bear on theories of the PF cortex that appeal to a global workspace, domain generality, or multiple cognitive demands (Baars et al., 2003; Duncan, 2010; Wilson et al., 2010). At the perceptual level, interspersed components of neural networks could process information in a specific domain, whereas, at the level of goals, a common processing resource could synthesize these domains into a coherent whole.

Methods

Two adult male rhesus monkeys (Macaca mulatta), 8.5 and 8.0 kg, were used in this study. All procedures were approved by the NIMH ACUC. The monkeys sat 29 cm from a video screen, with three 3 × 2 cm switches within reach. The switches were under the video screen, arranged left to right, separated by 7 cm. Both monkeys used their left hands to contact the keys. The stimulus material consisted of a 0.6° solid white circle, which always appeared in the center of the screen, a solid blue circle 3° in diameter and a solid red 3° ×3 ° square.

Tasks

The monkey began each trial by touching the central switch, which led to the appearance of a white fixation spot at the center of the video screen. The monkey then achieved and maintained central fixation and 400–800 ms elapsed. On each trial of the duration task (Fig. 1A), the blue circle and the red square then appeared in succession at the fixation point, in either order, separated by a variable delay period with only the fixation point. The first stimulus (S1) lasted 200–1200 ms, followed by the first delay period (D1) (400 ms or 800 ms, randomly selected). In a subset of sessions, we added a D1 period of 1200 ms and in another subset we used D1 periods of a fixed 1200 ms duration.

After the D1 period ended, the second stimulus (S2) appeared for 200–1200 ms. The duration of S1 and S2 always differed, and both were selected randomly from a set of stimulus durations varying from 200 ms to 1200 ms in steps of 200 ms. After S2, a second delay period (D2) usually occurred between stimulus offset and the “go” signal. The D2 period lasted either 0 ms, 400 ms, or 800 ms (randomly selected). The red and blue stimuli then reappeared, one 7.8° directly to the left of the fixation point and the other 7.8° to the right, randomly determined. This event served as the “go” cue and terminated the fixation requirement. To receive a reward, the monkeys had to touch the switch below the stimulus that had lasted longer on that trial. Otherwise, the trial terminated with no reward. The monkeys had 6 s to respond, but in practice both monkeys did so in less than 500 ms (Fig. S2). Overall, S1 and S2 had an equal likelihood of lasting longer on any given trial.

Each trial of the distance task (Fig. 1B) also began when the monkeys touched the central key. The white circle then appeared at the center of the screen. In the distance task, it served as a reference point rather than as a fixation point, as it did for the duration task. After either 400 ms or 800 ms, the red square and the blue circle appeared in succession, in a randomly determined order, for 1.0 s each. One stimulus appeared directly above the reference point, the other appeared directly below it, randomly determined. The relevant stimulus dimension was the relative distance of each stimulus from the reference point. In screen distance, the stimuli ranged from 8 mm to 48 mm from the reference point, in steps of 8 mm, which corresponded to 1.6°, 3.2°, 4.7°, 6.3°, 7.9° and 9.4° of visual angle. A delay period followed both S1 and S2. A randomly selected 400 ms or 800 ms delay period (D1) usually followed S1, although in one set of sessions we added a D1 period of 1200 ms and in another we used fixed D1 periods of 1200 ms. The D2 period in the distance task matched that in the duration task, as did the appearance of the choice stimuli. After this “go” cue, the monkeys chose the stimulus that had appeared farthest from the reference point in order to receive a reward.

The matching task (Fig. 1C) closely matched the duration task, both in requiring fixation at the center of the screen and in the durations and locations of the S1, D1, S2 and D2 periods. The matching task differed in that the same stimulus, either the red square or the blue circle, appeared as both S1 and S2. After S2, the matching task was identical to both the duration and distance tasks. After the “go” cue, the monkeys had to touch the switch below the stimulus that had appeared twice on that trial in order to receive a reward. In all three tasks, acoustic feedback signaled an error, and an intertrial interval of 300 ms followed both correct and incorrect choices. All the three tasks were run in consecutive blocks with no fixed order.

Surgery

Recording chambers were implanted over the exposed dura mater of the left frontal lobe, along with head restraint devices, using aseptic techniques and isofluorane anesthesia (1–3%, to effect). Monkey 1 had two 18 mm diameter chambers, and Monkey 2 had a single 27 × 36 mm chamber.

Data collection

We recorded eye position with an infrared oculometer (Arrington recording, Scottsdale, AZ), and single-cell activity was recorded using quartz-insulated platinum-iridium electrodes (0.5–1.5 MΩ at 1 KHz) positioned by a 16-electrode drive assembly (Thomas Recording, Giessen, Germany). The electrodes occurred in a concentric array with 518 μm spacing. Spikes were discriminated online using Multichannel Acquisition Processor (Plexon, Dallas, TX) and confirmed with Off Line Sorter (Plexon) based on principal component analysis, minimal interspike intervals, and clearly differentiated waveforms inspected individually for every isolated neuron.

Neurophysiological analysis

Our previous reports used the same neuronal dataset to analyze activity during either the distance (Genovesio et al., 2011) or duration (Genovesio et al., 2009) task. The present report compares activity in these two tasks, at the single-cell level, along with activity in the matching task. We focused the present analysis on the decision and RMT periods. Order- and feature-based relative-magnitude coding was assessed for all three tasks with two-way ANOVA, as described in the Results, using SPSS (Chicago, IL) and custom programs.

To compare the magnitude of cell preferences, we calculated activity (A) differences for each pair of tasks. For the distance task, one such measure involved order-based magnitude coding: A_S2 – A_S1, where A_S2 was the average discharge rate on trials when S2 was farther from the reference point, and A_S1 was the average rate when S1 was farther. For feature-based magnitude coding, we calculated A_Red – A_Blue, where A_Red and A_Blue indicate the activity rates for trials when the red or blue stimuli, respectively, appeared farther from the reference point. Analogous differences were calculated for the duration task. In the matching task we calculated only A_Red – A_Blue. Fig. S4 presents normalized preference indices in the form of contrast ratios: (A_Red – A_Blue)_/(A_Red+ A_Blue) or (A_S2 – A_S1)_/(A_S2+ A_S1).

We also used population activity averages to assess the magnitude of coding. We designated the condition associated with the highest discharge rate (e.g., S2 longer or farther) as the preferred condition and the one with the lowest activity as the antipreferred condition, and we calculated the population averages as the means of separately calculated single-cell averages, along with the SEM.

Histological analysis

Near the end of recording, we made electrolytic marking lesions (15 μA for 10 s). Ten days later, the monkeys were deeply anesthetized and perfused with 10% formol saline. We plotted recording sites on coronal Nissl-stained sections, by reference to the recovered marking lesions, pins inserted during the perfusion, and structural magnetic resonance images. Although the entry points for more posterior recordings (Fig. 1D) make it appear that many cells were located in the postarcuate cortex, track reconstructions based on the angle and depth of penetrations indicated that nearly all recordings in caudal PF came from prearcuate cortex, which corresponds to area 8.