Memory fields of neurons in the primate prefrontal cortex

Abstract

Many prefrontal (PF) neurons convey information about both an object’s identity (what) and its location (where). To explore how they represent conjunctions of what and where, we explored the receptive fields of their mnemonic activity (i.e., their “memory fields”) by requiring monkeys to remember both an object and its location at many positions throughout a wide portion of central vision. Many PF neurons conveyed object information and had highly localized memory fields that emphasized the contralateral, but not necessarily foveal, visual field. These results indicate that PF neurons can simultaneously convey precise location and object information and thus may play a role in constructing a unified representation of a visual scene.

Understanding the structure and organization of receptive fields has yielded important insights into visual system function. Yet, despite the fact that it receives a massive input from visual cortex (1, 2), little is known about receptive fields of neurons in the prefrontal (PF) cortex, a region that plays a central role in planning complex, intentional behavior.

While it has been shown that many PF neurons show selectivity for the location of a behaviorally relevant cue (3, 4), detailed mapping of their receptive fields has been conducted in monkeys passively viewing stimuli (5, 6) or performing uncomplicated oculomotor tasks (7). It has become increasingly apparent, however, that visual receptive fields throughout the neocortex can be strongly influenced by task demands (8–13). Thus, it is also important to characterize PF receptive fields in the context of tasks that exercise the functions of the PF cortex. Further, most studies have explored the spatial properties of PF neurons by using simple stimuli such as bars and spots of light. Many PF neurons, however, show selectivity for complex stimuli that resemble the objects monkeys encounter in their experiences outside the neurophysiological laboratory (14, 15). Little is known about how or whether PF neurons convey their spatial attributes.

The PF cortex plays an important role in a variety of functions critical for complex behavior, such as attention, response selection, and rule learning (16–20). Critical for these functions is the temporary maintenance of behaviorally relevant information (21, 22). In tasks that require monkeys to hold a stimulus in memory over a brief delay, PF neurons show high levels of sustained activity that maintains stimulus-related information (4, 12, 14, 23–27). Functional imaging studies also indicate sustained activation in the human PF cortex during memory tasks (28, 29). It has been shown recently that many PF neurons exhibiting this “delay activity” can convey information about an object and its location (12, 27). Neurons processing both kinds of information may play a role in maintaining information about conjunctions of object identity (what) and location (where). How PF neurons represent their conjunction is poorly understood because so little is known about the receptive fields of object-selective PF neurons. Previous studies have tested only PF responses to objects appearing at a few (2–4) visual field locations. Here, we explored receptive field properties of delay activity of PF neurons (i.e., their “memory fields”) by requiring monkeys to remember which object of a small set had appeared in which of 25 visual field locations.

METHODS

Animals.

Recordings were made in the lateral prefrontal cortex of two adult rhesus monkeys (Macaca mulatta) using a grid system (Crist Instrument Co., Damascus, MD) with 1-mm spacing between adjacent locations. Recording sites were localized by using magnetic resonance imaging. Using a previously described surgical procedure (30), the monkeys were affixed with recording cylinders above the lateral prefrontal cortex and with a scleral search coil for monitoring eye position. All animal care and experimental procedures were approved by MIT Animal Care and Use Committee and complied with Public Health Service Policy on the use of laboratory animals.

Task and Stimuli.

Monkeys performed a delayed-match-to-object-and-place (DMOP) task (Fig. 1a). Each trial began with the monkeys fixating a spot for 1,000 ms. They were required to maintain fixation for the duration of the trial. A sample object (2° in size) was presented for 1,000 ms at one of 25 visual field locations. These locations formed a 5 × 5 matrix, centered on the fovea, with 5° separating the locations on which the stimuli were centered. The matrix thus spanned about 20° of central vision, a region limited by the monkeys’ ability to identify peripheral objects. The monkeys needed to remember both the identity and location of the sample object. After a 1-s delay, a test object was presented. It was either a “spatial nonmatch” (the sample object presented in a different location), an “object nonmatch” (a different object than the sample but appearing in the same location), or a “match” (the sample object appearing in the same location). If a match, monkeys released a response lever within 1,000 ms to receive a juice reward. Two to five objects (typically five) were used as samples. The objects were square, “real world” pictures, 1–2° on a side, easily distinguishable from each other and from the (black) background. All contained complex shapes and were multicolored. The same objects were used throughout the experiment.

Figure 1.

(a) Sequence of trial events. Each trial began when the monkey grabbed a response lever and fixated a small fixation target at the center of a computer screen. (b) Recording sites. Each symbol represents a recording site where neurons with what, where, or what-and-where delay activity were found. Typically, several neurons were found at the same site; hence, many symbols overlap and some symbols indicate more than one neuron. Data are combined across both monkeys. A.S., arcuate sulcus; P.S., principal sulcus. All recordings were from the surface of the lateral PF cortex.

Data Analysis.

Delay activity was analyzed over the last 800 ms of the 1,000-ms delay after the sample. We did not include the first part of the delay so that responses related to the offset of the sample would be excluded. Visual responses to the sample were analyzed over an interval from 100 to 1,000 ms after sample onset. Baseline activity was calculated over a 900-ms time interval beginning 100 ms after fixation and ending at sample onset. Activity was appraised by using ANOVAs with a significance level at P < 0.05. To determine whether activity reflected the target object, its location, or both, a two-factor ANOVA was used. One factor was which object was the sample (OBJECT factor) and the other was its location (LOCATION factor). We collected about 10 trials of data for every experimental condition.

The receptive or memory field was defined as the area that elicited activity greater than half of the maximum response. We used a linear interpolation to estimate the level of activity between adjacent tested locations (31). Size was found by calculating its diameter, defined as the square root of the area. While size is reported for neurons whose memory fields (MFs) or receptive fields were wholly within the locations tested, similar results were obtained when all neurons were included. The field center was its “center of mass,” i.e., the geometric center weighted by the level of activity elicited by each location. Using the geometric center alone yielded similar results.

RESULTS

We recorded the activity of 184 neurons from the lateral PF cortex of two monkeys (Fig. 1b). During the delay interval between sample and test object presentation, many neurons (149/184, or 81%) showed activity that reflected either the sample object, its location, or both (ANOVAs, P < 0.05; Table 1). The “delay activity” of about half of these neurons (68/149, or 46%) conveyed what and where information simultaneously. It depended on both the object used as a sample and its location (two-way ANOVA, P < 0.05). We called them “what-and-where” neurons. Most of the remaining neurons (73/149, or 49%) were selective for the location of the sample object only and were termed “where” neurons. There were just a few “what” neurons (8/149, or 5%) selective for the sample object only.

Table 1.

Properties of neurons

	Sample interval			Delay interval
	Where only	What only	What and Where	Where only	What only	What and Where
Number of cells	61	3	86	73	8	68
Receptive field includes fovea	27	3	42	19	8	20
Mean baseline firing rate, spikes per s	16.4	3.7	16.6	16.5	11.7	16.0
Mean firing rate to optimal stimulus, spikes per s	38.1	7.2	33.8	34.9	38.9	33.9
Cells with MFs wholly within tested locations
Number of cells	22	—	36	29	—	31
Mean RF size, °	10.5	—	10.8	9.8	—	9.3
Mean eccentricity of RF center, °	4.5	—	3.3	5.1	—	5.6

Cell counts are based on ANOVA (see Methods), evaluated at P < 0.05. The receptive and memory field sizes were calculated by averaging each neuron’s activity across all objects. n = 184 cells. 

Object and location selectivity for a single what-and-where neuron is illustrated in Fig. 2 a and b. Fig. 2a shows histograms of the neuron’s activity to a preferred object appearing at each of the 25 tested locations. Note that this neuron was highly spatially selective; it showed strong sustained activity only when the sample object appeared at the two locations directly above fixation. Fig. 2b shows that this activity was also highly object-selective. A preferred object elicited robust sustained activity while a nonpreferred object elicited little or none. In fact, on average, what-and-where neurons showed a 53% increase in delay activity after a preferred object over that after a nonpreferred object. Fig. 2c shows MF plots of the delay activity of 18 what-and-where neurons. All of the neurons were highly object-selective; they showed robust delay activity to preferred objects and relatively little or no activity to nonpreferred objects. They were also highly spatially selective; delay activity was only evident when the objects had appeared in certain visual field locations. Note that even when neurons had relatively large MFs, they often had “hot spots,” MF subregions that elicited more delay activity than other regions (Fig. 2c). Thus, they could convey spatial information even within their MFs.

Figure 2.

(a) Histograms of a single PF neuron’s activity to an object appearing at each of the 25 tested locations. The line to the left of each histogram shows time of sample onset, and the line in the middle denotes sample offset. Bin width, 40 ms. The y-axis indicates firing rate in spikes per second, and the x-axis indicates time. The time scale for each histogram is identical to the histogram shown in b. (b) Activity of the same neuron to a preferred and nonpreferred object appearing within the neuron’s MF. (c) MF plots of 18 what-and-where PF neurons. Preferred and nonpreferred refer to the objects used to map the MFs shown in each square. Each square represents the tested 20° of central vision with fixation at the center. For each neuron, the blue-to-red color map indicates the level of delay activity elicited by a preferred or nonpreferred object appearing at that region of visual field. Blue indicates the neuron’s baseline level of activity, and red indicates the neuron’s maximum delay activity. A scale bar indicates, for each neuron, the relationship between color and firing rate in spikes per second. The neuron illustrated in a and b is depicted in the middle row, second from the left of c.

Neuronal properties are summarized in Table 1. The MFs of what-and-where neurons were highly spatially selective. The average MF diameter of the 31 what-and-where neurons whose MFs were contained wholly within the locations tested was about 9°. In fact, nearly half of them (14/31, or 45%) were sensitive to only a quarter or fewer of the 25 tested locations and almost all (29/31, or 94%) were sensitive to less than half of the locations. The MFs of what-and-where neurons did not differ in size from those of where neurons (t test, P = 0.57). Not surprisingly, a given what-and-where neuron showed similar MFs to different objects. For example, Fig. 2c shows that when nonpreferred objects elicited weak delay activity it was typically from similar locations as the stronger activity elicited by the preferred stimulus. For each neuron, we calculated the center of each object’s MF. Across the population of what-and-where neurons, the average difference between the MF centers to individual objects and the average MF center (averaged across all objects) was only 3°, less than the distance between adjacent locations. Thus, these neurons showed little scatter in the location of their MFs. Also, a given neuron’s sensory receptive field was similar to its memory field. The average difference between each neuron’s receptive field center calculated from sample interval activity and the neuron’s MF center was only 4.8°, also less than the distance between adjacent locations. Many of the neurons (43/68) showed an “on-response,” a brief phasic burst of activity shortly after sample onset, that was typically less selective than sustained activity (e.g., Fig. 2 a and b).

MF locations were biased toward contralateral visual space (Fig. 3). Significantly more what-and-where neurons had MF centers in the contralateral field (44/68 or 65%) than in the ipsilateral field (24/68, or 35%; χ², P = 0.015). Notably, however, there was no emphasis of foveal vision. Only 20 of 68 what-and-where delay neurons (29%) had MFs that included the fovea, and only a few (4, or 6%) showed maximal delay activity after a foveal sample object. By contrast, neurons in the inferior temporal (IT) cortex, which provides the lateral PF cortex with object information (32), have receptive fields that do emphasize foveal vision. IT receptive fields invariably include the fovea, and IT neurons typically respond best to foveal stimulation (33, 34).

Figure 3.

Plot illustrating the location of the MFs of 68 what-and-where neurons. The figure shows an overlay of the MF of all 68 neurons, with the color indicating how many neurons had an MF at that visual field location. The square represents the tested 20° of central vision. Fixation was at the center of the square. The right side was contralateral to the recorded neurons, and the left was ipsilateral.

Examination of the location of recorded cells revealed that in the posterior locations near the arcuate sulcus, where cells tended to predominate (Fig. 1b). These locations are near or in the frontal eye fields, which contain many saccade direction-selective neurons (35). In the more anterior sites around the principal sulcus, however, all three types of cells (what-and-where, what, where) were intermixed and often found at the same recording locations. Other than the contralateral bias, there was no obvious topographical organization in MF location.

DISCUSSION

The results of this study indicate that when monkeys need to remember an object and its location, the activity of many lateral prefrontal neurons reflects this combined what and where information. What-and-where neurons were able to simultaneously communicate the identity and location of a sample object throughout a large portion of the visual field at and near the fovea as well as in the periphery. The MFs of these neurons were similar in size and location to those of where neurons, and their spatial selectivity appears to be similar to those of neurons engaged by memory-guided saccadic eye movements (4). In this study, the task required a nonspatial behavioral response (bar release to a “match”). Thus, the spatial information conveyed by these neurons was likely to be sensory- and not motor-related (36, 37). Finally, unlike neurons in the IT cortex, only a minority of PF what-and-where neurons were sensitive to foveal stimulation. Indeed, the MFs of most of these neurons were entirely extrafoveal. Thus, they seem well suited to the task demand to represent objects and their locations throughout a wide portion of the visual field.

There has been some question about the degree of separation of object and spatial processing in the PF cortex. Some evidence suggests that ventrolateral PF neurons tend to have delay activity that is specialized for objects whereas dorsolateral PF neurons tend to have delay activity specialized for spatial information (26). Above and beyond any regional biases, however, it is apparent that both areas contain neurons that can process what and/or where (12, 27, 38, 39). Functional imaging studies in humans also indicate that the same PF regions can be activated by both object and spatial tasks (40, 41) and that the dorsolateral PF cortex is activated during nonspatial tasks (42). Even studies that find some separation of PF regions activated by object and spatial processing also find large regions of overlap (43). In this study, we found that except for the posterior recording sites there was an intermixing of object and spatial signals both on the regional and single-cell level. We did not record throughout the entire PF cortex and, of course, cannot know whether there is any region in which object processing may predominate.

In the visual system, there are clear differences between neocortical areas that appear to process object and spatial information (44, 45). To analyze coherent scenes, however, ultimately some neurons must have access to both types of information. What and where could combine through anatomical interconnections between the what and where pathways within the visual system (46–48) and/or between PF regions interconnected with these pathways (49, 50). Consistent with these connections are observations that visual cortical areas thought to be relatively specialized for processing object or spatial information also have neurons selective for, or modulated by, the other attribute (8, 51, 52). Also, some studies indicate the object and spatial information needed for perception are unified within one visual cortical pathway while the other pathway processes visual information needed for action (53). Indeed, ventral pathway neurons selective for objects do carry spatial information (33, 34, 54). Regardless of where they are first integrated, the present study shows that PF neurons can represent precise conjunctions of what and where, an attribute useful for the high-level cognitive functions that depend on the PF cortex.

ABBREVIATIONS

PF

prefrontal

MF

memory field

IT

inferior temporal