Maintenance of Spatial and Motor Codes during Oculomotor Delayed Response Tasks

Abstract

The most compelling neural evidence for working memory is persistent neuronal activity bridging past sensory cues and their contingent future motor acts. This observation, however, does not answer what is actually being remembered or coded for by this activity. To address this fundamental issue, we imaged the human brain during maintenance of spatial locations and varied whether the memory-guided saccade was selected before or after the delay. An oculomotor delayed matching-to-sample task (match) was used to measure maintained motor intention because the direction of the forthcoming saccade was known throughout the delay. We used a nonmatching-to-sample task (nonmatch) in which the saccade was unpredictable to measure maintained spatial attention. Oculomotor areas were more active during match delays, and posterior parietal cortex and inferior frontal cortex were more active during nonmatch delays. Additionally, the fidelity of the memory was predicted by the delay-period activity of the frontal eye fields; the magnitude of delay-period activity correlated with the accuracy of the memory-guided saccade. Experimentally controlling response selection allowed us to functionally separate nodes of a network of frontal and parietal areas that usually coactivate in studies of working memory. We propose that different nodes in this network maintain different representational codes, motor and spatial. Which code is being represented by sustained neural activity is biased by when in the transformation from perception to action the response can be selected.

Introduction

Working memory allows animals to use information that is not currently present in the environment but is vital to adaptive behavior. Sustained neural activity during the delay period between a sensory cue, say the position of a briefly flashed spot of light, and a later motor response, say a shift of gaze to the remembered location, is compelling evidence that this activity is a memory representation (Fuster and Alexander, 1971; Kubota and Niki, 1971; Gnadt and Andersen, 1988; Funahashi et al., 1989). However, what is actually being represented during working memory delays? Or in other words, what is being remembered? During a memory delay, one can look back to a past perceptual event, a “retrospective” sensory code, or can look forward to a future action, a “prospective” motor code, to link events that are separated in time but are contingent on one another (Boussaoud and Wise, 1993; Funahashi et al., 1993; Quintana and Fuster, 1999; Rainer et al., 1999; D'Esposito et al., 2000b).

In this study, we use event-related functional magnetic resonance imaging (fMRI) to clarify the nature of the representation maintained by sustained neural activity. We scanned subjects while they performed a classic oculomotor delayed-response task (see Fig. 1, Match), in which, on each trial, they simply made a saccade that shifted gaze to a location that matched the remembered location of a sample cue presented 10 sec earlier. This type of task has been used successfully in numerous investigations to separately analyze visual-, mnemonic-, and motor-related activity (Bruce and Goldberg, 1985; Gnadt and Andersen, 1988; Funahashi et al., 1990; Chafee and Goldman-Rakic, 1998). In the matching task, subjects can plan a saccade to acquire the target and simply delay the initiation of the saccade until after the delay. Delay-period activity should reflect this strategy, the maintenance of a prospective motor code, or motor intention. For comparison purposes, subjects also performed an oculomotor delayed nonmatching-to-sample task (Fig. 1, Non-Match) in which a memory-guided saccade was made to a location that did not match the location of the sample. Because a saccade was never made to the sample location and the nonmatching location was unpredictable, we reasoned that this manipulation biased the subject from maintaining a motor code during the delay. Instead, it encourages the maintenance of a retrospective sensory code, or sustained spatial attention.

Figure 1.

Schematic depiction of the oculomotor delayed-response tasks in which subjects used the location of the cue to make a memory-guided saccade. Both the matching-to-sample (top) and nonmatching-to-sample (bottom) tasks began with the brief presentation of a small green sample cue while the subject maintained central fixation. The cue appeared randomly at 1 of 16 possible locations at a 10° radius, none of which lie on the cardinal axes. A masking pattern was then briefly presented to disrupt iconic visual memory, followed by a long unfilled memory delay. During matching trials, the subject made a memory-guided saccade (depicted by the thin black line) after the disappearance of the fixation cue marking the end of the delay. Feedback was provided by the re-presentation of the cue. At this point, the subject corrected any errors by shifting gaze to the cue. The difference between the endpoint fixation after the memory-guided saccade and the fixation to acquire the feedback cue was used as an index of memory accuracy. During nonmatching trials, the subject made a saccade to the green square that did not match the location of the sample cue. ITI, Intertrial interval.

Decomposing the nature of persistent neural signals during working memory maintenance can help resolve controversies and disagreements regarding the functional organization of the prefrontal cortex (Courtney et al., 1998; Curtis et al., 2000; D'Esposito et al., 2000a; Levy and Goldman-Rakic, 2000; Owen, 2000; Postle et al., 2000; Rowe et al., 2000; Leung et al., 2002; Brown et al., 2004). In an effort to characterize what is actually being represented by persistent activity or “remembered,” we rely on three key experimental methods. First, we use an event-related fMRI design that allowed us to separately estimate delay-period activity from activity-related to stimulus-encoding and memory-guided motor responses. Second, we experimentally control when in the evolution from perception to action the memory-guided saccade can be selected, which biases the very nature of what is being remembered during the delay. Third, we record eye position in the scanner that allows us to correlate saccadic performance with maintenance-related activity. The combined result is that we can better infer whether the persistent delay-period signals are coding for motor intentions or spatial attention.

Parts of this paper have been published previously is abstract form (Curtis and D'Esposito, 2002).

Materials and Methods

Experimental methods. Fifteen healthy participants (eight females; ages 18-33 years), who gave informed consent according to procedures approved by the University of California, performed three runs of each of the oculomotor delayed-response tasks, matching and nonmatching to sample, as depicted in Figure 1. Both the matching-to-sample and nonmatching-to-sample tasks began with the brief presentation of a small green sample cue for 100 msec while the subject maintained central fixation. The cue appeared randomly at 1 of 16 possible locations at a 10° radius, none of which lie on the cardinal axes. A masking pattern was then briefly presented for 50 msec to disrupt iconic visual memory, followed by a long 9850 msec unfilled memory delay. During matching trials, the subject made a memory-guided saccade (depicted by the thin black line) after the disappearance of the fixation cue marking the end of the delay. Feedback was provided after 2000 msec by the re-presentation of the cue. At this point, the subject corrected any errors by shifting gaze to the cue. During nonmatching trials, the subject made a saccade to the green square that did not match the location of the sample cue. The order of the runs was counterbalanced and yielded a total of 48 trials of each type (one run of 16 trials was lost for one subject). The matching-to-sample and nonmatching-to-sample tasks were performed in separate runs to encourage the use of a stationary strategy or set.

Neuroimaging methods. T2^*-weighted echo planar images (EPI) sensitive to blood oxygenation level-dependent (BOLD) contrasts were acquired at 4 tesla with a Varian INOVA MR scanner and a TEM-send and receive RF head coil using a two-shot gradient echo-EPI sequence [22.4 cm square field of view with a 64 × 64 matrix size, resulting in an in-plane resolution of 3.5 × 3.5 mm for each of 18 3.5 mm axial slices with no interslice gap; repetition time (TR), 1 sec per half of k-space (2 sec total); echo time, 28 msec; flip angle, 20°]. Each shot was interpolated with its neighbor, resulting in an effective TR of 1 sec. Functional volumes were acquired during six runs lasting 448 sec each, resulting in 2688 volumes total covering the dorsal cortex. High-resolution MP-Flash three-dimensional T1-weighted scans were acquired for anatomical localization.

Oculomotor methods. Eye position was monitored in the scanner at 60 Hz with an infrared videographic camera equipped with a telephoto lens (model 504LRO; Applied Sciences Laboratories, Bedford, MA) that focused on the right eye viewed from a small dielectric flat surface mirror mounted inside the RF coil. Nine-point calibrations were performed at the beginning of the session and between runs when necessary. Eye-movement data were calibrated then transformed to degrees of visual angle using a third-order polynomial algorithm that fit eye positions to known spatial positions and scored offline with in-house software (GRAPES). The difference between the endpoint fixation after the memory-guided saccade and the fixation to acquire the feedback cue was used as an index of memory accuracy. Saccadic reaction times were estimated with semiautomatic routines that relied on the acceleration of the eye to determine the onset of saccades. Eye position data were not available for two subjects because of technical difficulties.

Data preprocessing and analysis. For all participants, a hemodynamic response function (HRF) was empirically derived (Aguirre et al., 1998) in response to 20 saccades made to flickering checkerboards (20 Hz) briefly presented (200 msec) to the left or right hemifield. The HRF used was derived from frontal eye fields (FEF) and did not differ in shape from HRFs derived from the supplementary eye fields (SEF) or the motor cortex. We modeled fMRI signal changes evoked by each epoch of the trial with a covariate shaped like the HRF by convolving it with each independent variable (spatial cue, delay, saccade response, and feedback) (Zarahn et al., 1997) and entering the result into the modified general linear model (GLM) (Worsley and Friston, 1995) for analysis using VoxBo (www.voxbo.org). In one GLM, the delay was further divided into an early and late period, in which the early delay covariate was placed 3900 msec after the short presentation of the stimulus cue and the late delay covariate was placed 6900 msec after the stimulus cue. For estimating general delay-period activity regardless of trail type (i.e., match or nonmatch) or epoch (i.e., early or late), a GLM was conducted that coded all of these trial periods as the same event, and delay activity was tested against the intertrial interval baseline. Group statistical parametric maps (t statistics) of key contrasts were generated after the individual subject data were spatially normalized into standard atlas space (Montreal Neurological Institute reference brain) using routines from SPM99 (www.fil.ion.ucl.ac.uk/spm), resampled to 2 mm isotropic voxels, and spatially smoothed with a Gaussian kernel to yield 8 mm full-width half-maximal. Statistical significance was set to a height threshold of p < 0.001 (t > 4.14) and an extent threshold of at least 20 contiguous suprathreshold voxels. Correlations between memory-guided saccade accuracy (Fig. 1) on each trial with the magnitude of the combined delay-period parameter estimates for each brain voxel were computed separately for nine of the subjects. The quality of the eye position data for three of the subjects was not sufficient to reliably assess accuracy on enough trials to be useful; signal instabilities and intermittent loss on a number of trials disqualified these subjects from the correlation analysis. One subject misunderstood task instructions and did not correct their errors, and therefore reliable scoring was impossible. With these subjects, nonetheless, we were able to reliably determine whether they had made correct or incorrect memory-guided saccades, but we could not ascertain how accurate they were on matching trials. Finally, correlation maps were transformed to z maps for each subject using Fischer's r-to-z transformation, spatially normalized, and subjected to a groupwise random effects analysis. Average time series, plus the SEM, were computed by taking the average of all voxels within each region of interest for each time point across all subjects. These values were converted into percentage signal change and plotted beginning at the start of the fixation preparation cue.

Results

Oculomotor results

On the matching-to-sample task, the average positional error of memory-guided saccades was 2.13 ± 0.43°. On the nonmatchingto-sample task, erroneous saccades to the matching cue were rare (3.37% of trials; range, 0-8.33%) and were almost always corrected before the feedback was given. Importantly, the mean saccadic reaction times between the matching (mean ± SD, 315.2 ± 59.2 msec) and nonmatching-to-sample (mean ± SD, 365.0 ± 67.5 msec) tasks differed significantly (t₍₉₎ = 2.34; p < 0.05). The Pearson correlation between saccadic reaction times and positional error on the matching task was not significant (r = 0.02; NS). The faster saccadic reaction times for match trials is consistent with the hypothesis that subjects prepared the memory-guided saccade during the delay.

Match versus nonmatch: stimulus cue period

We first compare differences in brain activation between the match and nonmatch trials at the time when the stimulus cue was presented for encoding. With regard to visual stimulation, this period is identical for match and nonmatch trials. However, the subject knew whether or not this location was the target for a memory-guided saccade made 10 sec later in the trial. In the case of match trials, the future memory-guided saccade vector can be selected shortly after the cue is presented. In both cases, they must remember the location of the cue to later execute the memory-guided saccade. The right FEF and the left middle frontal gyrus (BA 44/46 border) showed greater activity on match than non-match trials. As other studies have done, we defined the FEF as a region limited to the dorsolateral and dorsomedial segments of the precentral sulcus, near but not extending into the superior frontal sulcus (Paus, 1996; Courtney et al., 1998; Luna et al., 1998; Postle et al., 2000; Lobel et al., 2001). Bilateral posterior parietal cortex in and around the intraparietal sulcus (IPS) showed greater activity on nonmatch than match trials (Fig. 2, Table 1). Note that the IPS and FEF were active on both match and non-match trials; however, the level of activation depended on the task instructions.

Figure 2.

Stimulus cue. Statistical parametric t maps contrasting stimulus cue activity between the oculomotor delayed matching-to-sample and nonmatching-to-sample tasks. Warm colors depict regions with greater cue period activity on matching than nonmatching trials. Cool colors depict regions with greater cue period activity on nonmatching than matching trials. MFG, Middle frontal gyrus.

Table 1.

Matching-to-sample versus nonmatching-to-sample task activations

		Peak MNI coordinates
Region	Hemisphere	x	y	z	t value	BA
Stimulus cue period
Match > nonmatch
MFG	Left	−43	24	46	5.64	44/46
FEF	Right	64	−2	38	6.19	6
Nonmatch > match
IPS	Left	−54	−50	46	6.70	7/40
IPS	Right	50	−46	40	6.11	7/40
Early delay period
Match > nonmatch
Paracentral lobule	Right	8	−56	68	6.34	5
Precuneus	Right	10	−34	68	6.49	4
Pre-SEF	Right	5	6	56	6.35	6
FEF	Right	48	0	54	6.49	6
MFG	Right	42	23	56	4.57	9
rIPL	Right	50	−36	56	5.53	2/40
pIFS	Right	44	24	42	10.10	44
SFS	Right	23	46	42	5.01	9
pIFS	Left	−32	14	38	5.29	44
SFS	Left	−18	58	32	5.10	46/10
MFG	Left	−30	48	32	4.29	46
pCingulate	Left	−8	−50	30	5.41	23
pCingulate	Right	14	−51	32	4.30	23
Nonmatch > match
Cuneus	Right	5	−88	31	5.15	18
Late delay period
Match > nonmatch
SEF		0	−10	55	4.21	6
postCS	Right	50	−40	64	5.31	2
Nonmatch > match
IPS	Right	17	−72	58	5.22	7
IPS	Right	52	−50	52	5.71	40
IPS	Left	−50	−52	54	5.59	40
pIFS	Right	46	22	28	6.84	44/45
pIFS	Left	−44	22	30	5.34	44/45
IPS	Right	17	−72	58	5.22	7
Memory-guided saccade period
Match > nonmatch
SEF	Left	−8	−8	58	5.17	6
FEF	Right	52	−10	53	7.87	6
rIPL	Right	52	−33	58	4.93	2
SEF	Right	6	0	46	5.45	6
POS	Left	−20	−94	35	6.85	18
POS	Right	18	−94	34	6.81	18
iPCS	Left	−58	2	27	6.11	6/44
iPCS	Right	52	2	28	6.54	6/44
Nonmatch > match
Precuneus	Right	6	−56	54	8.34	5/7
preSMA	Right	2	26	52	5.06	8
MFG	Right	26	24	43	5.80	9/46
MFG	Left	−40	22	40	7.32	9
MFG	Right	28	40	36	8.02	46
IPS	Right	46	−43	40	7.87	40
Angular gyrus	Left	−42	−52	22	6.42	39

MFG, Middle frontal gyrus; SFS, superior frontal sulcus; iPCS, inferior precentral sulcus; postCS, postcentral sulcus; preSMA, presupplementary motor area; pCingulate, posterior cingulated; rIPL, rostral inferior parietal lobule; POS, parietal-occipital sulcus.

Delay-period activity

To estimate which brain areas show some evidence for sustained activity during the delay, we contrasted the intertrial interval baseline with the linear combination of all of the delay regressors, including the early and late delay components for both match and nonmatch trials. A widespread network of regions that have been implicated in spatial working memory are active for this contrast (Fig. 3). Notably, even under such low-memory load demands, the middle frontal gyrus [BA 46, (40, 36, 44); BA 8, (36, 6, 64)] is active, as are other parts of the frontal and parietal cortex. Bilateral FEF [right, (26, -8, 49); left, (-25, -7, 53)] was also present deep in the fundus of the precentral sulcus and is not visible on the surface rendering in Figure 3. Area 8 was easily distinguished from FEF in these data. FEF foci were limited to the precentral sulcus near the posterior branch of the superior frontal sulcus but not extending along it. Area 8 was in the middle frontal gyrus more anterior and lateral than the FEF foci. In addition, the right posterior inferior frontal gyrus (pIFG) (60, 8 30) and bilateral IPS [right, (38, -64, 62); left, (-36, -65, 65)] also showed significant maintenance-related activity.

Figure 3.

Statistical parametric t map of delay-period activity overlaid on a surface rendering of brain. The contrast was formed by collapsing across both oculomotor delayed matching-to-sample and nonmatching-to-sample delay periods. This delay-period contrast represents a linear combination of the delay-period regressors.

The full delay-period analysis was also performed for the match and nonmatch conditions separately. Very similar significant delay-specific foci in FEF, IPS, BA 8, and pIFG were found for both conditions. Delay activity in right BA 46 showed a similar focus of activity in the separate compared with the combined match and nonmatch conditions but did not reach statistical significance for the match or nonmatch.

Match versus nonmatch: delay period

We now compare differences in brain activation between the match and nonmatch trials during the delay period, when maintenance processes operate. If our task manipulation was successful at invoking a different maintenance strategy, then we expect to see differences in brain activity related to the different strategy. Indeed, many areas were sensitive to the match-nonmatch manipulation. During the early delay period, the FEF, pre-SEF, posterior inferior frontal sulcus (pIFS), and middle frontal gyrus, including BA 46, 9/46, all showed greater activity on matching than nonmatching trials (Fig. 4, Table 1). Only a small area in the right cuneus showed greater early delay-period activity on non-matching than matching trials (Table 1). During the late delay period, much greater activity was found on nonmatching trials. Several foci in the right and left posterior parietal cortex in and around the IPS showed greater activity on nonmatch than match trials. Bilateral pIFS also showed greater nonmatch than match late delay-period activity (Fig. 4, Table 1). The SEF were more active during match than nonmatch trials during this period. Note that this SEF foci was just ∼1 cm caudal to the pre-SEF foci (Grosbras et al., 2001) that was differentially active in the early delay-period contrast. Our task manipulation, controlling when response selection took place, identified two networks of brain regions that are active during a retention interval. These areas, therefore, are differentially sensitive to the type of representation that is being maintained.

Figure 4.

Delay period. Statistical parametric t maps contrasting oculomotor delayed matching-to-sample versus nonmatching-to-sample delay-period-specific activity. Early (a) and late (b) delay contrasts are shown. Warm colors depict regions with greater delay-period activity on matching than nonmatching trials. Cool colors depict regions with greater delay-period activity on nonmatching than matching trials. MFG, Middle frontal gyrus; iPCS, inferior precentral sulcus.

Linear relationship between delay-period activity and the fidelity of the mnemonic representation

The greater delay-period activity in the oculomotor regions during the match trials identified in the above analyses may simply reflect nonspecific motor preparation or anticipation that is not specifically tied to a future saccade to the remembered spatial location. We performed a correlation analysis between memory-guided saccade accuracy and the magnitude of delay-period activity to address this possibility. This analysis identified voxels whose delay activity scaled linearly with the accuracy of the memory-guided saccade made seconds later. Note that this correlation is based not on whether the memory-guided saccade was correct or not, a gross binomial outcome (Pessoa et al., 2002), but on how accurate the saccade was in continuous degrees of visual angle. A similar analysis was not possible with the nonmatching trials because the number of error trials were too infrequent. Significant positive brain-behavior correlations were found in FEF bilaterally [right coordinates, (43, -9, 49); mean r = 0.48; range r = 0.43-0.59] [left coordinates, (53, -15, 59); mean r = 0.41; range r = 0.28-0.62] and the right IPS [coordinates, (44, -48, 54); mean r = 0.43; range r = 0.33-0.61]. The only correlations found in the dorsolateral prefrontal cortex (DLPFC) were in the right superior frontal sulcus [coordinates, (29, 44, 44); mean r = 0.32; range r = 0.09-0.67]; these coefficients were variable in magnitude across subjects. The right FEF showed the strongest and most consistent correlations across subjects, and a scatterplot illustrating this relationship is displayed in Figure 5. Given such a relationship, it seems unlikely that FEF delay activity reflects a generalized motor anticipation. Rather, the correlation suggests a role in memory and may reflect a prospective intention to shift gaze to a prespecified, memorized location. Thus, the fidelity of the mnemonic representation, indexed here by the accuracy of the later memory-guided saccade, can be predicted by the degree of delay-period activity.

Figure 5.

Scatterplot showing the correlation between memory-guided saccade (MGS) accuracy and the magnitude of the matching-to-sample delay-period parameter estimates in the right FEF. More accurate memory-guided saccades were associated with greater delay-period activity. Approximately one-quarter of the variance in the accuracy of the memory-guided saccade was predicted by the magnitude of delay-period activity. Similar correlations were found in the left FEF and right IPS.

Match versus nonmatch: memory-guided saccadic response period

We now compare differences in brain activation between the match and nonmatch trials during the response period, when a saccade was generated to the remembered location of the stimulus cue (match) or to the stimulus cue that did not match its remembered location (nonmatch). Although both saccades are memory guided, on matching trials, the direction of the saccade can be prepared and the activity may just reflect the triggering of the saccade. On nonmatching trials, the direction of the saccade has remained unknown until this period. Therefore, nonmatching activity also includes selecting the correct saccade target. Many areas were sensitive to the match-nonmatch manipulation. Oculomotor areas, including the FEF and SEF, showed greater saccade activity on match than nonmatch trials. Areas in the middle frontal gyrus and posterior parietal cortex showed greater saccade activity on nonmatching than matching trials (Fig. 6, Table 1). Bilateral activations were also found in the parietal-occipital sulcus that were greater on match than non-match trials at the response period. These activations could be related to the processing of eye-position information after the memory-guided saccade. On match trials, the saccade is not visually guided, and thus the position of the eye in its orbit after a saccade is an important source of information that could be used as an index of spatial position (Nakamura et al., 1999). On non-match trials, the visual stimuli themselves would dominate as a source of spatial feedback.

Figure 6.

Memory-guided saccades. Statistical parametric t maps contrasting memory-guided saccades during the delayed matching-to-sample versus nonmatching-to-sample tasks. Warm colors depict regions with greater saccade period activity on matching than nonmatching trials. Cool colors depict regions with greater saccade period activity on nonmatching than matching trials. MFG, Middle frontal gyrus; SFS, superior frontal sulcus; iPCS, inferior precentral sulcus; POS, parietal-occipital sulcus; PCU, precuneus.

Match versus nonmatch: time courses

Finally, the across-subject average BOLD time courses are presented in Figure 7 to illustrate how the relative distribution of maintenance operations shifts between oculomotor and other prefrontal and parietal regions when the subject knows the direction of the forthcoming saccade during the delay. The average time courses extracted from key regions on interest confirm our statistical regression-based analyses and further highlight the differences between the performance of match and nonmatch trials.

Figure 7.

Average ± SE BOLD time series data (15 subjects) for matching-to-sample (thick black line) and nonmatching-to-sample (thin gray line) oculomotor delayed-response tasks. The solid gray bar represents the delay interval. The gray gradient in the background depicts the probability that the BOLD signal is emanating from the delay period, and the darker indicates more probable. The right FEF and SEF show greater delay-period activity during the matching task, in which an oculomotor strategy is efficient. The right IPS shows greater delay-period activity during the nonmatching task when subjects are biased from using such a strategy. The middle frontal gyrus (BA 46) shows a trend toward greater activity early in the delay on matching trials and then switches and shows significantly greater activity on the nonmatching trials. Error bars indicate SEM.

Discussion

Past human neuroimaging studies have generally implicated a widespread and distributed frontal-parietal network in the maintenance of spatial information (Sweeney et al., 1996; Courtney et al., 1998; LaBar et al., 1999; Postle et al., 2000; Rowe et al., 2000; Leung et al., 2002; Sakai et al., 2002; Brown et al., 2004). Additionally, single neurons in the frontal and parietal cortex of monkeys demonstrate patterns of activity consistent with both prospective and retrospective sensory and motor codes (Funahashi et al., 1990; Boussaoud and Wise, 1993; Chafee and Goldman-Rakic, 1998; Quintana and Fuster, 1999; Rainer et al., 1999; Takeda and Funahashi, 2002).

We provide human evidence for the existence of a frontal-parietal network whose component nodes maintain relatively different representations. We highlight the differences among nodes in this network by showing that their contributions to maintenance are a function of when in the transformation from perception to action the appropriate response can be selected. We manipulated whether the response could be selected before or after the delay with the use of matching-to-sample and nonmatching-to-sample delayed-response tasks. Experimentally controlling the timing of response selection allowed us to bias the nature of the sustained representation during the retention interval.

Maintenance of motor codes

Monkey (Bruce and Goldberg, 1985; Schlag and Schlag-Rey, 1987; Schall and Thompson, 1999; Andersen and Buneo, 2002) and human (Sweeney et al., 1996; Luna et al., 1998; Grosbras et al., 2001; Connolly et al., 2002; Cornelissen et al., 2002; Curtis and D'Esposito, 2003b; DeSouza et al., 2003; Brown et al., 2004) studies suggest that oculomotor centers, such as FEF and SEF, contain representations of saccade intentions. The FEF contains an organized map of visual space defined in oculomotor coordinates (Bruce and Goldberg, 1985). We provide evidence that these regions can maintain saccade intentions with sustained activity that likely reflects the representation of the saccade vector to acquire the target location. Supporting this conclusion, we found that activity in FEF and SEF was generally greater on the matching trials, when the forthcoming saccade vector was known throughout the delay and could be anticipated.

Given that the delay duration was fixed, subjects could clearly have adopted a strategy of anticipation (i.e., preparatory set). Preparatory set refers to the anticipation of a specific class of motor responses determined by context even when the specific response is not known and has been observed in the prefrontal and parietal cortices (D'Esposito et al., 2000b; Fuster, 2001; Curtis and D'Esposito, 2003a; DeSouza et al., 2003). It is unlikely, however, that FEF delay-period activity reflected general motor anticipation but instead represented a more evolved preparatory set in which the specific saccadic vector was prepared and maintained. We identified evidence in support of this claim. The reaction times were shorter for memory-guided saccades on matching compared with nonmatching trails, which is consistent with the hypothesis that subjects prepared the metrics of the saccade during the delay. Critically, we found that delay-period activity in FEF correlated positively with accuracy of the later memory-guided saccade; the greater the delay-period activity, the closer the endpoint of the memory-guided saccade was to the location of the sample cue. If it were the case that the delay-period activity and accuracy correlation were both driven by nonspecific preparation or arousal, then one would expect these two to be correlated. The accuracy and the reaction times of the memory-guided saccades were not correlated. Because the fidelity of the mnemonic representation can be predicted by the degree of delay-period activity, we think that the sustained activity during the delay in FEF reflects either a stored representation of the specific metrics of the saccade goal or possibly oculomotor rehearsal of the saccade metrics (D'Esposito et al., 2000b; Postle et al., 2000; Fuster, 2001; Curtis and D'Esposito, 2003a; Brown et al., 2004).

Maintenance of spatial codes

Activity in posterior parietal cortex has consistently been linked to the representation of space in electrophysiological studies of monkeys (Gnadt and Andersen, 1988; Constantinidis and Steinmetz, 1996; Gottlieb and Goldberg, 1999; Goldberg et al., 2002) and imaging studies of humans (Heide et al., 2001; Sereno et al., 2001; Merriam et al., 2003; Brown et al., 2004). Accordingly, we found that activity in posterior parietal cortex, in and around the IPS, was generally greater on nonmatching trials, when it was impossible to prepare a saccade until after the delay. This suggests that, when oculomotor areas such as FEF cannot maintain a prospective motor code for the intended saccade, as is the case for the nonmatching trials, the IPS maintains a retrospective spatial code for the location of the stimulus cue. Such a spatial code appears to have a retinotopic organization (Sereno et al., 2001; Merriam et al., 2003) and may underlie the local processing advantages that accompany spatially directed attention (Luck et al., 1997; Brefczynski and DeYoe, 1999; Martinez et al., 1999; Yeshurun and Carrasco, 2000; Awh and Jonides, 2001).

Memory-guided saccades

After the retention interval, when a memory-guided saccade must be generated, the matching and nonmatching trials resulted in distinct patterns of cortical activity. These differences could be the result of several strategic and stimulus-related factors. The direction and amplitude of the saccade are guided by memory in each trial type, but the nature of the guiding memory likely differs. Saccade-related activity in right FEF and SEF was greater on matching than nonmatching trials, which is consistent with the use of a maintained motor code. In fact, the right FEF showed greater match than nonmatch activity throughout the whole trial. This indicates that the FEF plays an important role in planning, maintaining, and eventually triggering memory-guided saccades.

Saccade related activity was greater on nonmatching than matching trials in several areas that have been implicated in spatial attention and working memory (Awh and Jonides, 2001; Kastner et al., 2001; Corbetta et al., 2002; Curtis and D'Esposito, 2003a), including IPS and precuneus in posterior parietal cortex, and middle frontal gyrus and superior frontal sulcus in dorsal prefrontal cortex. These differences may reflect processes related to response selection or the transformation of spatial representations to motor representations involved in the evolution of saccade selection (Andersen and Buneo, 2002).

Response selection

Right BA 46 showed clear evidence of delay-period activity during the oculomotor delayed-response tasks. This delay-period activity may reflect maintenance of spatial or motor codes or some process that supports maintenance. The activity may also reflect processes other than maintenance. For example, Pochon et al. (2001) reported right BA 46 activation during a delay only when subjects mentally prepared for an upcoming memory-guided sequence of actions and not when they simply maintained visuospatial information. This indicates motor preparation or rehearsal of motor plans and not necessarily maintenance. Rowe et al. (2000) suggest that right BA 46 does not store or maintain active representations, but, instead, it selects the appropriate memory-guided response. Response selection is the process of choosing among all potential motor responses on the basis of the task context (i.e., the most task-relevant external and internal representations). The right DLPFC has been shown to be sensitive to response selection demands even on tasks with no memory requirements (Schumacher and D'Esposito, 2002; Schumacher et al., 2003).

If we presume that response selection took place on matching trials some time after the sample cue was presented and on non-matching trials when the choice stimuli were presented after the delay, then we might expect to see BA 46 activity locked to these time points. Some evidence for this was found. Activity was much greater on nonmatch than match trials at the response period in right BA 46. When subjects had to choose between the two stimuli on the basis of the memory of the location of the target cue (high selection demands), there was greater right BA 46 activity than when subjects simply generated a memory-guided saccade to blank space (low selection demands). In this case, right BA 46 activity could be related to response selection demands. However, there were only nonsignificant trends for an increase in right BA 46 on match over nonmatch trials at the stimulus cue and early delay periods. The timing of response selection processes was under greater experimental control in the nonmatch trials: it must have occurred between the time the two-choice stimuli were presented and the saccade was generated. Although we believe our experimental manipulation biased response selection to occur much earlier on match trials, it could have taken place at any time point from the onset of the stimulus cue and the offset of the fixation point after the delay. Such a nonstationary process would be more difficult to image reliably. In addition, other factors make the response selection demands much more potent during nonmatch than match trials. At the time of response on nonmatch trials, there is a competition between the prepotent matching cue, whose location has been the focus of attention for 10 sec, and the nonmatching cue, whose location is the saccade goal. The greater right BA 46 activity could be linked to this increased response selection competition or could be linked to the act of inhibiting a saccade to the prepotent matching cue (DeSouza et al., 2003). In the absence of these speculations, our data partially support the role of right BA 46 in response selection.

We report here “relative” differences in BOLD signals that putatively reflect relative differences in the type of code being used to maintain information. The various regions involved in the maintenance of information do not seem to exclusively maintain a single code. For instance, Takeda and Funahashi (2002) recorded from neurons in the midposterior portion of the monkey principal sulcus during oculomotor delayed-response tasks. The majority of neurons that had delay-period activity that was selective for the spatial position of the cue also showed a spatially selective response to the cue. Furthermore, the majority of neurons that had delay-period activity that was selective for the direction of the upcoming motor response also showed a directionally selective response to the saccade generation. Both retrospective visuospatial information and prospective motor plans are represented by the sustained activity in the primate prefrontal cortex. Overall, what is being maintained or what is the significance of persistent neural activity during working memory delays depends on the most available and efficient strategies for bridging cross-temporal contingencies. Networks composed of oculomotor areas preferentially maintain saccadic intentions, whereas networks composed of higher-order areas can maintain sensory representations, such as space, within the focus of attention. Which network is active and which memory code, motor or spatial, is being represented is biased by when in the transformation from perception to action the memory-guided response can be selected.