Relation of ordinal position signals to the expectation of reward and passage of time in four areas of the macaque frontal cortex

Abstract

Neurons in several areas of the monkey frontal cortex exhibit rank selectivity, firing differentially as a function of the stage attained during the performance of a serial order task. The activity of these neurons is commonly thought to represent ordinal position within the trial. However, they might also be sensitive to factors correlated with ordinal position including time elapsed during the trial (which is greater for each successive stage) and the degree of anticipation of reward (which probably increases at each successive stage). To compare the influences of these factors, we monitored neuronal activity in the supplementary motor area (SMA), presupplementary motor area (pre-SMA), supplementary eye field (SEF), and dorsolateral prefrontal cortex during the performance of a serial order task (requiring a series of saccades in three specified directions), a variable reward task (in which a cue displayed early in the trial indicated whether the reward received at the end of the trial would be large or small), and a long delay task (in which the monkey had simply to maintain fixation during a period of time approximating the duration of an average trial in the serial order task). We found that rank signals were partially correlated with sensitivity to elapsed time and anticipated reward. The connection to elapsed time was strongest in the pre-SMA. The connection to anticipated reward was most pronounced in the SMA and SEF. However, critically, these factors could not fully explain rank selectivity in any of the areas tested.

to be able to perform a series of actions in the correct order requires that neurons somewhere in the brain keep track of the sequence of events. Neurons that systematically change their firing rate according to ordinal position within a serial order task are prominent candidates for this role. Such neurons had been often reported in frontal cortical areas including the supplementary motor area (SMA) (Berdyyeva and Olson 2010; Clower and Alexander 1998; Tanji and Shima 1994; Shima and Tanji 1998), presupplementary motor area (pre-SMA) (Berdyyeva and Olson 2010; Clower and Alexander 1998; Isoda and Tanji 2004; Shima and Tanji 1998), supplementary eye field (SEF) (Berdyyeva and Olson 2009 and 2010; Isoda and Tanji 2002 and 2003), and dorsolateral prefrontal cortex (dlPFC) (Averbeck et al. 2002 and 2007; Barone and Joseph 1989; Berdyyeva and Olson 2010; Inoue and Mikami 2006; Mushiake et al. 2006; Ninokura et al. 2003 and 2004). The same frontal areas are also known to carry reward-related signals (Amador et al. 2000; Campos et al. 2005; Ichihara-Takeda and Funahashi 2008; Matsumoto et al. 2003; Pan et al. 2008; Roesch and Olson 2003, 2004, 2005a, 2005b, and 2007; Uchida et al. 2007; Wallis and Miller 2003) and signals related to the passage of time in the trial (Akkal et al. 2004; Ohmae et al. 2008; Roesch and Olson 2003, 2005a, and 2005b; Yarkoni et al. 2005). Since an increasing expectation of reward and awareness of the passage of time normally accompany progression through a serial order trial, it is possible, at least in some areas, that neurons apparently signaling ordinal position do so because they are sensitive to these factors (Dominey 1998b; Farrel and McLaughun 2007).

To investigate this possibility, we recorded from neurons in four areas of the frontal cortex (SMA, pre-SMA, SEF, and dlPFC) during the performance of three tasks: 1) a serial action task, in which the visual cue presented in the beginning of each trial instructed the monkey to execute a series of saccades in a specified sequence of directions; 2) a variable reward task, in which a cue presented in the beginning of each trial indicated whether three drops or one drop of juice would be delivered after a later saccade toward the marked location; and 3) a long delay task, requiring the monkey to maintain central fixation during a long interval matched in duration to an entire trial in the serial action task. We have reported on rank selectivity in each area previously (Berdyyeva and Olson 2010). In addition, we have analyzed the relation between signals related to rank and those related to elapsed time and anticipated reward in the SEF (Berdyyeva and Olson 2009). The aim of the present analysis was to compare the four areas (SMA, pre-SMA, SEF, and dlPFC) with regard to the rate of occurrence of activity dependent on rank, time, and reward and with regard to how strongly a neuron exhibiting one kind of activity predicts its ability to exhibit the others.

MATERIALS AND METHODS

General Methods

Subjects.

Two adult male rhesus monkeys (Macaca mulatta) were used (monkey O had previously been extensively trained on the variety of manual and oculomotor tasks; monkey T was a naïve animal). Experimental procedures were approved by the Carnegie Mellon University Animal Care and Use Committee and were in compliance with guidelines set forth in the National Institutes of Health Guide for the Care and Use of Laboratory Animals.

Preparatory surgery.

At the outset of the training period, each monkey underwent sterile surgery under general anesthesia maintained with isofluorane inhalation. The top of the skull was exposed, bone screws were inserted around the perimeter of the exposed area, a continuous cap of rapidly hardening acrylic was laid down so as to cover the skull and embed the heads of the screws, a head restraint bar was embedded in the cap, and scleral search coils were implanted on the eyes, with the leads directed subcutaneously to plugs on the acrylic cap. After the initial training, recording chambers were implanted into the acrylic. At each selected site, a 2-cm-diameter disk of acrylic and skull was removed. A cylindrical recording chamber was cemented into the hole with its base flush to the exposed dural membrane.

Single-unit recording.

At the beginning of each day's session, a varnish-coated tungsten microelectrode with an initial impedance of several megohms at 1 KHz (Frederick Haer, Bowdoinham, ME) was advanced vertically through the dura into the underlying cortex using a hydraulic microdrive (Narishige, Tokyo, Japan). The electrode could be placed reproducibly at points forming a square grid with 1-mm spacing (Crist et al. 1988). In the case of monkey O, the action potentials of single neurons were isolated from the multineuronal trace by means of an online spike-sorting system using a template-matching algorithm (Signal Processing Systems, Prospect, Australia); the spike-sorting system, on detection of an action potential, generated a pulse, the time of which was stored with 1-ms resolution. In the case of monkey T, three microelectrodes were inserted simultaneously at different grid locations, and single neurons from each microelectrode were sorted using both online and offline template matching and principal components analysis (Plexon, Dallas, TX).

Behavioral control and data collection.

All aspects of the behavioral procedure, including the presentation of stimuli, monitoring of eye movements, and delivery of reward, were under the control of a computer running Cortex software (NIMH Cortex) in a DOS operating system. Eye position was monitored by means of a scleral search coil system (Riverbend Instruments, Birmingham, AL). The x- and y-coordinates of the eye position were stored at 4-ms intervals. Reward was delivered through a spigot under control of a solenoid valve on successful completion of each trial.

Recording sites.

Chambers were placed either at a medial location (over the SEF, pre-SMA, and SMA) or at a lateral location (over the dlPFC). The medial chambers were centered on the midline ∼21 mm anterior to the Horsley-Clarke interaural plane. The lateral chambers were centered approximately at anterior 23 mm and lateral 23 mm in the left hemisphere. Recording was carried out from the medial chamber first and from the lateral chamber second in monkey O. In monkey T, the order was reversed. The reversal of order was intended to minimize the possibility that measured differences between the lateral and medial areas were secondary to differences in the level of practice.

After installing each recording chamber, we measured its location relative to the gross anatomic landmarks by collecting structural magnetic resonance (MR) images. Images were collected with the use of a Bruker BioSpin 4.7-T magnet in which the anesthetized monkey was supported by an MR-compatible stereotaxic device. Fiducial marks made visible by means of a contrast agent included the centers of the ear bars and selected locations inside the recording chamber. Frontoparallel 2-mm-thick slices spanning the entire brain were collected. In addition, 2-mm-thick slices were collected in monkey T parallel to the cortical surface underlying each lateral chamber.

Recording sites under the lateral chambers were in the dlPFC within and dorsal and ventral to the principal sulcus as revealed by an examination of the structural MR images. They were located 7–16 mm anterior to the posterior tip of the principal sulcus in monkey O (mean: 10.5 mm; SD: 2.7 mm) and 7–17 mm anterior to the posterior tip of the principal sulcus in monkey T (mean: 11.1 mm; SD: 2.5 mm). They largely overlapped regions examined in comparable studies from other laboratories (Hasegawa et al. 2004; Shima et al. 2007) but may have extended a few millimeters farther anterior. Recording sites under the medial chambers were assigned to three areas (SEF, SMA, and pre-SMA) according to standard criteria (Matsuzaka et al. 1992; Picard and Strick 1996; Russo and Bruce 1993 and 2000) described in detail in an earlier publication (Berdyyeva and Olson 2010).

Task Design

Serial action task.

BASIC DESIGN PRINCIPLE.

On any given trial, the same image was placed at all three target locations. The identity of this image indicated in what order saccades must be made to the three locations.

SEQUENCE OF EVENTS IN A TRIAL.

The monkey initiated a trial by acquiring central fixation. Two hundred and fifty milliseconds after the attainment of fixation, three identical pictures (∼4.5° across) appeared at locations 11.4° eccentric spaced at equal intervals of 120° around fixation: straight up, down and to the right, and down and to the left. Disappearance of the central fixation spot after a variable interval (450–600 ms) signaled the monkey to make a saccade to the first location. One hundred milliseconds after the completion of the saccade, a feedback stimulus appeared: a white annulus centered on the target. After a variable additional interval of eccentric fixation (225–350 ms), all peripheral stimuli vanished, and the central fixation spot reappeared, signaling the monkey to execute a saccade back to the center. After 150 ms of central fixation, the array of pictures reappeared, and the series of events was repeated with the sole exception that the saccade must be directed to the second location. The third phase of the trial consisted of an equivalent series of events with the third location as the target. The trial terminated with 25 ms of central fixation followed by the offset of all stimuli and delivery of the reward. In the case of a saccade toward an incorrect location or a fixation break, the trial was aborted.

NUMBER OF CONDITIONS.

Six different pictures instructed six different movement sequences. These represented all possible sequences in which the three locations could be, once each, successively selected.

BLOCKING.

To reduce task difficulty, the monkey was allowed to complete a block of four successful trials under a given condition before another condition was imposed. Within a group of six successive blocks, each condition was imposed once.

NUMBER OF TRIALS.

The session terminated when the monkey had successfully completed 24 trials under each of the 6 conditions.

VISUALLY GUIDED TRIALS.

In the event that the monkey selected an incorrect target on three successive trials, visual guidance was provided on the fourth trial by making the image at the location of the current target brighter than the other two. The frequency of trials on which visual guidance was provided, expressed as a percentage of all correct trials, was small (monkey O: SMA 2.1%, pre-SMA 2.0%, SEF 2.7%, and dlPFC 2.5%; monkey T: SMA 13.2.0%; pre-SMA 14.2%, SEF 15.9%, and dlPFC 16.2%). Visually guided trials were excluded from all subsequent analysis.

Variable reward task.

BASIC DESIGN PRINCIPLE.

In the variable reward task, the color and shape of the target informed the monkey whether the reward delivered upon completion of the trial would be small (one drop of juice) or large (three drops of juice).

SEQUENCE OF EVENTS IN A TRIAL.

The monkey initiated a trial by acquiring central fixation. Shortly after the attainment of fixation (250 ms), a cue indicating the size of reward (big or small) appeared at one of three fixed peripheral locations identical to the locations at which targets were placed in the serial order tasks. Disappearance of the central fixation spot (450–600 ms after the cue appeared) signaled the monkey to initiate a saccade toward the cue. One hundred milliseconds after the completion of the saccade, a feedback stimulus appeared. This was a white annulus centered on the target. After an additional 225–350 ms, all stimuli were turned off, and the monkey received a reward of the predicted size. In the event of a fixation break or in the absence of a saccade to the target, the trial was aborted.

NUMBER OF CONDITIONS.

To dissociate neuronal selectivity for the visual attributes of the cue from selectivity for the size of the predicted reward, two different cues were used to signal a reward of each size. Crossing 4 cues with 3 locations gave 12 conditions.

NUMBER OF TRIALS.

A session terminated when the monkey had completed 10 trials successfully under each of the 12 conditions.

Long delay task.

BASIC DESIGN PRINCIPLE.

In a delayed saccade task, the interval between onset of the target and permission to execute a saccade had a duration (4.7 s) approximately equal to the duration of an entire trial in the serial action or serial object task.

SEQUENCE OF EVENTS IN A TRIAL.

The sequence of events was the same as in the variable reward task with the exception that the delay period had a duration of 4.7 s. The target was a gray disk.

NUMBER OF CONDITIONS.

There were three conditions in which the target appeared at the three possible locations.

NUMBER OF TRIALS.

A session terminated when the monkey had completed 20 trials successfully under each of the 3 conditions.

Neuronal Database

We discontinued recording at a given site if it appeared that neuronal activity was unmodulated during performance of the first task (always the serial action task) as indicated by an inspection of the raster and histogram displays. We restricted consideration to neurons characterized in the context of all three tasks. In an offline test, we verified that all neurons that appeared task related in the serial action task at the time of recording met the following generous criterion for task-related activity as applied to data collapsed across all trials. During at least one within-trial epoch in at least one task the firing rate should be significantly different (t-test, P < 0.05) from the firing rate during a pretrial baseline period of 200 ms terminating with the onset of the fixation spot. The within-trial epochs considered in this analysis were as follows. For the serial action task, there were three postarray epochs, each lasting 500 ms after the onset of the array, with one epoch after the array of each rank. For the variable reward task, there was one postcue epoch, lasting 500 ms after the onset of the cue. For the long delay task, there were three equal epochs, representing the beginning, middle, and final thirds of the full 4.7-s delay period. All neurons met this criterion. The number of neurons studied in each area in each monkey is shown in Table 1.

Table 1.

Counts of neurons in the database

	SMA	Pre-SMA	SEF	dlPFC
Monkey O	47	66	83	172
Monkey T	187	76	311	154
Combined	234	142	394	326

SMA, supplementary motor area; pre-SMA, presupplementary motor area; SEF, supplementary eye field; dlPFC, dorsolateral prefrontal cortex.

Counts of Significant Neuronal Effects

Serial action task.

To characterize the dependence of each neuron's firing rate on rank and saccade direction, we carried out a two-factor ANOVA. The firing rate during the postarray epoch (lasting 500 ms after each array onset) was a dependent variable, and the two factors were rank (first, second, or third) and saccade direction (left, up, or right). The threshold for significance was taken as P < 0.05.

Variable reward task.

To characterize the dependence of each neuron's firing rate on predicted reward size and saccade direction, we carried out a two-factor ANOVA. The firing rate during the postcue epoch (lasting 500 ms after the reward cue onset) was the dependant variable, and the factors were reward size (small or large) and saccade direction (left, up, or right). The threshold for significance was taken as P < 0.05.

Long delay task.

To characterize the dependence of each neuron's firing rate on the passage of time and the direction of the impending saccade, we performed an ANOVA with firing rate as the dependant variable and with time (beginning, middle, or final third of the 4.7-s delay period) and saccade direction (left, up, or right) as factors. The threshold for significance was taken as P < 0.05.

Population Histograms

Serial action task.

We parsed the data from each trial into six blocks. These were obtained by crossing three trial phases (first, second, and third) with two periods (array and saccade). The array period extended from 150 ms before to 450 ms after the array onset. The saccade period extended from the offset of the fixation spot to a point in time 600 ms later. The saccade period encompassed the planning and execution of both the centrifugal and centripetal saccades. Cutting out these blocks compensated for variability in trial duration arising from jitter introduced deliberately into the delay periods and from spontaneous variation in the behavioral reaction times. Having computed the average firing rate of each neuron in each 10-ms bin, we then computed the average across neurons of the firing rate in that bin. For purposes of display, we spliced the six blocks into a continuous histogram. The perievent histogram was smoothed with a 20-ms Gaussian kernel.

Variable reward task.

We parsed the data from each trial into two blocks. The cue period extended from 150 ms before to 450 ms after the target onset. The saccade period extended from the offset of the fixation spot to a point in time 300 ms later. Having computed the average firing rate of each neuron in each 10-ms bin, we then computed the average across neurons of the firing rate in that bin. For purposes of display, we spliced the two blocks into a continuous histogram. The perievent histogram was smoothed with a 20-ms Gaussian kernel.

Long delay task.

Throughout the 4.7-s delay period after the target onset, we computed the average firing rate of each neuron in each 10-ms bin. We then computed the average across neurons of the firing rate in that bin. The perievent histogram was smoothed with a 20-ms Gaussian kernel.

Cross-Task Correlations

Reward index versus serial action rank index.

To quantitatively assess the concordance between rank signals in the serial action task and reward size signals in the reward task, we analyzed the correlation across neurons of normalized indexes of rank and reward selectivity. On the basis of data from the serial action task, we computed for each neuron a normalized rank index. This was (A₃ − A₁)/(A₃ + A₁), where A₁ and A₃ are the firing rates during the postarray epoch (lasting 500 ms after each array onset) of phase 1 and phase 3. We focused on A₁ and A₃, ignoring the firing rate during the postarray epoch of phase 2 (A₂), to test the hypothesis that neurons firing most strongly during the earliest phase of the trial (A₁) would favor a small reward, whereas those firing most strongly during the latest phase of the trial (A₃) would favor a large reward. There was no intermediate level of reward analogous to the middle phase of the trial. On the basis of data from the variable reward task, we computed for each neuron a normalized reward index. This was (R_b − R_s)/(R_b + R_s), where R_s and R_b are the firing rates during the postcue epoch (lasting 500 ms after each array onset) on small reward and large reward trials. We then computed the coefficient of correlation across neurons between the rank and reward vectors using the Matlab corrcoef function.

To assess whether areas differed with respect to the degree of correlation between rank and reward indexes, we carried out comparisons between pairs of R values. Using the following standard Fisher R-to-z transformation, we converted the R value from each area into a z-score so as to approximate a normal distribution:

$$z_R^1=0.5\left[{\log }_e\left(1+R\right)-{\log }_e\left(1-R\right)\right]$$

where z_R¹ is the z-score for area 1.

$$\Delta Z=z_R^1-z_R^2$$

where z_R² is the z-score for area 2.

$$Z_{\text{stat}}=\Delta Z/s$$

where s is the SE of ΔZ.

$$s=\left\{1/\left[\left(\text{sample}\hspace{0.17em}\text{size}\hspace{0.17em}\text{in}\hspace{0.17em}area\hspace{0.17em}1\right)-3\right]+1/\left[\left(\text{sample}\hspace{0.17em}\text{size}\hspace{0.17em}\text{in}\hspace{0.17em}area\hspace{0.17em}2\right)-3\right]\right\}$$

We then computed the P value reflecting the significance of the difference between the areas by taking the area under the normal distribution curve to the right of Z_stat.

Time index versus serial action rank index.

On the basis of data from the serial action task, we computed for each neuron the mean firing rates during the postarray epoch (lasting 500 ms after each array onset) of phase 1 (A₁), phase 2 (A₂), and phase 3 (A₃). We then normalized the values to their sum so as to obtain a three-component normalized rank vector: (A₁, A₂, A₃). The normalization step had no effect on the outcome of the correlation analysis but was desirable for graphic presentation of the results (see Fig. 9). By an identical procedure carried out on firing rates from the beginning, middle, and final thirds of the 4.7-s delay period of the long delay task (T₁, T₂, and T₃, respectively), we obtained a second three-component normalized rank vector: (T₁, T₂, T₃). We computed the coefficient of correlation across neurons between the two rank vectors using the Matlab corrcoef function R = corrcoef(X,Y), where X and Y are matrixes whose rows are observations and whose columns are variables (vectors). If C is the covariance matrix, then C = R(i, j) = C(i, j)/[C(i, i) × C(j, j)].

Fig. 9.

Correlation across neurons between the time index measured in the long delay task and the rank index measured in the serial action task. Correlation analysis was carried out on three-dimensional vectors as described in the text. However, for graphic presentation, each neuron contributed three time indexes from the long delay task, (M − B)/(M + B), (E − B)/(E + B), and (E − M)/(E + M), where B, M, and E represent firing rates during the beginning, middle, and end of the delay period, and three corresponding rank indexes from the serial action task, (2 − 1)/(2 + 1), (3 − 1)/(3 + 1), and (3 − 2)/(3 + 2), where 1, 2, and 3 represent firing rates during trial phases 1, 2, and 3.

RESULTS

Selectivity for Rank in the Serial Action Task

We first characterized neuronal activity in the serial action task (Fig. 1A). The database consisted of all neurons studied in this task as well as in the variable reward and long delay tasks (Table 1). This is a subset of the neurons described in a prior report focused on rank selectivity (Olson and Berdyyeva 2010).

Fig. 1.

Sequence of events in a typical trial in the serial action task (A), variable reward task (B), and long delay task (C). Arrows indicate saccades, and dotted circles indicate the direction of gaze. The time indicated at the bottom of each image is the minimal possible duration of fixation during the corresponding stage of the trial. A: in the serial action task, the identity of a picture served as a sequence cue (e.g., the picture of a butterfly instructed the monkey to make an upward saccade during the first trial phase, a saccade down and to the right during the second phase, and a saccade down and to the left during the third phase). Six different pictures signaled the six different sequences in which the three targets could be selected. B: in the variable reward task, the bottom image illustrates the visual cues used to signal the reward a monkey would receive after saccade toward the cue; cues in one pair signaled a small reward (one drop of juice) and cues in the other pair signaled a large reward (three drops of juice). C: in the long delay task, the interval between onset of the target and permission to execute a saccade had a duration (4.7 s) that was approximately equal to the duration of an entire trial in the serial action task.

To classify each neuron as selective for rank, direction, or the interaction of the two, we used an ANOVA (α = 0.05) with firing rate during the postarray epoch (lasting 500 ms after the onset of each array) as the dependent variable and with the rank of the saccade (first, second, or third) and saccade direction (left, up, or right) as factors. Although our primary interest was in the impact of rank, we included direction as a factor so as to account for variance that otherwise would be processed as noise. The counts of neurons significantly influenced by each factor in each area are shown in Fig. 2. We found that the majority of neurons in each area was significantly influenced by rank (Table 2). However, the dlPFC had a significantly lower proportion of rank-selective neurons than any other area (χ²-test, P < 10⁻⁴ for all pairs: SMA-dlPFC, pre-SMA-dlPFC, and SEF-dlPFC).

Fig. 2.

Each Venn diagram shows the proportion of neurons in a given area with a main effect of rank (Rank), a main effect of direction (Dir), and an interaction effect between the two factors (Intx). Note that a neuron sensitive to both rank and direction would contribute to the count in the top central sector if the two signals combined strictly linearly or somewhere in the bottom circle if the combination was not strictly linear. Each bar histogram shows the proportion of neurons in a given area that exhibited significant rank selectivity and favored the first (red), second (blue), or third (black) rank. SMA, supplementary motor area; pre-SMA, presupplementary motor area; SEF, supplementary eye field; dlPFC, dorsolateral prefrontal cortex.

Table 2.

Neurons with a significant main effect of rank in the serial action task

	SMA		Pre-SMA		SEF		dlPFC
	Monkey O	Monkey T	Monkey O	Monkey T	Monkey O	Monkey T	Monkey O	Monkey T
First rank	1	37	24	19	12	44	38	44
Second rank	6	30	20	9	25	100	14	19
Third rank	39	57	10	38	28	135	35	34
Total	170 (73% of 234)		120 (85% of 142)		344 (87% of 394)		184 (56% of 326)

We characterized rank-selective neurons in each area with respect to the rank preferred (Table 2). We identified the preferred rank of each neuron with a significant main effect of rank as the phase of the trial during which the mean firing rate was highest. The proportion of rank-selective neurons preferring each rank among the whole population of selective neurons is shown in Fig. 2, bottom. The timing and strength of activity in each rank-preferring group of neurons can be judged from the population histograms shown in Fig. 3. Examples of neurons preferring the first, second, and third rank are shown in Fig. 4, top, middle, and bottom. To determine whether areas differed with respect to the distribution, among rank-selective neurons, of selectivity for the first, second, and third rank, we carried out a χ²-test. We found that the variation across areas in the distribution of selective neurons according to preferred rank was significant (P < 10⁻¹⁰). The underlying pattern was apparent from the distribution of neurons according to preferred rank as shown in the bar histograms in Fig. 2, bottom, and summarized in the counts in Table 2. In the SMA and SEF, more neurons preferred the third rank than the first rank, whereas in the dlPFC and pre-SMA, approximately equal proportions of neurons preferred the first and third ranks. The observation that pre-SMA neurons tended to show maximum activity for the first ordinal position more frequently than the SMA is consistent with the results of a previous study (Sohn and Lee 2007).

Fig. 3.

Mean firing rate as a function of time for the SMA, pre-SMA, SEF, and dlPFC neurons classified as selective for the first (red), second (blue), or third (black) rank on the basis of activity in the serial action task. To the right of each histogram representing activity in the serial action task are histograms representing the activity of the same neurons in the variable reward task and long delay task. The conventions applying to each histogram are indicated in the set at the bottom, which represents the activity of third-rank neurons in the dlPFC. Serial action task: alternating epochs are aligned on the array onset (Arr) and offset of the central fixation spot (Go). Variable reward task: successive epochs are aligned on the onset of the cue signaling the size of the impending reward (Cue) and offset of the central fixation spot (Go). Long delay task: a single epoch extending from the onset of the peripheral target (Target) to offset of the central fixation spot (Go).

Fig. 4.

Neurons exhibiting strong rank selectivity and not exhibiting modulation in either of the control tasks. Top: SMA neuron selective for the first rank. Middle: SEF neuron selective for the second rank. Bottom: dlPFC neuron selective for the third rank. Alignment events are the same as in Fig. 3.

Selectivity for Reward Size in the Variable Reward Task

We analyzed the activity of the same neurons in the context of a variable reward task (Fig. 1B) by means of an ANOVA (α = 0.05) with firing rate during the postcue epoch (0–500 ms after the cue onset) as the dependent variable and with reward size (small or big) and saccade direction (left, up, or right) as factors. Although our primary interest was in the impact of reward, we included direction as a factor so as to account for variance that would otherwise have been treated as noise. Neurons exhibiting a main effect of reward were common in all areas (Fig. 5 and Table 3).

Fig. 5.

Each Venn diagram shows the proportion of neurons in a given area with main effects of reward (Rew) and direction (Dir) and interaction effects involving both (Intx). Each bar histogram shows the proportion of neurons in a given area that exhibited significant reward selectivity and favored the small (red) or big (black) reward.

Table 3.

Neurons with a significant main effect of reward in the variable reward task

	SMA		Pre-SMA		SEF		dlPFC
	Monkey O	Monkey T	Monkey O	Monkey T	Monkey O	Monkey T	Monkey O	Monkey T
Small reward	4	10	6	2	12	40	23	19
Large reward	25	32	13	37	18	122	30	29
Total	71 (30% of 234)		58 (41% of 142)		192 (49% of 394)		101 (31% of 326)

To assess the significance of the variation across areas in the rate of incidence of main effects of reward (Table 3), we performed a χ²-test on the proportion of neurons exhibiting a main effect of reward as a function of cortical area (three degrees of freedom, data combined across two monkeys). This revealed that there was significant variation across areas (P < 10⁻⁶). To investigate the nature of the cross-area variation, we compared areas pairwise (χ²-test conducted separately for data from each monkey). The single observation consistent across monkeys was that the dlPFC contained a lower proportion of neurons with a main effect of reward than the SEF (P < 10⁻⁴ in the combined data).

In each area and each monkey, among neurons exhibiting a significant main effect of reward, those preferring a large reward formed a majority (SMA: 80%, pre-SMA: 86%, SEF: 73%, and dlPFC: 58%; data combined across monkeys). In the SMA and pre-SMA, the number of neurons preferring a small reward, measured as a fraction of all neurons tested, did not actually significantly exceed the 2.5% expected by chance (SMA: P = 0.1 and pre-SMA: P = 0.3; data combined across monkeys). In the SEF and dlPFC, the number did exceed chance (SEF: P < 10⁻⁶ and dlPFC: P < 10⁻⁶ ; data combined across monkeys; each trend still significant at P < 0.013 when data were split by monkey).

To determine whether areas differed significantly with respect to the proportion of neurons preferring a large reward among those exhibiting a significant main effect of reward, we carried out a χ²-test (data combined across monkeys). This revealed the presence of a significant difference across areas (P = 0.0005). In both monkeys, the dlPFC contained a higher proportion of neurons preferring a small reward than any other area, and the SEF contained a higher proportion of such neurons than either the SMA or pre-SMA. In post hoc pairwise χ²-tests with the Bonferroni correction (α = 0.0083), one comparison attained significance: the dlPFC contained significantly more small reward neurons than the pre-SMA (P = 0.004).

Selectivity for Time in the Long Delay Task

We analyzed the activity of the same neurons in the context of the long delay task (Fig. 1C). This task required the monkey to maintain fixation for a period of time equal to the duration of an average trial in the serial order task but without any parsing of the trial into marked stages. We carried out an ANOVA (α = 0.05) with firing rate as the dependant variable and with time (the beginning third, middle third, or end third of the 4,700-ms delay period) and saccade direction (left, up, or right) as factors. Although our primary interest was in the impact of time, we included direction as a factor so as to account for variance that would otherwise have been treated as noise. Neurons exhibiting a main effect of time were common in all areas (Fig. 6 and Table 4).

Fig. 6.

Each Venn diagram shows the proportion of neurons in a given area with main effects of time (Time) and direction (Dir) and interaction effects involving both (Intx). Each bar histogram shows the proportion of neurons in a given area that exhibited significant time selectivity and fired most strongly during the early (red), middle (blue), or late (black) third of the delay period.

Table 4.

Neurons with a significant main effect of time in the long delay task

	SMA		Pre-SMA		SEF		dlPFC
	Monkey O	Monkey T	Monkey O	Monkey T	Monkey O	Monkey T	Monkey O	Monkey T
Beginning	10	37	26	13	29	61	52	41
Middle	3	10	12	5	8	35	7	2
End	22	26	8	22	24	116	15	10
Total	108 (46% of 234)		86 (61% of 142)		273 (69% of 394)		127 (39% of 326)

To determine whether there was a significant tendency for the proportion of neurons exhibiting a main effect of time to vary across areas, we carried out a χ²-test on data combined between monkeys. This revealed that the difference across areas was highly significant (three degrees of freedom, P < 10⁻¹⁰). The single observation consistent across monkeys was that the dlPFC contained a lower proportion of neurons with a main effect of time than the SMA, pre-SMA, and SEF. In post hoc pairwise χ²-tests with the Bonferroni correction (α = 0.0083), two comparisons attained significance: the dlPFC contained fewer time-sensitive neurons than the pre-SMA (P < 10⁻⁴) and SEF (P < 10⁻⁶).

A time-sensitive neuron might fire most strongly at the beginning, middle, or end of the trial (Fig. 7). However, the rate of occurrence of the three patterns did not seem to be equal (Fig. 6). To determine whether the counts in the three categories were significantly different, we carried out a χ²-test on data from each area. This revealed, in each area and each monkey, that neurons preferring the beginning or end of the trial were more numerous than those preferring the middle (the latter constituted only 12%, 20%, 16%, and 7% of the population in the SMA, pre-SMA, SEF, and dlPFC, respectively). In the dlPFC, the number of neurons preferring the middle of the trial was not significantly above the 1.67% expected by chance (P = 0.07). In the SMA, pre-SMA, and SEF, the number was significantly above chance expectation (SMA: P = 0.006, pre-SMA: P = 0.0003, and SEF: P < 10⁻⁶; each trend was present in each monkey).

Fig. 7.

Neurons exhibiting strong rank selectivity together with concordant modulation in at least one control task. Top: SMA neuron selective for the first rank. Middle: SEF neuron selective for the second rank. Bottom: dlPFC neuron selective for the third rank. Top: dlPFC neuron, selective for the first rank, which also fired maximally at the beginning of the trial in the long delay task. Middle: pre-SMA neuron, selective for the second rank, which also fired maximally in the middle of the trial in the long delay task. Bottom: SMA neuron, selective for the third rank, which also fired maximally when a large reward was impending (in the variable reward task) and at the end of the trial (in the long delay task). Alignment events are the same as in Fig. 3.

To determine whether areas differed significantly with respect to the distribution of time-sensitive neurons across the three categories, we performed a χ²-test (data combined across two monkeys). This revealed a significant tendency for the proportions of neurons in the three categories to vary across areas (three degrees of freedom, P < 10⁻⁷). The pattern of cross-area variation was consistent across monkeys: the dlPFC had a higher proportion of neurons preferring the beginning of the trial than any other area. This pattern proved significant in Bonferroni-corrected post hoc comparisons (χ²-test; dlPFC vs. SMA, P = 0.0001; dlPFC vs. pre-SMA, P = 0.0005; and dlPFC vs. SEF, P < 10⁻⁶).

Rank Selectivity Correlated With Reward Selectivity

Neurons apparently selective for rank could actually have been sensitive to the intensity with which the monkey anticipated reward. A neuron that fired most strongly when reward anticipation was strong (at the end of the trial) would have appeared selective for rank 3. A neuron that fired most strongly when reward anticipation was weak (at the beginning of the trial) would have appeared selective for rank 1. Neurons with these properties should exhibit selectivity for reward size in the variable reward task. A strong correlation between selectivity for rank in the serial action task and selectivity for reward size in the variable reward task would suggest that rank selectivity is an artifact of reward selectivity.

There were cases in which a neuron exhibited clearly concordant selectivity for rank and reward size. An example from the SMA is shown in Fig. 7, bottom. This neuron favored rank 3 in the serial action task and fired more strongly in anticipation of a large reward in the variable reward task. However, there were also instances of rank selectivity in the absence of selectivity for reward size (Fig. 4). At the population level, concordance was most evident in the tendency for SMA, pre-SMA, and SEF neurons favoring rank 3 to fire more strongly in anticipation of a large reward compared with a small reward (Fig. 3).

To assess concordance quantitatively, we analyzed the correlation across neurons of normalized indexes of rank and reward selectivity. The rank index was (A₃ − A₁)/(A₃ + A₁), where A₁ and A₃ are the average firing rates in the postarray onset epochs of trial phase 1 and phase 3 in the serial action task. The reward index was (R_b − R_s)/(R_b + R_s), where R_s and R_b are the firing rates during the postcue epoch on small reward and large reward trials, respectively. The SMA, SEF, and pre-SMA displayed progressively smaller but in all cases positive and significant correlations (Fig. 8). In the dlPFC, the correlation was very small, negative, and barely significant. On comparing correlation strengths between pairs of areas (see materials and methods), we discovered two patterns that were both consistent across monkeys and statistically significant. First, the degree of correlation was lower in the dlPFC than in all other areas (P < 0.0001). Second, the degree of correlation was lower in the pre-SMA than in the SEF (P = 0.03). We note that, even in the area with the strongest correlation (SMA), the reward index, as measured in the variable reward task, explained less than a quarter of the variance across neurons in the rank index, as measured in the serial action task.

Fig. 8.

Correlation across neurons between the reward index measured in the variable reward task and the rank index measured in the serial action task. Reward index: (B − S)/(B + S), where B and S represent firing rates during large reward and small reward trials. Rank index: (3 − 1)/(3 + 1), where 1 and 3 represent firing rates during trial phase 1 and phase 3 in the serial action task.

Rank Selectivity Correlated with Time Selectivity

Neurons apparently selective for rank could actually have been sensitive to the passage of time during the trial. A neuron that fired most strongly at the beginning, middle, or end of the trial would have appeared to be selective for the first, second, or third rank. Such a neuron would, however, have exhibited matching time sensitivity in the long delay task. A strong correlation between selectivity for rank in the serial action task and selectivity for time in the long delay task would suggest that rank selectivity is an artifact of time sensitivity.

There were cases in which a neuron exhibited clearly concordant selectivity for rank and time. Three examples are shown in Fig. 9. However, there were also instances of rank selectivity in the absence of selectivity for time (Fig. 4). At the population level, concordance emerged as a tendency for neurons favoring ranks 1, 2, or 3 to fire most strongly at the beginning, middle, or end of the trial (Fig. 3).

To assess concordance quantitatively, we analyzed the correlation across neurons between a vector representing rank selectivity and a vector representing time selectivity. The raw three-component rank vector was (A₁, A₂, A₃) where A₁, A₂, and A₃ were the average firing rates in the 500-ms postarray onset epochs of trial phases 1, 2, and 3 in the serial action task. We normalized this to (A₁, A₂, A₃), where A₁ = A₁/(A₁ + A₂ + A₃) and likewise mutatis mutandis for A₂ and A₃. The three-component time vector (T₁, T₂, T₃) was obtained by normalization of the raw time vector (T₁, T₂, T₃), where T₁, T₂, and T₃ are the average firing rates during the beginning, middle, and final thirds of the 4.7-s delay period in the long delay task. The coefficient of correlation was significantly positive in all four areas (Fig. 9).

On comparing correlation strengths between pairs of areas (see materials and methods), we discovered one pattern that was both consistent across monkeys and statistically significant. The degree of correlation was lower in the SEF than in all other areas (P < 0.005). We note that, even in the area with the strongest correlation (pre-SMA), the time index, as measured in the long delay task, explained only around a third of the cross-neuron variance of the rank index, as measured in the serial action task.

Independent Contributions of Reward and Time to Rank Selectivity

To determine how reward selectivity and time selectivity combined to predict rank selectivity, we carried out a linear regression analysis based on the rank, reward, and time indexes of all neurons in each area. The rank index was (A₃ − A₁)/(A₃ + A₁), where A₁ and A₃ were defined as above; the reward index was (R_b − R_s)/(R_b + R_s), where R_s and R_b were defined as above; and the time index was (T₃ − T₁)/(T₃ + T₁), where T₁ and T₃ were defined as above. We computed R² (variance explained by regression) for three different models: 1) the rank index as a linear function of the reward index (model R), 2) the rank index as a linear function of the time index (model T), and 3) the rank index as a linear function of the reward index and time index (model RT) (Table 5). We then used a partial F-test to determine whether the increase in explainable variance for the full model (model RT) compared with the reduced model (model R or model T) was significantly greater than expected simply from the presence of an additional degree of freedom. In each area except the SEF, the proportion of variance explained by model RT was significantly higher than that explained by model R (SMA: P < 10⁻⁴, pre-SMA: P < 10⁻⁴, SEF: P = 0.09, and dlPFC: P < 10⁻⁴). In all four areas, the proportion of variance explained by model RT was significantly higher than that explained by model T (SMA: P < 10⁻⁴, pre-SMA: P = 0.007, SEF: P < 10⁻⁴, and dlPFC: P = 0.0001). Thus, in all areas except the SEF, reward selectivity and time selectivity contributed independently to explaining rank selectivity.

Table 5.

Fractional variance (R²) of the rank index explained by the reward index and time index in three regression models

	SMA	Pre-SMA	SEF	dlPFC
Reward index	0.23	0.09	0.22	0.01
Time index	0.21	0.41	0.03	0.15
Reward index and time index	0.29	0.44	0.22	0.19

Note that R² (reward index) + R² (time index) may be equal to or greater than or less than R² (reward index and time index) with the outcome contingent on the degree and sign of correlation between reward and time.

Rank Selectivity Uncorrelated With Reward and Time Selectivity

In every area, most of the cross-neuron variance in the rank index could not be explained on the basis of the reward index and time index (Table 5). Of the residual variance, some might be genuinely associated with rank, but some might be the product of noise. To establish a lower limit on the fraction of variance genuinely related to rank, we considered a subset of neurons studied not only in the three tasks discussed so far but also in a fourth task, the serial object task. This was closely parallel to the serial action task in matters such as timing but differed in requiring the monkey to select three objects in sequence rather than three actions. The details of this task and results obtained in its context have been presented elsewhere (Bedyyeva and Olson 2009 and 2010). We concerned ourselves solely here with the fact that it provided an independent measure of rank selectivity: (O₃ − O₁)/(O₃ + O₁), where O₁ and O₃ are the firing rates during 500-ms postarray onset epochs in phase 1 and phase 3 of the trial. We carried out linear regression analyses, with the rank index in the serial action task as the dependent variable and with three factors in seven possible combinations as regressors. These factors were the reward index from the variable reward task (R), the time index from the long delay task (T), and the rank index from the serial object task (O). The percentage of explainable variance captured by each of the seven models is shown in Fig. 10. Two critical findings stand out. First, the regressor O, employed in isolation, explained much more variance than R or T employed in isolation. Second, adding R or T to O produced a relatively modest increase in variance explained, whereas adding O to R or T produced a marked increase. We conclude that neurons in all four areas carried rank signals in the serial action task that could be predicted much better from rank signals in the serial object task than from reward signals in the variable reward task or time signals in the long delay task.

Fig. 10.

Results of linear regression analyses with the rank index in the serial action task as a dependent variable and with three regressors in all seven possible combinations. The regressors were the reward index from the variable reward task (R), the time index from the long delay task (T), and the rank index from the serial object task (O). The percentage of explainable variance captured by each model is indicated by the area of the corresponding disk and the number within it. The models formed a nested hierarchy. For each step from a simpler model to a fuller model, we used a partial F-test to determine whether the increase in variance explained was significantly greater than would be expected simply from adding another free parameter. Arrows with filled heads indicate cases in which the improvement was statistically significant (α = 0.05); arrows with open heads indicate cases in which the improvement was not statistically significant.

DISCUSSION

Overview

The aim of this experiment was to determine whether frontal areas in which neurons carry rank signals (SMA, pre-SMA, SEF, and dlPFC) differ from each other with regard to the degree to which those signals depend on factors commonly associated with rank, notably the anticipation of reward and the passage of time. Before proceeding to consider the existence of some minor differences among the four areas, we should note that the major conclusion is the same for all of them. Although some rank-related activity can be explained in terms of sensitivity to reward and time, the bulk of it cannot. This conclusion emerged most dramatically from the regression analysis described at the end of results and shown in Fig. 10. The essential conclusion of this analysis is that rank selectivity in the serial action task is much better predicted by rank selectivity in the serial object task than by reward sensitivity in the variable reward task or time sensitivity in the long delay task. It is possible but not certain that rank signals mediate the monkey's awareness of ordinal position. It is certain, however, that rank-related activity is conserved across the two serial order tasks and depends only to a minor degree on the anticipation of reward and the passage of time.

Relation Between Rank and Reward Selectivity

If neuronal signals apparently related to rank were associated with the anticipation of reward (low at the beginning and high at the end of the trial), then there would have been a positive correlation across neurons between the tendency to fire early (or late) in the serial action task and the tendency to fire strongly in anticipation of a small (or large) reward in the variable reward task. There was indeed an overall trend in this direction. However, the trend was weak in the pre-SMA and absent in the dlPFC (Fig. 8).

This outcome fits with several observations indicating that the pre-SMA and dlPFC differ from the SMA and SEF in being more involved in cognitive and less involved in motor and reward-related functions: 1) the dlPFC is involved in the representation of abstract concepts and rules (Miller et al. 2002; Wallis et al. 2001) even when they are not crucial for reward prediction (Hasegawa et al. 2000 and 2004; Shima et al. 2007); 2) the pre-SMA is similar to the dlPFC in anatomic connectivity and functional properties (Akkal et al. 2002; Pickard and Strick 2001) and therefore is thought to mediate relatively cognitive aspects of motor control (Tanji 2001; Hikosaka et al. 2002); and 3) reward-related activity is less common in the dlPFC than in the SMA (Roesch and Olson 2003).

Relation Between Rank and Time Selectivity

If neuronal signals apparently related to rank were associated with the passage of time during the trial, then there would have been a positive correlation across neurons between the tendency to fire in conjunction with rank 1, 2, or 3 in the serial action task and the tendency to fire at the trial's beginning, middle, or end in the long delay task. This tendency was present and significant in all areas and was especially strong in the pre-SMA, which differed significantly from all other areas in this regard (Fig. 9).

This result suggests caution in the interpretation of the robust rank selectivity noted in previous studies of the pre-SMA (Berdyyeva and Olson 2010; Clower and Alexander 1998; Isoda and Tanji 2004; Shima and Tanji 2000; Sohn and Lee 2007) and fits with previous indications that neurons in the pre-SMA exhibit build-up firing over the course of a trial if its duration is known (Akkal et al. 2004). The idea that the area is involved in time awareness is concordant with the results of human functional imaging studies documenting its recruitment under conditions requiring the integration of temporal intervals and ordinal information (Sakai et al. 2002; Bengtsson et al. 2004) or attention to temporal aspects of a task (Coull 2004; Knutson et al. 2004; Lejeune et al. 1997; Macar et al. 2002; Nobre and O'Reilly 2004; Schubotz et al. 2000).

It might be argued that the dependence of firing rate on time was secondary to its dependence on reward because the time-discounted value of an anticipated reward increases as the trial progresses (Roesch and Olson 2005a). We cannot rule out a contribution from this effect. We note, however, that it cannot account for the full pattern of results. In particular, it cannot explain the fact that reward sensitivity and time sensitivity contributed independently to predicting rank sensitivity (Fig. 10: R ≠ T and RT > R, T).

Functional Significance

In principle, any sequential behavior could arise from a chain of associations in which each stage leads to the next; however, it has long been recognized that chaining is implausible as a general account of serial order behavior (Lashley 1951; Henson 1998). Moreover, in tasks such as the ones used here, behavioral evidence strongly supports the idea that subjects do not proceed from action to action by a chain of reflexes but instead maintain an incrementally updated internal representation of ordinal position and select at each stage the action associated with the current representation (Orlov et al. 2002). Several successful computational models exploit explicit representations of either absolute ordinal position (Beiser and Houk 1998; Botvinick and Watanabe 2007; Dominey et al. 1998a; Fukai 1999; Orlov et al. 2002; Salinas 2009) or ordinal position as defined relative to the beginning and end of a sequence (Henson 1998; Rosenbaum et al. 2007). Our findings are compatible with these ideas in that they demonstrate the presence in frontal cortex of ordinal position signals independent of signals reflecting extraneous factors commonly correlated with rank (Fig. 4). However, they call into question the idea that the representation of ordinal position is the specialized function of a particular area. On the contrary, it is widely distributed, as indicated by the fact that neurons in all four studied areas carried rank signals of approximately equal strength (Supplemental Material, Supplemental Table S1).¹

GRANTS

This work was supported by National Institutes of Health (NIH) Grants RO1-EY-018620 and P50-MH-45156, with technical support by NIH Grants P30-EY-08098 and P41-RR-03631.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).