Beyond the labeled line: variation in visual reference frames from intraparietal cortex to frontal eye fields and the superior colliculus

Abstract

We accurately perceive the visual scene despite moving our eyes ~3 times per second, an ability that requires incorporation of eye position and retinal information. In this study, we assessed how this neural computation unfolds across three interconnected structures: frontal eye fields (FEF), intraparietal cortex (LIP/MIP), and the superior colliculus (SC). Single-unit activity was assessed in head-restrained monkeys performing visually guided saccades from different initial fixations. As previously shown, the receptive fields of most LIP/MIP neurons shifted to novel positions on the retina for each eye position, and these locations were not clearly related to each other in either eye- or head-centered coordinates (defined as hybrid coordinates). In contrast, the receptive fields of most SC neurons were stable in eye-centered coordinates. In FEF, visual signals were intermediate between those patterns: around 60% were eye-centered, whereas the remainder showed changes in receptive field location, boundaries, or responsiveness that rendered the response patterns hybrid or occasionally head-centered. These results suggest that FEF may act as a transitional step in an evolution of coordinates between LIP/MIP and SC. The persistence across cortical areas of mixed representations that do not provide unequivocal location labels in a consistent reference frame has implications for how these representations must be read out.

NEW & NOTEWORTHY How we perceive the world as stable using mobile retinas is poorly understood. We compared the stability of visual receptive fields across different fixation positions in three visuomotor regions. Irregular changes in receptive field position were ubiquitous in intraparietal cortex, evident but less common in the frontal eye fields, and negligible in the superior colliculus (SC), where receptive fields shifted reliably across fixations. Only the SC provides a stable labeled-line code for stimuli across saccades.

INTRODUCTION

The perceived locations of visual stimuli are derived from a combination of the location of retinal activation and information about the direction of eye gaze. How these signals are combined to synthesize a representation of visual space as the eyes move is unknown. The process is computationally complex, because any site on the retina could correspond to any given site in the visual scene, but only the correct correspondence for a particular eye gaze position is operative at any given time. Coordinate transformations are therefore both flexible and precise, and it has been suggested that they unfold as a gradual process across multiple brain regions.

Which visual areas are truly retinotopic, or eye-centered, and which employ higher order representations incorporating information about eye movements is uncertain. The retina is thought to provide the brain with an eye-centered map of the locations of visual stimuli, but after that, the recurrent interconnectivity of the brain in principle permits adjustment of reference frame. Several studies have indicated effects of eye position or movement on visual responses as early as the lateral geniculate nucleus (Lal and Friedlander 1989, 1990a, 1990b). In V1, some studies have found evidence that eye position modifies visual signals (Trotter and Celebrini 1999), and some have not (Gur and Snodderly 1997; Motter and Poggio 1990). Later visual structures exhibit more extensive sensitivity to eye position/movements (e.g., Bremmer 2000; Bremmer et al. 1997; Squatrito and Maioli 1996).

We focus in this report on a quantitative comparison of the reference frames employed in three interconnected brain regions involved in guiding saccadic eye movements to visual targets (Fig. 1): the lateral and medial banks of the intraparietal sulcus (LIP/MIP), the superior colliculus (SC), and the frontal eye fields (FEF) (e.g., Bisley and Goldberg 2003; Bruce and Goldberg 1985; Huerta et al. 1986; Freedman and Sparks 1997; Lee et al. 1988; Schall 1991; Schall and Hanes 1993; Scudder et al. 2002; Sparks et al. 1976; Stanton et al. 1988; Steenrod et al. 2013; Waitzman et al. 1988; Wurtz and Goldberg 1971; Wurtz et al. 2001). Electrical stimulation in the SC and FEF evokes short-latency saccades at low thresholds (Bruce et al. 1985; Robinson 1972; Robinson and Fuchs 1969; Schiller and Stryker 1972). Stimulation in parietal cortex can also evoke saccades (Constantin et al. 2007, 2009; Thier and Andersen 1996, 1998). Lesions of both the SC and FEF together eliminate saccades, suggesting that these two structures are necessary for saccade generation, whereas lesions of parietal cortex can produce a more general hemineglect (Bisiach and Luzzatti 1978; Dias and Segraves 1999; Schiller et al. 1980, 1987; Sommer and Tehovnik 1997).

Fig. 1.

Schematic of the connections between the areas LIP/MIP, SC, and FEF, their visual inputs, and their projections to the brain stem saccade generator. The LIP/MIP and FEF are highly interconnected and send excitatory projections to the intermediate and deep layers of the SC (continuous arrows indicate direct projections). The FEF also sends inhibitory indirect projections to the SC through the caudate and the substantia nigra pars reticulata (dotted arrows indicate indirect projections). Both the SC and the FEF directly project to the various areas of the brain stem saccade generator system. The LIP/MIP and the FEF receive visual inputs from extrastriate visual areas. The SC receives visual inputs mainly in its superficial layer from the primary and secondary visual cortices and the FEF, and also directly from the retina (for reviews, see Blatt et al. 1990; Schall et al. 1995; Sparks and Hartwich-Young 1989). Connections between oculomotor areas are shown in gray and visual inputs in black.

Parietal cortex was one of the first brain regions in which eye movements and visual signals were shown to interact (Andersen et al. 1993). These response patterns were initially characterized as “gain fields,” in which receptive fields were stable in eye-centered location but exhibited a response modulation with different eye positions (Andersen et al. 1985). Subsequent studies involving complete sampling of receptive field positions as the eyes moved suggested that receptive fields could also adopt new positions on the retina at different eye positions (Mullette-Gillman et al. 2005, 2009). These changes in receptive field position produced a code that varies across neurons and ranges from predominantly eye-centered to predominantly head-centered, with most neurons exhibiting “hybrid” response patterns that could not be categorized as either of these options. In contrast, the SC, though exhibiting gain fields (Van Opstal et al. 1995), employs a predominantly eye-centered code when tested and analyzed the same way as LIP/MIP (Lee and Groh 2012; see also DeSouza et al. 2011; Klier et al. 2001; Sadeh et al. 2015).

Considerable interest has focused recently on eye position gain fields in the FEF (Cassanello and Ferrera 2007) and on how receptive fields in the FEF change transiently at the time of the eye movement (Sommer and Wurtz 2006; Zirnsak and Moore 2014; Zirnsak et al. 2014). In addition, a detailed quantitative assessment of torsional, eye-in-head, and head-on-body components of the FEF’s reference frame has been conducted in a paradigm in which the both the head and eyes were free to move with respect to each other and to the body (Keith et al. 2009; Sajad et al. 2015, 2016). However, a quantitative assessment of the reference frame during steady fixation with the head restrained (important for our larger purpose of comparing visual and auditory coding) has not, to our knowledge, been conducted.

In this article we report that the reference frame of visual signals in the FEF is intermediate between the SC and LIP/MIP. The results support the view that reference frames evolve along brain pathways involved in controlling visually guided behavior, becoming a plausible labeled line for eye-centered stimulus location only at the level of the SC.

MATERIALS AND METHODS

The task and recording conditions for the FEF data set have been explained previously (Caruso et al. 2016). We briefly report them below. All experimental procedures conformed to NIH guidelines (National Research Council 2011) and were approved by the Institutional Animal Care and Use Committee of Duke University. Two adult rhesus monkeys (Macaca mulatta) were implanted with a head holder to immobilize the head, a scleral search coil to track eye movements, and a recording cylinder over the left or right FEF. Similar procedures were used to prepare for recordings in LIP/MIP and SC, as reported previously (Lee and Groh 2012, 2014; Mullette-Gillman et al. 2005, 2009).

During data collection, the monkeys were seated 150 cm from a light-emitting diode (LED) display board with their heads restrained. The experiments took place in a dark (monkey F, male) or dimly illuminated room (monkey N, female); the dim illumination limited normal dark-induced nystagmus (Mulch and Lewitzki 1977) but did not provide any useful visual cues. Indeed, saccade accuracy was comparable for the two monkeys (Caruso et al. 2016) and with previous studies in complete darkness (Metzger et al. 2004; Mullette-Gillman et al. 2005, 2009).

All data were recorded while the monkeys performed visually or aurally guided saccades, randomly interleaved, in an overlap saccade paradigm. Only visual trials were analyzed in this study. In each trial, a target was presented while the monkey fixated a visual fixation stimulus (Fig. 2, A and B; all visual stimuli were produced by green LEDs). The monkey withheld the saccade for 600–900 ms until the offset of the fixation target, permitting the dissociation of sensory-related activity from motor-related activity. The targets were located in front of the monkeys at 0° elevation and between −24° and +24° horizontally (9 locations separated by 6°, Fig. 2A). In each session, all saccades started from three initial fixation locations at −12°, 0°, +12° along the horizontal direction and at an elevation chosen to best sample the receptive field of the neuron under study.

Fig. 2.

Stimuli, task, and classification system of receptive fields. A: locations of stimuli and initial fixations. Varying the initial fixation permits the separation of eye- and head-centered reference frames by measuring the relative alignment of the responses in head- and eye-centered coordinates. B: task. Each trial starts with the appearance of a fixation light, which the monkey is required to fixate. A target then appears, but the monkey needs to wait until the fixation goes out before making a saccade to the target. C: predominantly eye-centered response patterns. The 3 tuning curves obtained for the 3 initial fixation locations align best in eye-centered coordinates (perfect alignment, R_eye ≈ 1; right), whereas in head-centered coordinates (left), they are shifted by the distance between the initial eye positions (i.e., steps of 12° in the present task, resulting in R_head < 1). C1 and C2 depict closed and open receptive fields, respectively. The classification metric is applicable in both cases. D: predominantly head-centered responses. The pattern is the opposite of that in C: the 3 tuning curves are aligned in head-centered coordinates and separated by 12° in eye-centered coordinates. D1 and D2 show that the classification is appropriate for both closed and open receptive fields. E: hybrid-partial shift response pattern. The 3 tuning curves are not well aligned in either head- or eye-centered coordinates: as the initial eye direction shifts left or right (red or blue, respectively), the tuning curves only partially move apart, by less than 12°. E1 and E2 show that both closed and open receptive fields are classified as hybrid-partial shift when the shift is less than the distance between initial fixations. F: hybrid-complex coordinates. The initial eye location affects the shape, gain, and/or alignment of the tuning curves in unpredictable ways that have no obvious relationship in either eye- or head-centered coordinates. G–J: schematics of population analysis. When R_head is plotted vs. R_eye, the data points should lie below the line of slope = 1 if the reference frame is predominantly eye-centered (G), above the line of slope = 1 if head-centered (H), along the line of slope = 1, but at positive values, if hybrid-partial shift (I), and randomly if hybrid-complex (J).

The behavioral paradigm, the acquisition of eye trajectory, and the recordings of single-cell activity were controlled using the Beethoven program (Ryklin Software). Eye gaze was sampled at 500 Hz. Single-neuron extracellular activity was acquired using a Plexon system (Sort Client software; Plexon) through tungsten microelectrodes (FHC; 0.7 to 2.5 MΩ at 1 kHz).

Analysis

All analyses were conducted with custom-made routines in MATLAB (The MathWorks). Only correct trials were included.

Spatial selectivity analysis.

This analysis has been described in detail by Caruso et al. (2016), Mullette-Gillman et al. (2005), and Lee and Groh (2012). Briefly, we defined a baseline period, comprising the 0–500 ms of fixation before the target onset, and a sensory window as the period 0–500 ms after target onset. The sensory window captured both the transient and sustained responses to visual targets. We selected the motor window differently in different areas, to better capture the saccade-related burst, which has different temporal characteristics across regions. The motor window was defined to start before the saccade onset (20 ms before saccade onset for the SC data, 50 ms before onset for the FEF data, and 150 ms before onset for the LIP data) and to end at saccade offset. Saccade onset and offset were defined as the moment, at 2-ms resolution, that the instantaneous speed of the eye movement exceeded or dropped below a threshold of 25°/s. (In addition to these fixed analysis windows, we also analyzed sliding 100-ms windows throughout the interval from target onset to the saccade, detailed below).

Neurons were considered responsive in the sensory/motor intervals if a two-tailed t-test between their baseline activity and relevant response period was significantly different with P < 0.05. Spatial selectivity of responses (in the sensory or motor period) was assessed in both head- and eye-centered reference frames, using two two-way ANOVAs. Each ANOVA involved the three levels of initial eye position (−12°, 0°, +12°) as well as five levels of target location (−12° to +12° in 6° increments), defined in head-centered coordinates for the first ANOVA and in eye-centered coordinates for the second ANOVA. Cells were classified as spatially selective if either of the two ANOVAs yielded a significant main effect for target location or a significant interaction between the target and fixation locations (Lee and Groh 2012, 2014; Mullette-Gillman et al. 2005, 2009). In all tests, statistical significance was defined as P value <0.05 (Table 1). To be consistent with our previous analyses and because these tests were used for inclusion criteria rather than hypothesis testing, we did not apply Bonferroni correction.

Table 1.

Spatially selective populations in LIP/MIP, FEF, and SC

		No. of Spatially Selective Cells
	No. of Recorded Cells	Sensory period	Motor period	Sensory and motor period
LIP/MIP	275	125 (45%)	121 (44%)	69 (25%)
FEF	324	174 (54%)	160 (49%)	117 (36%)
SC	179	162 (91%)	161 (90%)	153 (85%)

For each area (LIP/MIP, FEF, and SC) and each time window (sensory and motor periods; see materials and methods), the number of spatially selective cells is indicated, with percentages in parentheses. A cell was considered spatially selective if its response was modulated by target location in either head- or eye-centered coordinates, according to two 2-way ANOVAs with target location and fixation position as the two factors (one 2-way ANOVA for target locations defined with respect to the eye and one 2-way ANOVA for target locations defined with respect to the head; target location main effects, P < 0.05; interaction terms, P < 0.05).

Reference frame analysis.

To distinguish eye-centered and head-centered reference frames, we quantified the degree of alignment between eye-centered and head-centered tuning curves obtained from trials with initial eye positions at −12°, 0°, +12° along the horizontal axis. This analysis was applied to single cells during different time windows throughout the trials. In particular, for each time window considered, we constructed the three response tuning curves for the three fixation locations with target locations defined in head- or eye-centered coordinates (schematized in Fig. 2) and quantified their relative shift with an index akin to an average correlation coefficient (Eq. 1; Mullette-Gillman et al. 2005). We call it “reference frame index,” and for each response we calculate two indexes, in head-centered and in eye-centered coordinates, according to the formula:

$$R={\displaystyle \sum }_i\frac{1}{2}\left(\frac{\left(\overrightarrow{r_{L,i}}-\overline{r}\right)·\left(\overrightarrow{r_{C,i}}-\overline{r}\right)}{\left|\overrightarrow{r_{L,i}}-\overline{r}\right|·\left|\overrightarrow{r_{C,i}}-\overline{r}\right|}+\frac{\left(\overrightarrow{r_{R,i}}-\overline{r}\right)·\left(\overrightarrow{r_{C,i}}-\overline{r}\right)}{\left|\overrightarrow{r_{R,i}}-\overline{r}\right|·\left|\overrightarrow{r_{C,i}}-\overline{r}\right|}\right),$$ (1)

where r_L,i, r_C,i, and r_R,i are the vectors of average responses of the neuron to a target at location i when the monkey’s eyes were fixated at the left (L), right (R), or center (C). Only the target locations that were present for all three fixation positions in both head- and eye-centered frames of reference were included (5 locations: −12°, −6°, 0°, 6°, and 12°). The reference frame index is primarily sensitive to the relative translation of the three tuning curves and is comparatively insensitive to possible gain differences between them, provided the sampling includes some inflection point in the response curve. This can occur either by sampling from both sides of the receptive field center or by sampling from locations that are both inside and outside of the receptive field. The reference frame index values range from −1 to 1, where 1 indicates perfect alignment, 0 indicates no alignment, and −1 indicates perfect negative correlation (Lee and Groh 2012, 2014; Mullette-Gillman et al. 2005, 2009; Porter and Groh 2006). We calculated the 95% confidence intervals of the index with a bootstrap analysis (1,000 iterations of 80% of data for each target-fixation combination). Each set of responses was classified, based on the quantitative comparison between the eye- and head-centered indexes, as 1) eye-centered if the eye-centered reference index was statistically higher than the head-centered reference index (that is, if the 95% confidence interval of eye-centered reference index was positive and larger than the 95% confidence interval of head-centered reference index); 2) head-centered if the opposite pattern was found; 3) hybrid-partial shift if the two indexes were statistically different from each other and at least one of them was statistically different from zero (that is, if the 95% confidence intervals of the eye-and head-centered indexes overlapped with each other and at least one did not include 0), or 4) hybrid-complex if the two indexes were not statistically different from each other and from zero (that is, if the 95% confidence intervals of the eye-and head-centered indexes overlapped with each other and with 0; see Fig. 4). These latter two categories were combined in our previous reference frame analyses of activity patterns in the SC and LIP/MIP (Lee and Groh 2012; Mullette-Gillman et al. 2005, 2009). In this study, we consider them both separately and together as appropriate to provide a more comprehensive comparison of the reference frames across regions. We conducted the reference frame analysis for each cell during the sensory and motor periods (see Figs. 4 and 5, A and B) and across time in 100-ms windows sliding with steps of 50 ms from the target onset and from saccade onset (Figs. 4 and 5, C and D).

Fig. 4.

Reference frames in the FEF population. A and B: the reference frame indexes in head-centered and eye-centered coordinates are plotted for spatially selective cells in each time window: visual sensory (A) and visual motor (B). Responses are classified as eye-centered if the 95% confidence interval of eye-centered coefficient was positive, larger than, and nonoverlapping with the 95% confidence interval of head-centered coefficient (bootstrap analysis; see materials and methods); these responses are indicated in orange. Responses were classified as head-centered with the opposite pattern (blue). Finally, hybrid-partial shift reference frames (dark gray) have non-zero overlapping 95% confidence intervals, whereas hybrid-complex responses (light gray) have reference frame indexes not statistically different from zero. The pie charts summarize the proportion responses classified as eye-centered, head-centered, and the subtypes of hybrid for each time window. C and D: time course of the average eye-centered (black) and head-centered (gray) reference frame indexes for the FEF population of spatially selective responses (see materials and methods). The indexes were calculated in bins of 100 ms, sliding with a step of 50 ms and averaged across the population. Trials were aligned at target onset (C) and at saccade onset (D). Filled circles indicate that the difference between the 2 average indexes is statistically significant (t-test for each bin, P < 0.05). The time frames displayed in C and D overlap and include the variable time to the offset of the fixation light as well as saccade reaction time (averaging ~200 ms) as indicated by the shaded boxes.

Fig. 5.

Reference frame indexes during sensory and motor periods in LIP/MIP, FEF, and SC. A and B: the percentage of cells classified as eye-centered (orange), head-centered (blue), and hybrid (dark gray, hybrid-partial shift; light gray, hybrid-complex) are shown for all spatially selective cells in LIP/MIP, FEF, and SC in the sensory period (A) and in the motor period (B). For the sensory response, the time window is the 500 ms immediately after target onset. For the motor response, the time window changes with the saccade duration, ranging from −20 (SC), −50 (FEF), or −100 ms (LIP/MIP) from saccade onset to saccade offset (see materials and methods). For the FEF, the data are the same as in Fig. 4, A and B. C and D: time course of the eye-centered reference frame indexes for the populations of spatially selective cells (C) and for all recorded cells (D) in LIP/MIP (tan), FEF (green), and SC (dark red). Filled circles indicate that the eye-centered correlation coefficient was significantly larger than the head-centered one (2-tailed t-test for each bin, P < 0.05). Gray boxes indicate approximate timing of fixation light offset as well as saccade onset, as in Fig. 4, C and D. The FEF data are the same as in Fig. 4, C and D. The LIP/MIP and SC data were collected in studies by Lee and Groh (2012, 2014) and Mullette-Gillman et al. (2005, 2009).

LIP/MIP and SC Data Sets

Figure 5 includes data from area LIP/MIP and SC that we have previously collected and described (Lee and Groh 2012, 2014; Mullette-Gillman et al. 2005, 2009). The tasks and recording techniques were the same as in the present study, but each data set was recorded from different monkeys (2 monkeys per brain area). The SC data set consists of a total of 179 single cells recorded in the left and right SC of 2 monkeys. The LIP/MIP data set consists of a total of 275 single cells recorded in the left and right LIP/MIP of 2 monkeys. For these two additional data sets, responsiveness, spatial selectivity, and reference frame were assessed the same way as for the FEF data set. Some of the data in Fig. 5 were reanalyzed in different time windows than in the original studies to allow for a fair comparison across brain areas. However, changing the analysis windows did not change the overall results of the previous studies.

RESULTS

Overview

We first describe our new data concerning the visual reference frame in FEF before quantitatively comparing FEF with our previously reported results in LIP/MIP and the SC (Lee and Groh 2012, 2014; Mullette-Gillman et al. 2005, 2009). We recorded single-cell response profiles while the monkeys performed visually guided saccades, interleaved with aurally guided saccades that were not analyzed in this study (see materials and methods). The task and the location of the stimuli are described in Fig. 2, A and B.

We tested whether the receptive fields of FEF neurons (N = 324) shifted with the eyes or stayed fixed relative to the head by defining two reference frame indexes, R_eye and R_head, in eye- and head-centered coordinates. These indexes are akin to the average correlation coefficient between the tuning curves obtained from the three initial fixations (see materials and methods). Thus they measure the relative alignment of the tuning curves in eye-centered or head-centered coordinates, without regard to changes in overall response magnitude, which does not contribute substantially to this measure. Figure 2 schematically shows our classification system based on the quantitative comparison between R_eye and R_head. Response patterns were considered either “eye-centered” (Fig. 2C; the tuning curves aligned in eye-coordinates better than in head-coordinates), “head-centered” (Fig. 2D; opposite pattern), or “hybrid” if the response patterns were not well described in either coordinate system. A hybrid reference frame refers to circumstances such as when the retinal positions of receptive fields differ across eye positions but not in the way consistent with head-centered coordinates, either because the receptive fields shift by only part of the difference in fixation position (Fig. 2E; we refer to this scenario as “hybrid-partial shift”) or because the receptive fields change across fixations in unpredictable ways that appear random (Fig. 2F, “hybrid-complex”). Note that the classification depends uniquely on the alignment, and not on the shape of the receptive fields (Fig. 2, C1, C2, D1, D2, E1, and E2). Figure 2, G–J, shows the patterns expected when R_head is plotted against R_eye in each of these scenarios: data points would cluster below (G), above (H), or along the diagonal (I and J) depending on the observed coordinate system. It should be emphasized that this classification scheme is merely a tool to capture features of the neural population as a whole; there is little evidence to suggest these labels represent functionally distinct categories, but rather reflect a continuum of coding properties.

A final note is that in these experiments, head, body, and world are immobile with respect to each other. Thus it is not possible to distinguish these reference frames from one another.

Example Neurons in the FEF

Figure 3 shows the responses from nine individual cells during the sensory (Fig. 3, A, C, E, G, and I) and motor periods (Fig. 3, B, D, F, H, and J). During both the sensory period and the motor burst, the most common pattern was eye-centered (Fig. 3, A–D) with an unambiguous difference between high reference frame indexes in eye-centered coordinates (close to 1) and very low reference frame indexes in head-centered coordinates (the examples shown in Fig. 3, A–D, had, respectively: R_eye = 0.97, 0.93, 0.82, and 0.97 vs. R_head = −0.17, 0.18, 0.37, and 0.35).

Fig. 3.

Examples of responses in the FEF. A–J: each panel shows the tuning curves for various example cells during the sensory or motor period (see materials and methods). The tuning curves are plotted in both head-centered (left) and eye-centered coordinates (right), and the reference frame indexes R_head and R_eye are indicated. A–D show examples of eye-centered responses (R_eye statistically higher than R_head) during the sensory period (A and C) and during the motor burst (B and D). The two responses in A and B (from the same cell at different time windows) show a complete sampling of the receptive field, whereas the examples in C and D show a partial sampling of the receptive fields. E: head-centered responses (R_eye statistically smaller than R_head) during the sensory period. F: head-centered responses during the motor burst. G: hybrid-complex responses (R_eye not statistically different from R_head and both not statistically different from zero) during the sensory period. H: hybrid-partial shift responses (R_eye not statistically different from R_head but at least one of them statistically higher than zero) during the motor burst. I: untuned responses during the sensory period. J: untuned responses during the motor burst. The reference frame index R was not calculated for the responses not significantly modulated by the target location as shown in I and J. In A–J, the thin, colored horizontal lines represent the average baselines for the 3 different fixation locations. Ipsi, ipsilateral; Contra, contralateral.

Some neurons were classified as head-centered on the basis of the quantitative comparison of the reference frame indexes (R_head > R_eye), but as can be seen in Fig. 3, E and F, these response patterns were not as strongly head-centered as the eye-centered neurons were eye-centered. The values of R_head for these two examples were 0.68 and 0.58, whereas the values of R_eye were 0.31 and 0.28.

There were also neurons that exhibited hybrid response patterns. The example in Fig. 3G shows hybrid-complex tuning curves: two of the curves are well aligned in head-centered coordinates, representing the receptive fields computed from the left (red) and central (green) fixation locations, and two are well aligned in eye-centered coordinates, representing the receptive fields computed from the center (green) and right (blue) fixation locations. The example in Fig. 3H shows hybrid-partial shift tuning curves: the curves clearly shift laterally when the eyes move to different initial fixation locations, but not by the correct amount. In both of these examples, R_eye and R_head are about equal to each other (Fig. 3G: R_eye = R_head = 0.34; Fig. 3H: R_head = 0.81, R_eye = 0.80).

A final response pattern involved responses that were only weakly modulated by target location in any reference frame. Figure 3, I and J, illustrates examples in which responses exceeded baseline for all locations tested but for which there was little evidence of a circumscribed receptive field among the locations tested. Target-evoked responses even occurred for locations well into the ipsilateral field of space. Note that these responses were not necessarily identical for all locations and could vary with eye position (e.g., the curves in Fig. 3, I and J, show an overall gain sensitivity to eye position). However, the lack of spatial sensitivity in these neurons makes it impossible to evaluate their reference frame. Accordingly, we tested for spatial sensitivity using an ANOVA involving the 5 targets at −12°, −6°, 0°, 6°, and 12° (Table 1; see materials and methods and results, Overview). Neurons that failed to show spatial sensitivity in either head- or eye-centered coordinates made up about half of the sample in FEF (as well as LIP/MIP), and these neurons were excluded from the population analyses described in the next section.

Population Results in the FEF

Eye-centered responses in the FEF are prevalent in both sensory and motor periods, corresponding to ~60% of the population. Figure 4, A and B, shows this quantitatively. Like Fig. 2, D, F, H, and J, these graphs plot R_eye vs. R_head. The cross hairs indicate the 95% confidence intervals on those values (see materials and methods). Data points are color coded orange if the 95% confidence intervals suggest R_eye > R_head (the 95% range of R_head is lower than the 95% range of R_eye), blue if R_head > R_eye, and gray if R_eye and R_head are approximately equal; gray shading corresponds to R_eye ≈ R_head and either R_eye > 0 or R_head > 0 (dark) and R_eye ≈ R_head  ≈ 0 (light). The pie charts in Fig. 4, A and B, indicate the percentages of cells classified as eye-centered (orange), head-centered (blue), or the various subtypes of hybrid coding (light and dark gray).

The pattern of reference frames was similar in the sensory vs. motor periods, and there was little evidence of a systematic change in the representation when timescale was investigated more closely. Figure 4, C and D, shows the average R_eye and R_head values in 100-ms time bins sliding in 50-ms increments from target onset (Fig. 4C) and from saccade onset (Fig. 4D). R_eye averages ~0.4, whereas R_head averages ~0.1, with little change except for slight upticks at target onset and saccade onset. Although R_eye is consistently significantly greater than R_head (filled symbols; t-test on each time bin, P < 0.05), its value is not very high in absolute terms. This is because only ~60% of FEF neurons are classifiable as having eye-centered responses, and among these, many R_eye values were not very high. On the whole, the reference frame in the FEF is more eye-centered than it is head-centered, but it is not fully eye-centered.

Comparison with LIP/MIP and the SC

We next asked how the degree of eye-centeredness in the representation in the FEF compares with the representations in parietal cortex and the SC. Figure 5 presents a comparison of the signals observed in each brain region under identical experimental conditions [see materials and methods and Lee and Groh (2012, 2014) and Mullette-Gillman et al. (2005, 2009) for descriptions of the 2 additional data sets recorded in different monkeys with the same technique and during the same task]. As in the analysis of the FEF, we focused on two measures. First, we compared the number of cells classified as eye-centered, head-centered, or hybrid across areas (Fig. 5, A and B). Second, we compared the time course of the population-averaged eye-centered reference frame in the LIP/MIP, FEF, and SC (Fig. 5, C and D).

The results indicate a continuum of reference frame across these brain areas, with LIP/MIP predominantly hybrid, SC predominantly eye-centered, and FEF intermediate between the two. In both sensory and motor periods, the proportion of eye-centered responses remains a minority in the LIP/MIP (increasing from ~20% to ~40% in time) and a majority in the SC (~80% consistently in the 2 time periods). The FEF is between the two trends, with ~60% eye-centered responses in both time periods. Conversely, hybrid responses dominate in LIP/MIP (from ~60% to ~40%) and fall to <40% in the FEF and to <20% in the SC. Head-centered responses are a minority in all three areas, although a trend can be seen in both periods with the LIP/MIP having around one-fifth of responses head-centered, to the FEF having fewer than 10%, to the SC having almost no head-centered signals (Fig. 5, A and B). The proportion of neurons classifiable as eye-centered across areas differed significantly according to a χ² test in both sensory and motor periods (Table 2; P < 0.05). Post hoc analyses comparing FEF to LIP/MIP and FEF to SC confirmed that during both sensory and motor periods, response patterns classifiable as eye-centered are more prevalent in FEF than in LIP/MIP and less prevalent in FEF than in SC (Table 2).

Table 2.

Comparison of eye-centered prevalence across areas

	LIP/MIP	FEF	SC	Chi-Square Test	Post hoc LIP/MIP vs. FEF	Post hoc FEF vs. SC
Sensory responses	28/125	112/174	140/162	χ2 = 122.81	χ2 = 51.46	χ2 = 21.76
				df = 2	df = 1	df = 1
				P = 0	P = 7 × 10−13	P = 3 × 10−6
Motor responses	20/121	94/160	121/161	χ2 = 98.50	χ2 = 50.94	χ2 = 9.76
				df = 2	df = 1	df = 1
				P = 0	P = 1 × 10−12	P = 0.0018

A similar picture emerges when the average values of the reference frame indexes across time are considered. In Fig. 5, C and D, the average value of R_eye for each brain area is plotted across time. Because the proportion of spatially selective cells varied in the three areas, we repeated the analysis twice: for the subpopulation of spatially selective cells (Fig. 5C) and for all recorded cells (Fig. 5D). In either population, the visual responses are significantly more eye-centered than head-centered in all areas, but with a clear grading from SC being the most eye-centered (average index around 0.6/0.8 during the trial) to FEF (average index around 0.2/0.4 during the trial, occasionally reaching 0.5) to LIP/MIP (average index consistently lower than 0.2).

DISCUSSION

Our quantitative assessment of the reference frame for visual space representation in the FEF shows that visual signals were mostly but not completely eye-centered. Around 60% of individual neurons were classified as eye-centered, whereas one-third of neurons had hybrid reference frames and a minority had weakly head-centered reference frames. At the population level, the eye-centered reference frame index averaged ~0.2–0.4 and was largely stable across time. Compared with those in the SC and LIP/MIP, the pattern of reference frame in the FEF was less eye-centered than in the SC but more eye-centered than in LIP/MIP, which is predominantly hybrid in its coding format.

There is controversy in the literature about the reference frames employed in the FEF and LIP/MIP. Caution is particularly warranted because terms such as “retinotopic” are sometimes used to describe visual organization when eye position is fixed or when visual sampling is shifted for each fixation position; such studies do not directly assess reference frames. The most apt comparisons with our experiments are studies that 1) systematically varied initial eye position, 2) sampled receptive fields along the same dimension as the variation in eye position, and 3) quantified the results at the population level. For a more detailed discussion of how the assessment of reference frame can go wrong when these conditions are not met, see Figs. 10 and 11 of Mullette-Gillman et al. (2009).

With these caveats in mind, our FEF results are actually on the whole consistent with the most relevant previous studies. The closest related studies involved visual memory-guided saccades in head-unrestrained monkeys (Sajad et al. 2015, 2016). These studies allowed natural variation in initial eye and head position and used a different analysis method to assess a variety of different reference frames distinguished in part on the basis of torsion (Keith et al. 2009). An eye-centered reference frame was the most common best match, although most other reference frames could often not be statistically ruled out. Several studies involving receptive field mapping at the time of eye movements have confirmed the proof of principle that many FEF receptive fields do not maintain a fixed position on the retina across eye movements (Sommer and Wurtz 2006; Umeno and Goldberg 1997, 2001; Zirnsak et al. 2014).

Studies employing electrical stimulation have also shown mixed reference frame information. Although stimulation does not typically drive the eyes to a fixed position in the orbits, the direction and amplitude of the stimulation-evoked saccade can vary with initial eye position (Bruce et al. 1985; Dassonville et al. 1992; Monteon et al. 2013; Robinson and Fuchs 1969; Russo and Bruce 1993). Monteon et al. (2013) reported that ~71% of sites yield saccades of a stable vector across different eye positions (i.e., eye-centered) and ~29% yield saccades that vary considerably, more consistent with a head- or body-centered coordinate system.

There is also a previous study of auditory signals in the FEF and how they are affected by eye position (Russo and Bruce 1994); we plan to consider this question further in a future paper.

The range of reference frames observed in the FEF provides a context for interpreting those observed in both the SC and LIP/MIP. The SC has been characterized as an eye-centered structure (Klier et al. 2001; Lee and Groh 2012; Schiller and Stryker 1972), whereas the LIP/MIP is not. Mullette-Gillman et al. (2005, 2009) were the first to describe the reference frame of LIP/MIP as predominantly hybrid. Andersen and colleagues (e.g., Andersen and Mountcastle 1983; Andersen et al. 1985, 1990) had already reported an interaction between visual inputs and eye position in the LIP/MIP, which they interpreted as reflecting eye centered gain fields. However, their sampling of receptive fields sometimes focused on the dimension orthogonal to the change in fixation position, which made their results difficult to interpret. Adopting a more appropriate sampling technique across brain areas, we have demonstrated not only strong eye-centered signals (the vast majority in the SC) but also signals along a continuum between eye-centered and head-centered coordinates (in different proportions, in all 3 areas). Thus the hybrid reference frames that we have characterized are unlikely to be due to methodological shortfalls. Furthermore, the identification, under the same conditions, of a gradual shift in the strength of eye-centered representations from LIP/MIP to FEF to SC supports the view that there is a genuine transition in coding between these different brain areas. This shift in coding was evident even though the different brain areas were assessed in different individual monkeys. How this observation may interact with potential differences in the breadth of receptive fields across these structures remains to be determined. In general, although these structures are reciprocally interconnected, it appears that there is a sequential element in how signals are processed from LIP/MIP to FEF to SC.

Why the brain uses reference frames that are impure, and why this should vary across brain areas, is unclear. The receptive field as a labeled line for stimulus location is a concept that dates back to the discovery of receptive fields. However, if receptive fields can vary in their position and do not show stability with respect to either eye- or head-centered coordinates, such neurons on their own cannot provide a labeled line for stimulus location. Rather, the activity of a population of such neurons would be required to disambiguate the spatial signals. Although this is very possible (e.g., Pouget and Sejnowski 1997), why it is desirable from a coding perspective has not been shown. Models of coordinate transformations show that the brain should be capable of transforming reference frames in a single step without the use of intermediate stages (Groh and Sparks 1992).

An understanding of how coordinate transformations unfold in a sensorimotor context has implications more broadly, because this is an example of a many-to-many mapping that occurs in numerous other contexts. For example, numerous types of perceptual constancy involve physical stimuli that vary but produce similar percepts (space constancy, size constancy, color constancy, the same musical melody in different keys, etc.). Such constancy can require a mapping from nearly any possible range within a physical stimulus dimension to a common perceptual dimension. Similarly, categorization requires mapping different types of physical stimuli that can be grouped into a category, and associative learning involves relating two different physical stimuli in a potentially arbitrary fashion. The underlying computation supporting these abilities may be very similar to that involved with spatial coordinate transformations.

GRANTS

This work was supported by National Institute of Neurological Disorders and Stroke Grant R01 NS50942-05.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

V.C.C. and J.M.G. conceived and designed research; V.C.C. and D.S.P. performed experiments; V.C.C. analyzed data; V.C.C. and J.M.G. interpreted results of experiments; V.C.C. and J.M.G. prepared figures; V.C.C. and J.M.G. drafted manuscript; V.C.C., D.S.P., M.A.S., and J.M.G. edited and revised manuscript; V.C.C., D.S.P., M.A.S., and J.M.G. approved final version of manuscript.