Imaging retinotopic maps in the human brain

Abstract

A quarter-century ago visual neuroscientists had little information about the number and organization of retinotopic maps in human visual cortex. The advent of functional magnetic resonance imaging (MRI), a non-invasive, spatially-resolved technique for measuring brain activity, provided a wealth of data about human retinotopic maps. Just as there are differences amongst nonhuman primate maps, the human maps have their own unique properties. Many human maps can be measured reliably in individual subjects during experimental sessions lasting less than an hour. The efficiency of the measurements and the relatively large amplitude of functional MRI signals in visual cortex make it possible to develop quantitative models of functional responses within specific maps in individual subjects. During this last quarter century, there has also been significant progress in measuring properties of the human brain at a range of length and time scales, including white matter pathways, macroscopic properties of gray and white matter, and cellular and molecular tissue properties. We hope the next twenty-five years will see a great deal of work that aims to integrate these data by modeling the network of visual signals. We don’t know what such theories will look like, but the characterization of human retinotopic maps from the last twenty-five years is likely to be an important part of future ideas about visual computations.

Introduction

Light absorption is a fundamental but insufficient competence for a visual system. Most organisms that absorb light have no sight: To see requires encoding the spatial structure of the image. In human the image spatial structure is preserved by many different optical and neural systems. The cornea and lens, and then the photoreceptor sampling mosaic, maintain the spatial arrangement of the image. The image spatial structure is further preserved by image processing within the retina; specifically, the receptive field centers of the retinal output neurons (ganglion cells) form an orderly mosaic that samples the visual field. While the spatial map is not fully preserved in a cross-section of the axons within the optic nerve (Fitzgibbon & Taylor, 1996, Horton, Greenwood & Hubel, 1979), the map is resurrected in the pattern of connections formed by axonal projections in the lateral geniculate nucleus.

It has been more than a century since Henschen (1893), Inouye (1909), and Holmes (1918a, 1916) discovered that the spatial arrangement of the image is maintained in primary visual cortex (V1): stimuli adjacent in the visual field are represented in adjacent positions in visual cortex. More surprising than the existence of a single V1 map was the subsequent discovery that many species have multiple retinotopic maps in visual cortex (Allman & Kaas, 1971, Cowey, 1964, Gattass, Nascimento-Silva, Soares, Lima, Jansen, Diogo, Farias, Marcondes, Botelho, Mariani, Azzi & Fiorani, 2005, Hubel & Wiesel, 1965, Talbot & Marshall, 1941, Talbot, 1940, Talbot, 1942 , Thompson, Woolsey & Talbot, 1950, Tusa, Palmer & Rosenquist, 1978, Zeki, 1969b, Zeki, 1971, Zeki, 1976 ), including animals like mice with very poor visual acuity (Wang & Burkhalter, 2007). The value of arranging neurons into multiple retinotopic maps, so that each location in the visual field is represented many times in cortex, calls for an explanation (Barlow, 1986). Perhaps the need to combine information from nearby locations in the image remains important to many cortical functions (stereo, motion and color); it is sometimes argued that certain types of efficiencies, such as minimal wiring costs, arise from using short axonal connections that reflect the computational objectives (Chklovskii & Koulakov, 2004).

While image spatial relationships are preserved in many regions of cortex, they are not absolutely preserved. There are important deviations (discontinuities) from retinotopy which may result from compromises between the multiple objectives of visual computations. For example, in primate the visual field is divided along the midline so that each hemisphere receives a spatial map of only half of each retina. Why the representation of the retina should have such a discontinuity in the primate cortex, but not other species (e.g., mouse) or even in all individuals of the same species (e.g., albinos (Guillery, Hickey, Kaas, Felleman, Debruyn & Sparks, 1984, Hoffmann, Tolhurst, Moore & Morland, 2003, Huang & Guillery, 1985, Morland, Baseler, Hoffmann, Sharpe & Wandell, 2001)) is an interesting question. Perhaps in primate the importance of binocular vision, coupled with limitations in axon guidance mechanisms, makes it necessary to divide the human V1 map into two parts in order to achieve binocular integration.

We summarize advances in understanding the number, organization and functional responses of visual field maps (also called retinotopic maps) in the human brain. We have been asked to emphasize discoveries made over the last twenty-five years, and we can report that during this period the advances were extraordinary. There are excellent reviews that emphasize the longer history (Glickstein & Whitteridge, 1987, Zeki, 1993) as well as reviews that focus on more recent developments (Silver & Kastner, 2009, Tootell, Dale, Sereno & Malach, 1996, Tootell, Tsao & Vanduffel, 2003, Wandell, Brewer & Dougherty, 2005, Wandell, Dumoulin & Brewer, 2007). Following our discussion of the past, we speculate on what may be in store for the next twenty-five years.

Cortical visual field maps

Progress in magnetic resonance imaging (MRI) technologies enabled measurements of the human brain that were beyond any expectations of the scientists working in 1985. These measurement technologies have been supported by new experimental methods and software tools that clarify the arrangement and properties of retinotopic maps in healthy human observers.

The three columns in Figure 1 offer a visual impression of the advances in brain imaging technology. In the mid-80s magnetic resonance imaging was in its infancy, and functional magnetic resonance imaging based on the blood oxygen signal had not yet been invented. The only method for imaging brain activity in healthy humans was positron emission tomography (PET) (Fox, Miezin, Allman, Van Essen & Raichle, 1987, Fox, Mintun, Raichle, Miezin, Allman & Van Essen, 1986). These PET images (Figure 1, left column) were among the first images of activity in V1 of healthy human subjects, and they also offered a glimpse of extrastriate activity. The PET data were sufficient to confirm some of the inferences about maps from neurology and electrocorticography in surgical patients (Brindley & Lewin, 1968, Dobelle & Mladejovsky, 1974, Dobelle, Turkel, Henderson & Evans, 1979).

Figure 1. Progress in measuring human visual cortex topography over the past 25 years.

Each column contains two images using the same technology. The images in the left column were measured using PET. The outline indicates the rough position of the border of the brain (sagittal view, occipital lobe on the right). The brightness measures the difference in PET signal when subjects viewed a uniform field and a contrast pattern presented either at the fovea (0.1–1.5 deg, top) or near fovea (1.5–5.5 deg, bottom). After Fox et al. (1986, Figure 3). The images in the middle column are the first measurements of human cortical topography using functional MRI (fMRI). The image planes are parasagittal and show regions near the calcarine sulcus (V1). The upper image measures response differences between visual contrast patterns in the left (blue) and right (yellow) visual field; the two slices are parasagittal planes from different hemispheres. The bottom image measures response differences to stimuli in the upper (yellow) and lower (blue) visual field. The color scale bars are p-values from a t-test of the response differences. The small white-line insets are approximately 1 cm. After Schneider et al. (1993, Figure 1). Recent fMRI measures, as in the right column, show the visual topography in multiple cortical maps. The anatomical underlay is a flattened representation of cortex near the occipital pole and including calcarine: dark indicating a sulcus and light a gyrus. The color overlay measures the visual field position that is most effective at stimulating each cortical position; the top image shows the most effective angle and the bottom image the most effective eccentricity. Boundaries between maps can be seen in the angle representation. For example, the boundary between V1/V2d is located at the lower vertical meridian representation (yellow); at this position the change in angle representation reverses direction (top image). Other boundaries can be found by similar reversals. There is good visibility of the V1–V3 eccentricity maps, and it is plain that V2 and V3 surround V1 (bottom). Distinct foveal representations can be seen in dorsal and ventral regions beyond the V3 border. These fall within other map clusters (V3A, VO-1, not indicated). After Schira et al. (2009, Figure 5). In all three columns the bottom images show the spatial resolution (voxel size) of the corresponding measurements. The z-axis is the slice thickness, and the x- and y-axes indicate the "in-plane" resolution. The ratios of the voxel volumes across the three studies are 1600:8:1. For PET, the inplane voxel size is the reported point spread function in the image (full-width at half maximum amplitude).

The images make clear that there are significant limitations to these PET measurements. First, the signal-to-noise is low so that the authors combined data from six different subjects. Combining data across subjects is not desirable because the V1 size and stereotaxic border positions vary greatly between subjects (Dumoulin, Hoge, Baker, Hess, Achtman & Evans, 2003, Stensaas, Eddington & Dobelle, 1974). The V1 size differences are not predicted by overall brain size and thus the size variance is not easily normalized away (Dougherty, Koch, Brewer, Fischer, Modersitzki & Wandell, 2003). Second, these PET measurements had coarse spatial resolution - a point spread function of 18 mm (full width at half the maximum amplitude). Perhaps because of this limitation, the authors could not improve on the map of human V1 proposed by Holmes and Lister (1918a, Holmes, 1944) which differed from V1 maps in other primates. Moreover, limitations in the data made it appear that primary visual cortex ‘failed to extend onto the lateral surface of the occipital lobe’, contrary to what is now routinely observed in functional imaging measurements. Third, there was limited ability to identify extrastriate maps from the extrastriate responses.

While these PET measurements were a very important step forward, many open questions remained. Summarizing the state of our knowledge of human visual cortex, Sereno and Allman (1991) wrote:

The only human visual area whose borders are surely known is V1. Recent advances in anatomical techniques for monitoring activity (e.g., positron emission tomography, Miezin et al. 1987) are beginning to change this. Fixed-tissue injections suggest that human visual areas V1 and V2 are organized quite similarly to those of other primates (Burkhalter and Bernardo, 1989). Also, there is a heavily myelinated, ellipsoidal region located in a dorsolateral occipital sulcus (Fig. 7.5) that may correspond to human visual area MT.

Anatomical MRI

Horton and Hoyt (1991b) combined the spatial resolution of anatomical MRI with neurological investigations of cortical damage, making two important advances. First, reporting on subjects with focal lesions in occipital cortex, they were able to correct some inaccuracies in Holmes and Lister’s visual field map, showing that the map failed to allocate enough cortical territory to the central visual field. This measurement brought the human map into better agreement with estimates from closely related nonhuman primates.

In a second paper, Horton and Hoyt (1991a) used anatomical MRI to draw conclusions about two human extrastriate maps, V2 and V3. They analyzed images from two subjects with quadrantanopia, a homonymous field defect with a sharp edge on the horizontal meridian. Prior to this analysis, the cause of a sharp loss of vision at the horizontal meridian was uncertain. Holmes (1918b) suggested that optic radiation fibers carrying signals from the upper and lower visual fields were separated, perhaps by the ventricle (Monbrun, 1919); a sharp quadrantic field defect could be explained by a lesion to one of the two parts of the optic radiation. Using anatomical MRI, Horton and Hoyt could see lesions located in extrastriate cortex at locations that appeared to correspond to V2 and V3 gray matter, rather than in the optic radiation. They acknowledged that in human there was uncertainty about the locations of these maps, writing: “Little is known about the organization of extrastriate visual areas in the human brain. Therefore, to construct our proposal we must draw upon data from experimental work in monkeys. Our argument hinges upon the topographic arrangement of the first 3 cortical visual areas: V1, V2 and V3.” They concluded that the quadrantanopia was explained by cortical lesions to V2/V3; in turn, they used their analysis of quadrantanopia to support the hypothesis that human V2 and V3 surround V1, as they do in nonhuman primates (see below).

Anatomical measurements continue to be important, although these developments have been somewhat overshadowed by the ability to make functional measurements. Among the advances in anatomical measures we can list better identification of different brain tissues, including gray matter and white matter; analyses of the geometry of cortical folding patterns; measurements of cortical thickness; and the assessment of integrity of different types of tissues (Deoni, Rutt, Arun, Pierpaoli & Jones, 2008, Fischl & Dale, 2000, Meyers, Laule, Vavasour, Kolind, Madler, Tarn, Traboulsee, Lee, Li & MacKay, 2009, Nordahl, Dierker, Mostafavi, Schumann, Rivera, Amaral & Van Essen, 2007, Sowell, Thompson, Leonard, Welcome, Kan & Toga, 2004). These measures have been applied to understanding developmental disorders or disease conditions, notably blindness (Noppeney, Friston, Ashburner, Frackowiak & Price, 2005, Park, Lee, Kim, Park, Oh, Lee & Kim, 2009, Shimony, Burton, Epstein, McLaren, Sun & Snyder, 2006). There also have been significant developments in both MR acquisition and analysis methods - particularly those based on diffusion-weighted and spectroscopic imaging. In the final section of this article we return to describe some of these methods, and how they are applied to understanding human visual field maps (Edden, Muthukumaraswamy, Freeman & Singh, 2009, Kim, Ducros, Carlson, Ronen, He, Ugurbil & Kim, 2006, Muthukumaraswamy, Edden, Jones, Swettenham & Singh, 2009).

Functional MRI

The development of fMRI was rooted in the systematic study of MR contrast mechanisms carried out by S. Ogawa and his collaborators. In a series of studies using animal models, Ogawa and colleagues demonstrated an in vivo magnetic resonance contrast that is blood oxygen level dependent (BOLD) (Ogawa & Lee, 1990, Ogawa, Lee, Kay & Tank, 1990, Ogawa, Lee, Nayak & Glynn, 1990). They recognized that the blood oxygen level, in turn, depends on neural activity. The work by Ogawa and colleagues in animal motivated several groups to examine whether these BOLD effects could also be measured in human; in 1992 three groups reported a BOLD signal in human cortex with two groups showing activation in visual cortex (Kwong, Belliveau, Chesler, Goldberg, Weisskoff, Pncelet, Kennedy, Hoppel, Cohen, Turner, Cheng, Brady & Rosen, 1992, Ogawa, Tank, Menon, Ellermann, Kim, Merkle & Ugurbil, 1992) and one in motor cortex (Bandettini, Wong, Hinks, Tikofsky & Hyde, 1992). Images of the functional responses from the two papers that measured in occipital cortex are reprinted in Figure 2.

Figure 2. Pioneering fMRI measurements in human visual cortex.

(A) The images are (a) T1-anatomicals showing the calcarine sulcus, (b) a gradient echo image in the same slice, and (c) the difference (arbitrary units) between the average of eight gradient echo images acquired during photic stimulation and eight acquired during darkness. The three time series plots (d) are the response levels of the gradient echo image as light stimulation is introduced and removed. The region of interest for each time series is labeled by the red squares in (b). After Ogawa et al. (1992, Figure 1). (B) These axial images are inversion recovery (IR) measurements; the first is the baseline and the remaining images are the difference from baseline at various points in time. The IR measurement is sensitive to blood flow, and the bright image regions are functional responses to photic stimulation. The time series in the bottom is the mean IR level in a 60 mm² region within visual cortex as light is turned on and off. After Kwong et al. (1992, Figures 1 and 2)

While Ogawa and colleagues’ work made clear that the BOLD response was connected to neural activity, there remained much uncertainty about the specific cellular and molecular mechanisms mediating the relationship between neural signals and BOLD. This uncertainty raised questions about the the value of BOLD to neuroscience and in particular the spatial resolution of the technique (Frahm, Merboldt, Hanicke, Kleinschmidt & Boecker, 1994, Kim, Hendrich, Hu, Merkle & Ugurbil, 1994, Turner, 2002). The neural mechanisms that give rise to BOLD remain under active investigation (Lauritzen, 2001, Logothetis, 2008, Logothetis & Wandell, 2004, Nir, Fisch, Mukamel, Gelbard-Sagiv, Arieli, Fried & Malach, 2007, Viswanathan & Freeman, 2007), but much progress has been made and certain general principles are established - such as the fact that the BOLD signal is not driven uniquely by action potentials but rather reflects a range of metabolically demanding neural signals. Despite an incomplete understanding of the full set of cellular mechanisms, fMRI is a useful tool for noninvasively studying responses in the human brain (Bandettini, 2009), and is especially well suited for measuring visual field maps in individual human subjects.

Visualization

Shortly after the demonstration of functional responses in human, several research groups developed experimental and software methods to identify and characterize the human visual field maps. Engel et al. (1994, 1993) (see also (DeYoe, Bandettini, Neitz, Miller & Winans, 1994)) introduced a method for measuring retinotopic maps efficiently. The method is based on stimuli that create traveling waves of activity in primary visual cortex. One stimulus comprises a set of rings of increasing radius; this expanding ring stimulus is designed to measure eccentricity maps (distance from the center of gaze). A second stimulus comprises a set of wedges, each with its tip at the center of gaze but extending in different directions; responses to these wedge stimuli are designed to measure angle maps (orientation with respect to the center of gaze). Combining data from rings and wedges several groups identified visual field maps (DeYoe, Carman, Bandettini, Glickman, Wieser, Cox, Miller & Neitz, 1996, Engel, Glover & Wandell, 1997, Sereno, Dale, Reppas, Kwong, Belliveau, Brady, Rosen & Tootell, 1995). This approach, sometimes called phase-encoded retinotopy or traveling-wave methods, has become a standard technique in human neuroimaging. Some groups use a related method, in which wedge and ring fragments are presented in independent, quasi-random orders (m-sequences); visual field maps can be identified by analyzing the stimulus-referred fMRI response to each of the fragments in each voxel (Hansen, David & Gallant, 2004, Vanni, Henriksson & James, 2005).

In addition to the fMRI measurement methods, a variety of visualization methods have become common (Figures 1, 3, 4 and 5). These methods begin by segmenting the white and gray matter from anatomical images. The white matter is generally surrounded by gray matter (but not in the ventricles), so that the boundary between white and gray forms a surface. This surface can be defined using a triangular mesh; the triangles are built on the exposed faces of the white matter voxels. Statistical summaries of the fMRI time series at different points in the gray matter (e.g., coherence, phase, statistical significance) are visualized by coloring the triangles that are adjacent to the gray matter (Dougherty, 2010, Goebel, 2010, Smith, van Essen, 2010).

Figure 3. The visual field eccentricity map in human primary visual cortex (V1).

The image at the top is a sketch of the estimated eccentricity map in calcarine cortex as deduced from lesion data (Horton & Hoyt, 1991b). The image on the lower left shows the arrangement of V1, V2 and V3 in a single human subject. The image at the lower right shows the eccentricity map in the same subject. The color bar for this image represents log-scaled eccentricity. The field map locations and eccentricity map were measured with fMRI using pRF methods (Dumoulin & Wandell, 2008).

Figure 4.

Angle measurements in posterior occipital cortex shown on a very smoothed representation of the white matter surface. The small image on the left indicates the region shown in magnified form on the right. The smoothed cortical surface is shown as if the viewer is behind the occipital pole and looking forward. Light and dark shading indicates gyri and sulci, respectively. The calcarine sulcus (CalS), collateral sulcus (CoS), fusiform gyrus (FG), and occipitotemporal sulcus (OTS) are indicated. The image on the right shows the angle data and map outlines overlaid on a magnified representation of the inflated hemisphere. The color indicates the angle (with respect to the center of gaze) that most effectively stimulates each location. The eccentricity representation generally changes along a direction perpendicular to the angle maps; that is, eccentricity changes along the iso-angle contours. Central field representation positions are denoted by the white asterisk. These maps are also shown in a conventional view in Figure 5. After Winawer et al. (2010).

Figure 5. Posterior visual field maps shown on a slightly smoothed boundary of the cortical surface.

The maps are indicated by the colored and labeled regions. This image shows only some of the reported maps. Additional maps have been reported in the intraparietal sulcus (IPS-0-4) and anterior ventral occipital cortex (para-hippocampal cortex) (PHC-1,2). Other labels as in Figure 4.

In a standard 3D rendering of the mesh, the responses in the sulci are occluded. To make these responses visible, it is possible to inflate or smooth the mesh. Alternatively, cuts can be introduced into the mesh and the nodes of the triangles can be transformed to fall within a plane without introducing any folds. This computational procedure, which is called flattening the cortical surface, was pioneered by investigators studying the macaque cortex (Cragg, 1969, Daniel & Whitteridge, 1961, Van Essen & Zeki, 1978). The flattening method also can be applied to the cortical sheet directly (Sincich, Adams & Horton, 2003, Tootell, Silverman, Switkes & De Valois, 1982). The method is widely used in human neuroimaging (Carman, Drury & Van Essen, 1995, Dale, Fischl & Sereno, 1999, Fischl, Sereno & Dale, 1999, Wandell, Chial & Backus, 2000). The images in Figures 1c, 3, 4 and 5 illustrate the relationship between V1, V2 and V3 using both flattened meshes and smoothed (or inflated) 3D representations.

Identifying visual field maps

A large fraction of V1 is located within the calcarine sulcus, located on the medial surface of the occipital lobe; the sulcus is large and identifiable in virtually every human subject (Figure 3). As Henschen (1893) inferred, each hemisphere has a contralateral hemifield representation. As Inouye (1909) discovered, the eccentricity representation (fovea to periphery) runs from the occipital pole to anterior calcarine. The near foveal cortical representation occupies a large surface area compared to the peripheral representation. The expansion of the foveal representation is often called cortical magnification and the magnification is quantified as the length of cortex per degree of visual field representation. The foveal expansion in human is quantitatively similar in non-human primates (Brewer, Press, Logothetis & Wandell, 2002, Fize, Vanduffel, Nelissen, Denys, Chef d’Hotel, Faugeras & Orban, 2003, Qiu, Rosenau, Greenberg, Hurdal, Barta, Yantis & Miller, 2006) and approximately matches the visual field sampling density of the cones and ganglion cells (Rodieck, 1973, Wassle, Grunert, Rohrenbeck & Boycott, 1990). The map inferred from anatomical lesions (Figure 3, upper panel) corresponds well to the eccentricity representation measured in a single individual using fMRI (Figure 3, lower right). There is significant variability between individuals in the size of human V1 (Andrews & Pollen, 1979, Dougherty et al., 2003, Stensaas et al., 1974).

The boundaries between visual field maps are typically defined by the locations of the vertical meridian representations (Figure 4). For example, in nonhuman primates the organization of the V2 and V3 maps surrounding V1 was first understood using anatomical lesions of the corpus callosum that identified the positions of the vertical representations (Cragg, 1969, Zeki, 1969b). In human the fMRI angle maps and specifically the vertical meridian representations are also the key markers used to identify visual field map boundaries (DeYoe et al., 1994, Engel et al., 1997, Sereno et al., 1995).

The angle maps in Figure 4, drawn on a smoothed representation of cortex to expose the sulci, include multiple vertical meridian representations. The change in the eccentricity representation, while not shown, is usually in the direction perpendicular to the angle maps; that is, eccentricity changes along the iso-angle contours. The position of the central field representations are denoted by the white asterisk.

The V1 map is on the medial surface of the occipital lobe, extending around the pole. Its boundaries can be identified by the lower vertical meridian (red band) at the V1/V2d border, and the upper vertical meridian (blue band) at the V1/V2v border. The vertical meridians are clearly visible in the region that represents the near periphery.

Using fMRI, as well as single unit electrophysiology or cytoarchitectonic criteria, the boundaries between V1, V2 and V3 are difficult to distinguish in the fovea (Dougherty et al., 2003, Schira, Tyler, Breakspear & Spehar, 2009, Zeki, 1969a). In many papers, the foveal region of V1, V2 and V3 is simply described as the ‘confluent foveal representation’. The measurement limits in this region are evident in Figure 4 both from the disorganized angle map at the occipital pole, and from the fact that there is a region (uncolored) in the image in which responses are incoherent with the stimulus (below threshold). There are several reasons why the boundaries in the foveal region are difficult to measure in the fMRI angle maps: (a) fixation instability and the presence of a fixation marker interfere with measurements in the very central fovea (b) the voxel size (2.5mm) is large compared to the width of the foveal portion of the maps, and (c) in many subjects there are large veins (the dural sinuses) near these regions that introduce instrumental artifacts (Winawer, Horiguchi, Sayres, Amano & Wandell, 2010). Recently, using high resolution and optimized methods, Schira et al. (2009) traced the angle maps to the central fovea and showed that the foveal representations of V1, V2 and V3 are distinct (Figure 1c).

The V2/V3 boundary is unusual in that it arises at a horizontal (green), not vertical (red or blue), representation. The dorsal V1/V2 boundary represents the lower vertical meridian; the angle map continues toward the horizontal meridian that defines the dorsal V2/V3 boundary, where it then reverses back to the lower vertical meridian. There is a corresponding reversal at the horizontal representation separating V2/V3 on the ventral surface. The concentric arrangement of V1-V3 splits the V2 and V3 maps at the horizontal midline into dorsal and ventral subdivisions -referred to as V2d, V2v and similarly for V3. The reversals in the direction of change of the angle maps distinguish V1-V3; the eccentricity maps are aligned with one another (Figure 3).

From this summary it is clear that there are important discontinuities in the V2 and V3 maps. The V2 and V3 maps have both the right/left hemifield discontinuity and a second upper/lower field discontinuity. In human, the split horizontal meridian arrangement is not present in other extrastriate maps, so that the horizontal discontinuity in V2 and V3 is the exception, not the rule.

While in many respects the V1-V3 human and macaque cortical maps are similar, most importantly in the concentric arrangement, there are differences as well. In macaque a great deal of V1 is located on the operculum, posterior and lateral to the medial position of the calcarine sulcus; human V1 sometimes extends in the posterior direction, from the calcarine onto the occipital pole and lateral surface; this posterior lateral extension is not as large or typical as in macaque. A particularly salient difference between the species is that macaque V3 occupies a very small surface area compared to V1 and V2 (Burkhalter, Felleman, Newsome & Van Essen, 1986, Gattass, Sousa & Gross, 1988), whereas human V3 is larger (Dougherty et al., 2003, Tootell, Mendola, Hadjikhani, Ledden, Liu, Reppas, Sereno & Dale, 1997). Perhaps because of the relatively small size of V3 in macaque, V3 is often omitted entirely from models of visual recognition (Bar, Tootell, Schacter, Greve, Fischl, Mendola, Rosen & Dale, 2001, Deco & Rolls, 2004, DiCarlo & Cox, 2007, Riesenhuber & Poggio, 2000). We stand here in support of studying the function of V3 ;).

Three additional maps, hV4, VO-1, and VO-2, are located on the ventral surface, adjacent to V3v. The hV4 angle map extends over a full hemifield, but not all subjects show a complete hemifield representation (Hansen, Kay & Gallant, 2007, Larsson & Heeger, 2006, Winawer et al., 2010). Also, notice that the hV4 eccentricity map does not follow along the full length of V3v, but rather hV4 stops short and emphasizes the central visual field. The VO-1 map abuts the anterior portion of hV4 and a ventral portion of V3v. In this subject the VO-1 map spans a full hemifield. The VO-2 map abuts VO-1 and V3v; in this data set there is only a slight hint of the lower field representation. While the V1-V3 eccentricity map runs posterior-anterior, the VO eccentricity map runs lateral-medial with the relatively peripheral representation bordering V3v (Brewer, Liu, Wade & Wandell, 2005). Additional ventral maps (PHC-1/2) anterior to VO-2, but not shown in these images, have been reported (Arcaro, McMains, Singer & Kastner, 2009).

Another set of maps are located on the lateral occipital surface: LO-1, LO-2, TO-1, and TO-2 (Amano, Wandell & Dumoulin, 2009, Dukelow, DeSouza, Culham, van den Berg, Menon & Vilis, 2001, Georgieva, Peeters, Kolster, Todd & Orban, 2009, Huk, Dougherty & Heeger, 2002, Larsson & Heeger, 2006, Smith, Greenlee, Singh, Kraemer & Hennig, 1998)¹. These extend from V3d and cover a large portion of the lateral occipital cortex. The maps labeled TO-1 and TO-2 (temporal-occipital) fall within the region of motion-selective cortex that is frequently described as MT+ (DeYoe et al., 1994). The LO-1/2 and TO-1 maps all include red, green and blue regions consistent with visual field coverage that extends through a hemifield. Analyses of visual field coverage and functional responses of the LO-1/2 maps has been reported by (Amano et al., 2009, Larsson & Heeger, 2006).

Finally, another pair of maps, V3A and V3B is present on the dorsal surface. These maps both include angle responses that span the hemifield. The eccentricity representation in these maps does not align with V1-V3 (Press, Brewer, Dougherty, Wade & Wandell, 2001, Tootell et al., 1997, Wandell et al., 2005, Wandell et al., 2007), but rather is distinct rather like the VO-1/2 maps. Further dorsal, beyond the V3A and V3B maps running into the intraparietal sulcus, there are additional maps named IPS-0,1,2,3 and SPL-1 (Superior Parietal Lobule). IPS-0 was first reported by Tootell et al. (1998) and originally it was labeled V7. IPS-3 was described by Sereno et al. (Hagler, Riecke & Sereno, 2007, 2001) and sometimes referred to as LIP; SPL-1 was first described by Konen and Kastner (2008a). These maps and their functional responses have been described in other reports as well (Konen & Kastner, 2008b, Lauritzen, D’Esposito, Heeger & Silver, 2009, Levy, Schluppeck, Heeger & Glimcher, 2007 , Schluppeck, Glimcher & Heeger, 2005, Silver & Kastner, 2009, Silver, Ress & Heeger, 2005, Swisher, Halko, Merabet, McMains & Somers, 2007).

A summary of the positions of these maps, shown in more conventional lateral (upper left) and medial (lower right) views, and without an expansion of the sulci, is shown in Figure 5. Note that there is a region between the LO/TO maps and the VO maps where no reliable maps have been identified (Figures 4 and 5). This region is close to the transverse sinus, a large blood vessel that distorts the magnetic field. It is possible that this portion of cortex, unlike all the regions surrounding it, contains no maps; it may contain only object selective regions with no visual field representation. Alternatively, instrumental and methodological limitations may limit our ability to measure maps in this region (Winawer et al., 2010).

Maps and reliability

The V1-V3 cortical maps can be found using fMRI in nearly every subject by every skilled experimental group. In contrast, no group reports measuring all the extrastriate maps every time in every subject. There are several reasons for the difficulty in measuring maps beyond V1-V3. First, these maps are generally smaller than V1-V3, so that the center-to-center spacing per square degree of visual field is smaller. Second, in extrastriate regions the combined spatial receptive fields of the neurons in a voxel, called the population receptive field (pRF), spans more of the visual field than the pRF in V1-V3 voxels (Dumoulin & Wandell, 2008, Kastner, De Weerd, Pinsk, Elizondo, Desimone & Ungerleider, 2001, Smith, Singh, Williams & Greenlee, 2001, Tootell et al., 1997). When the pRFs of adjacent voxels overlap, stimuli in adjacent visual field positions produce only a small response difference; thus there is a smaller signal-to-noise ratio (SNR) available to specify the cortical map. Third, extrastriate maps appear to be more selective to specific types of stimuli, and we are not yet certain about the most effective stimuli to use for measuring these maps. Fourth, investigators are still discovering instrumental and biological limitations of the measurements. Some maps are located in hard to measure places, such as near the ear canals; other maps (or parts of maps) are near large veins that introduce MR artifacts that limit the ability to measure cortex (Winawer et al., 2010).

Given these challenges, investigators do not interpret the failure to measure a map in an individual as a rejection of the hypothesis that the map exists. If a map is found reliably in two or three subjects, but it is not found in five others, investigators do not compute the average and conclude the map does not exist. Rather, they accept that the map exists in some subjects and try to understand the reasons for the failure to find the map in the other subjects. As enumerated above, there are a large range of possibilities for the failure - including some that are mundane and others that are interesting. For example, it is possible that there is inter-subject variation in the presence and coverage of the maps; but most investigators adopt the hypothesis that the failures are instrumental limitations.

Organization of the maps

There are several major theories proposing overall organizations of the visual field maps. One influential theory proposed that early cortical areas separate into dorsal and ventral processing streams, and that each of these streams has a different computational objective (Goodale & Milner, 1992, Ungerleider & Mishkin, 1982). Using the distinct properties of afferent and efferent reciprocal connections (Rockland & Pandya, 1979), Felleman and Van Essen (1991) proposed a now iconic model of the hierarchical organization of visual areas (not necessarily maps). Young (1992) used multi-dimensional scaling to search for structure among the areas. Malach and his colleagues conceived of retinotopy as an organizing principle that was followed strictly in the early visual pathways and then gave way to a more loosely defined eccentricity bias such that different regions of cortex would be grouped according to whether they primarily receive input from central or peripheral retina (Hasson, Levy, Behrmann, Hendler & Malach, 2002). Finally, it has been observed that the visual maps tend to fall in clusters that share a common eccentricity map and perhaps a computational objective (Kolster, Mandeville, Arsenault, Ekstrom, Wald & Vanduffel, 2009, Wandell et al., 2005). These organizational schemes do not necessarily conflict; they are all tentative and each may have some merit. The principal value of identifying these organizational schemes may be to help theorists structure computational models of visual processing.

The multiplicity of visual field maps in so many species suggests their importance – but there is no direct demonstration that having adjacent locations in cortex respond uniquely to adjacent locations in the visual field is essential for visual perception. An interesting and open question is whether the presence of a regular spatial structure, such as a map, is essential for proper visual function (Horton & Adams, 2005). Neurological case studies report that achiasmic subjects (Prakash, Dumoulin, Fischbein, Wandell & Liao, 2010, Victor, Apkarian, Hirsch, Conte, Packard, Relkin, Kim & Shapley, 2000) and albino subjects (Hoffmann et al., 2003) can have spatial vision but unconventional V1 field maps. In a dramatic example, Muckli et al. (2009) recently observed the case of a young girl who developed with only one hemisphere (see also Werth (2006)). Even so, she is capable of spatial vision in both hemifields although her V1 visual field map differs from controls.

On the other hand, a variety of psychophysical experiments suggest that the spatial arrangement of the retinotopic maps has consequences for visual perception. The spatial extent of crowding - the phenomenon in which object recognition is degraded by the presence of nearby stimuli - appears to be determined by the spacing between the stimuli on the V1 retinotopic map (Pelli, 2008). In binocular rivalry, transitions in perceptual dominance from one eye to the other can follow waves that sweep out a path along the V1 map (Lee, Blake & Heeger, 2005, Wilson, Blake & Lee, 2001). Aftereffects of facial identity, resulting from prolonged viewing of the image of a particular face, are most pronounced when a test face is viewed in close retinal proximity to the adapting face (Afraz & Cavanagh, 2008). Moreover, a number of groups have reported that recognition of newly learned visual objects is poorer in locations in the visual field in which the objects were not learned (reviewed in Kravitz et al. (2008)). The results from aftereffects and object learning have been interpreted as evidence that the ability to recognize objects despite changes in position is not automatic; it requires learning at many different retinal locations. This stands in contrast to the claim that shape, for purposes of recognition, is represented independently of location (Biederman & Cooper, 1991). An interesting question to ask is whether perceptual judgments of individuals with disorganized retinotopic maps have behaviors that are localized within the visual field.

Visual field position is one of several types of features that are mapped onto the cortical surface. Local contrast orientation is also represented in orderly maps in V1 (Blasdel & Salama, 1986, Hubel, Wiesel & Stryker, 1978); the principal direction of motion is represented in MT columns (Albright, 1984). The discovery that different features are represented as maps at a range of spatial scales made investigators wonder about the relationship between the maps (Chklovskii & Koulakov, 2004, Swindale, 2001). Questions about the relationship between human maps remain largely unanswered because it has not yet been possible to measure maps reliably at multiple scales in the same subject. Retinotopic maps can be measured reliably (Kirson, Huk & Cormack, 2008); but finer scale maps such as orientation maps can only be measured under limited conditions at 3T (Cheng, Waggoner & Tanaka, 2001, Goodyear, Nicolle & Menon, 2002, Menon, Ogawa, Strupp & Ugurbil, 1997). There is the possibility that such measurements will become more reliable and routine in the future, say using Hahn spin-echo at 7T (Adams & Horton, 2009, Yacoub, Shmuel, Logothetis & Ugurbil, 2007).

Functional specialization and maps

In parallel with the development of human visual field mapping, investigators sought to understand response selectivity to different stimulus categories. Early functional studies focused on the broad question of specialization: Does cortex contain distinct regions that are specialized for particular perceptual functions?

Questions of function hold more interest than the identification of maps, but the functional parcellation of visual cortex is fraught with technical limitations. Most obvious is that the borders of the responsive region depend on arbitrary experimental decisions including the stimulus properties and the statistical thresholds used to identify a reliable response. Hence, the retinotopic mapping community has been cautious in accepting definitive functional parcellations. Similarly, the functional community was cautious concerning the application of retinotopic mapping procedures (“It would therefore be appropriate to treat the results obtained by this method with some caution, despite the appealing look of the well-displayed maps on flattened cortices (Bartels & Zeki, 2000)”). But, over the years investigators from the two communities have increasingly combined their results, so that cortical maps and functional responses are frequently measured together. The retinotopic maps provide investigators a reliable means for matching regions of interest across different subjects.

Just as mapping technology has evolved, so too thinking about functional responses is advancing. It is widely agreed that using the subtraction methodology to show that a cortical region is more responsive to one stimulus characteristic (such as color, depth or motion) than another does not imply that this region performs a single function, or that this function is computed only in this region. A recent approach is to replace the subtraction methodology with more extensive models aimed at improving our understanding of the neural computations performed within the visual pathways (Dumoulin & Wandell, 2008, Kay, Naselaris, Prenger & Gallant, 2008, Smith et al., 2001, Thirion, Duchesnay, Hubbard, Dubois, Poline, Lebihan & Dehaene, 2006). Another approach to understanding the function of cortical areas compares the similarity of BOLD responses within a region of interest to the similarity of judgments made by the subject (Brouwer & Heeger, 2009, Haushofer, Livingstone & Kanwisher, 2008, Weber, Thompson-Schill, Osherson, Haxby & Parsons, 2009).

Color and motion selective cortex

In their pioneering work, Zeki and colleagues (Lueck, Zeki, Friston, Deiber, Cope, Cunningham, Lammertsma, Kennard & Frackowiak, 1989, Watson, Myers, Frackowiak, Hajnal, Woods, Mazziotta, Shipp & Zeki, 1993, Zeki, Watson, Lueck, Friston, Kennard & Frackowiak, 1991) demonstrated one extrastriate region that responds more to moving stimuli than to matched stationary stimuli, and a second distinct extrastriate region that responds more to chromatic spatial patterns than to matched luminance patterns. The extrastriate regions are separated in cortex, which is called functional segregation. The region responsive to motion is on the lateral surface of the brain near the border of the occipital-temporal lobe; the region responsive to color is in ventral occipital cortex (Figure 6).

Figure 6. Pioneering measurements of functional specializations for color and motion measured using PET.

The responses to motion and color stimuli produce distinct spatial patterns of activation. (A) When a subject views a large set of colored rectangles and then a luminance-matched set of rectangles, regions near V1–V3 respond to both stimuli at roughly the same amplitude. There is a region on the ventral occipital surface that responds more powerfully to the colored rectangles. (B) When a subject views a set of moving random dots and then a single static frame of random dots, regions near V1–V3 respond equally. There is a region in lateral occipital cortex that responds more powerfully to the moving dots. After Zeki et al. (1991).

The color-responsive region in ventral occipital cortex was first identified by neurological (Meadows, 1974) and PET (Lueck et al., 1989) methods. It was then shown, using fMRI, that there is a retinotopic organization in the color responsive region. This was demonstrated by placing the color-exchange stimuli in either the upper or lower visual field (McKeefry, Watson, Frackowiak, Fong & Zeki, 1997). As stimuli shifted from upper to lower visual field, the response peaks shifted positions within ventral occipital cortex.

The presence of a color-responsive region in ventral cortex was confirmed by Hadjikhani et al. (1998) although there were disputes concerning nomenclature and the position of these hemifield responses with respect to the visual field maps. These differences were examined in a series of scientific exchanges spanning a decade (Brewer et al., 2005, Hadjikhani et al., 1998, Hansen et al., 2007, Tootell & Hadjikhani, 2001, Wade, Augath, Logothetis & Wandell, 2008, Zeki, 2003). At this moment we cling to the hope that the final word on this subject is in a recent report by the authors (Winawer et al., 2010).

Progress in resolving the relationship between color-selectivity and visual field mapping in ventral occipital cortex advanced slowly because it takes time to understand which parts of human cortex are organized as expanded version of macaque, and which parts have fundamentally different arrangements. In human, a critical region for color vision differs from macaque. Specifically, in human color responses are located in ventral occipital cortex (Meadows, 1974, Zeki, 1990), spanning hV4 and the adjacent VO maps. By contrast, the same color-exchange experiment in macaque produces responses that span ventral and dorsal cortex (Wade et al., 2008).

The strong response to moving stimuli on the lateral occipital lobe was first measured using PET (Watson et al., 1993, Zeki et al., 1991) (Figure 6b) and then confirmed using fMRI (DeYoe et al., 1994, Tootell, Reppas, Kwong, Malach, Born, Brady, Rosen & Belliveau, 1995). DeYoe et al. (DeYoe et al., 1994) emphasized that this functionally-defined region might contain a multiplicity of visual field maps, in homology to the collection of maps that include and are adjacent to MT in nonhuman primates (Desimone & Ungerleider, 1986, Komatsu & Wurtz, 1988) and recommended referring to the motion-responsive region as MT+.

It seemed likely that a retinotopic map could be measured in human MT+, given that soon after the anatomical identification of MT in monkey (Cragg, 1969, Kuypers, Szwarcbart, Mishkin & Rosvold, 1965, Zeki, 1969a) it was shown to have a retinotopic map (Allman & Kaas, 1971). In practice, however, it was difficult to establish a retinotopic map using fMRI in human motion-selective cortex (Tootell et al., 1995). As methods developed and MR scanner technology advanced, it became possible to identify retinotopic organization in MT+, and several groups now report that the region appears to have at least two visual field maps (Amano et al., 2009, Dukelow et al., 2001, Georgieva et al., 2009, Huk et al., 2002).

There is another human map in dorsal occipital lobe, V3A, that responds powerfully to motion (Tootell et al., 1997). This map is located anterior to V3d and has an eccentricity map that is distinct from the representation for V1-V3 (Figure 3). The response properties within the V3A map were examined in an important paper by Tootell and colleagues that showcases the power of fMRI to go beyond localization. They observed that (a) in many subjects the V3A foveal representation is separate from the confluent V1-V3 foveal representation, (b) it is possible to use the time course of the BOLD signal to estimate the spatial receptive field sizes in V3A, and (c) the fMRI functional selectivity measured in human V3A differs from the single-unit functional selectivity measured in macaque V3A, which is not reported to be strongly motion-selective (Gaska, Jacobson & Pollen, 1988, Vanduffel, Fize, Mandeville, Nelissen, Van Hecke, Rosen, Tootell & Orban, 2001).

Seeing form

Functional specializations for interpreting form, such as objects and faces, were measured using PET in the early 1990s (Corbetta, Miezin, Dobmeyer, Shulman & Petersen, 1990, Haxby, Grady, Horwitz, Ungerleider, Mishkin, Carson, Herscovitch, Schapiro & Rapoport, 1991, Sergent, 1994). Many such specializations were reported in lateral and ventral occipital cortex (Figure 7). In an important early study applying fMRI to object perception, Malach et al. (1995) compared responses to images of objects with texture patterns that were matched for various characteristics (e.g., stimulus contrast). These experiments revealed a cortical region that responds more powerfully to objects than to matched textures; the region is in lateral-posterior occipital lobe, adjacent to the posterior aspect of the motion-responsive cortex, and extending into ventral-occipital cortex. This large region is commonly described as the lateral occipital complex (LO or LOC). Subsequently, other regions were identified using fMRI that respond more strongly to one stimulus type over others, such as faces (Kanwisher, McDermott & Chun, 1997), places (Epstein, Harris, Stanley & Kanwisher, 1999), and words (Ben-Shachar, Dougherty, Deutsch & Wandell, 2006, Cohen, Dehaene, Naccache, Lehericy, Dehaene-Lambertz, Henaff & Michel, 2000).

Figure 7. The relative positions of visual field maps and object-selective regions.

Visual field maps and object-selective regions are near each other and sometimes overlap in lateral and ventral occipital cortex. In the left panel, visual field maps for LO-1,2 and TO-1,2 were measured in several subjects and warped into a common rectangular atlas space (Amano et al., 2009). Shown in this space the responses to a motion localizer (top; contrast of moving vs static dots) and an object localizer (bottom; contrast of intact vs scrambled objects) align with distinct maps. Areas TO-1 and TO-2 respond strongly to motion. LO-2 but not LO-1 responds strongly to objects. The smoothed hemisphere on the right is seen from below. The approximate location of several retinotopic maps and object areas are shown in a typical individual (courtesy of Weiner, Yoon and Grill-Spector). The more posterior object selective area (blue) overlaps the LO maps, in agreement with the left figure. The place-selective region on the collateral sulcus (green) partially overlaps the VO maps. The PHC maps were not measured in this subject, but would be expected to overlap with the rest of the place selective area, anterior to the VO maps (Arcaro et al.).

For the next decade, it was believed that LO and VO responses are not retinotopic, or contained only an eccentricity bias without an angular representation (Levy, Hasson, Avidan, Hendler & Malach, 2001, Tootell & Hadjikhani, 2001). However, a number of recently reported visual field maps overlap with these object-selective regions. For example, the retinotopic map, LO-2 (but not LO-1), is highly responsive to objects compared to textures (Amano et al., 2009, Larsson & Heeger, 2006). Similarly, ventral occipital maps (VO-2, PHC-1/2) overlap place-selective cortex. There is steady progress in retinotopic methods, and it has been proposed that over time we will discover that all of visual cortex is tiled by maps (Tyler, Likova, Chen, Kontsevich, Schira & Wade, 2005). It may be that the early distinction between retinotopic cortex and non-retinotopic, object-selective cortex was premature.

Attention

In the early 1990s PET experiments demonstrated that shifting spatial attention produces widespread responses in cortex (Corbetta, Miezin, Shulman & Petersen, 1993). In the late 1990s a number of investigators used the resolution and sensitivity of fMRI to show that shifting spatial attention to a visual field location produces responses at cortical locations that align with the several visual field maps (Beauchamp, Cox & DeYoe, 1997, Gandhi, Heeger & Boynton, 1999, Kastner, De Weerd, Desimone & Ungerleider, 1998, Ress, Backus & Heeger, 2000, Somers, Dale, Seiffert & Tootell, 1999, Tootell et al., 1998). For example, inducing subjects to shift spatial attention continuously from fovea to periphery produces fMRI traveling waves of activity in several cortical maps, including V1-V3 (Brefczynski & DeYoe, 1999). The spatial attention responses can be measured using fMRI in human V1, but such attention responses are not clearly identified in nonhuman primates (Maunsell & Cook, 2002). There are unresolved questions about whether these discrepancies are due to differences in species or measurement methods (Yoshor, Ghose, Bosking, Sun & Maunsell, 2007).

Modulating spatial attention evokes responses in many other extrastriate regions. Using attention manipulations, Tootell et al. identified a new map, V7, located in the posterior intraparietal sulcus (IPS). This map turned out to be one of a series located in the IPS, and each one appears to represent a hemifield (Press et al., 2001, Silver et al., 2005, Swisher et al., 2007). Manipulations of visual short-term memory, saccadic control, and multisensory stimuli also enhance IPS responses. Combining these task manipulations with stimulus-driven activity can be helpful in identifying additional IPS maps (Gandhi et al., 1999, Kastner, Pinsk, De Weerd, Desimone & Ungerleider, 1999, McMains, Fehd, Emmanouil & Kastner, 2007, Saygin & Sereno, 2008, Schluppeck et al., 2005, Silver et al., 2005, Swisher et al., 2007). The IPS maps beyond V7 were labeled IPS-1, IPS-2 and IPS-3. Swisher et al. (2007) suggested that for clarity V7 be renamed IPS-0 (see also Wandell et al. (2007)). Advances in the analysis of these maps and their functional properties are reviewed by Silver and Kastner (2009)

Sub-cortical visual field maps

There has been good progress in measuring retinotopic responses from several sub-cortical regions, including the lateral geniculate nucleus (LGN, Figure 8), superior colliculus, and the pulvinar (Chen, Zhu, Thulborn & Ugurbil, 1999, Schneider & Kastner, 2005, Ugurbil, Hu, Chen, Zhu, Kim & Georgopoulos, 1999), as well as identifying responses that arise from separate layers within the LGN (Schneider, Richter & Kastner, 2004). Responses from these regions are evoked or modulated by attention, binocular rivalry, spatial position judgments, and color (Cotton & Smith, 2007, Haynes, Deichmann & Rees, 2005, Kastner, O’Connor, Fukui, Fehd, Herwig & Pinsk, 2004, Kastner, Schneider & Wunderlich, 2006, Schneider & Kastner, 2009, Schneider et al., 2004, Smith, Cotton, Bruno & Moutsiana, 2009, Sylvester, Haynes & Rees, 2005, Sylvester & Rees, 2006). Both anatomical and functional analyses are being used to evaluate a thalamic role in the developmental disorder of amblyopia (Barnes, Li, Thompson, Singh, Dumoulin & Hess, Hess, Thompson, Gole & Mullen, 2009, Mullen, Dumoulin & Hess, 2008).

Figure 8. LGN responses to visual stimuli measured with fMRI.

Observers viewed flickering checkerboards (7.5 Hz) alternately confined to either the right or left visual hemifield; the lateral geniculate nucleus (LGN) and visual cortex responded to stimuli in the contralateral visual field. The central images are coronal (top) and axial (bottom) slices from a single, representative observer. The red/yellow and blue/green colors indicate significant responses to checkerboards on the right or left of fixation, respectively (scale in z-scores). The large regions of activation near the occipital pole are in visual cortex. The smaller, more anterior regions of activation (arrows) are LGN activations. After Kastner et al. (2004).

Measuring subcortical maps presents a particular set of technical challenges. Unlike cortical maps, the subcortical regions do not neatly tile a single 2 dimensional sheet. Hence the convenience of identifying retinotopic maps based on angle reversals at area borders is not available. Moreover transforming the 2-dimensional slices acquired in MR imaging into an appropriate visualization, such as computationally inflated meshes or flattened sheets, is not yet routinely applied to subcortical areas. Subcortical regions are small relative to cortical areas. The superior colliculus in particular is close to the brainstem, resulting in pulsatility that induces motion artifacts (DuBois & Cohen, 2000). With improvements in resolution, visualization, and MR technology, we expect to see significant progress in studying the function and organization of subcortical maps.

Comparative measurements

It has been nearly 70 years since the discovery of a second cortical visual field map (V2) in cat and rabbit (Talbot & Marshall, 1941, Talbot, 1940, Talbot, 1942 , Thompson et al., 1950, Tusa et al., 1978); it has been fifty years since a second map was described in the squirrel monkey (Cowey, 1964), and a third map in cat (Hubel & Wiesel, 1965). Yet, until the early 70s most thinking about visual cortex was dominated by measurements in primary visual cortex (V1). For example, in their important Ferrier Lecture, Hubel and Wiesel (1977) make only passing reference to signals in extrastriate maps. Over the last twenty-five years the emphasis has changed enormously. The Ferrier Lecture contributed by Zeki (2005) opens with the sentence ‘The visual brain consists of many different areas.’

The striking change in emphasis occurred during the period from the late 60s through the early 90s as investigators developed methods to parcellate extrastriate cortex in the nonhuman primate into visual areas (Allman & Kaas, 1971, Cragg, 1969, Dubner & Zeki, 1971, Felleman & Van Essen, 1991, Kuypers et al., 1965, Zeki, 1969a). Working with the tools of single-unit recording, it was quite difficult to establish the existence of a map because the field of view of these technologies is small. The process of using electrodes to identify a map was described as “a dismaying exercise in tedium, like trying to cut the back lawn with a pair of nail scissors (Hubel & Wiesel, 2005).” Perhaps because of this difficulty, the parcellation into multiple areas relied significantly on other criteria: (i) architecture, (ii) connectivity, (iii) visual topography, and/or (iv) functional characteristics’ (Van Essen, 2003). Differences in cortical regions as revealed by any of these measures raised the possibility that a new visual area might be found; in the context of this array of measurements, the presence of an organized visual field map (topography) was neither unique nor decisive. We emphasize that the distinction between a map and an area is significant (Wandell et al., 2005). This can be seen in the discussions about whether macaque V3 should be divided into two areas that together comprise one map (V3-dorsal and VP) (Burkhalter et al., 1986, Burkhalter & Van Essen, 1986, Lyon & Kaas, 2002, Orban, Van Essen & Vanduffel, 2004, Wilms, Eickhoff, Homke, Rottschy, Kujovic, Amunts & Fink, Zeki, 2003).

Despite these limitations, the identification of visual areas in nonhuman primate served as an important guide to understanding the organization of field maps in human visual cortex. Establishing a correspondence between human and nonhuman visual field maps offers the promise of using circuit level information obtained in nonhuman primates to model human cortex and behavior. One way in which the data might be brought closer is to establish methods to make more types of measurements in the human brain. One important circuit-level comparison that has been established is the presence of ocular dominance columns in V1. These columns can be identified using cytochrome oxidase markers in post-mortem analysis. There have been several reports that ocular dominance columns can be identified using fMRI (Adams & Horton, 2009, Cheng et al., 2001, Goodyear et al., 2002, Menon et al., 1997, Yacoub et al., 2007), but the method has not yet been mastered by the field or put to common use. There have also been very significant advances in measuring human cortex - including architecture, connectivity and even receptor distributions (Eickhoff, Rottschy, Kujovic, Palomero-Gallagher & Zilles, 2008, Zilles, Palomero-Gallagher & Schleicher, 2004). The ability to make comparisons based on an array of experimental measures in both human and macaque will surely deepen our understanding of the similarities and differences.

A second way to coordinate measurements between species is to compare quantitative measures derived by different techniques - say by comparing the size and properties of maps derived from single unit measurements in macaque to maps derived from fMRI in human. In recent years it has also been possible to use fMRI methods in both species to compare visual field maps (Brewer et al., 2002, Fize et al., 2003, Kolster et al., 2009, Wade et al., 2008). The fMRI measurements of the human and macaque visual field maps are similar, although there are quantitative differences. For example human maps are substantially larger than those in macaque, roughly 4x larger in V1 and an even greater factor for V3 (Brewer et al., 2002, Kolster et al., 2009, Tootell et al., 2003). The homology beyond V1, V2, and V3, is considerably more difficult to establish, particularly in inferior temporal and intraparietal regions (Orban et al., 2004). There has been a significant migration of fMRI techniques into single unit electrophysiology, so that investigators now use fMRI to identify cortical regions that are good targets for further single-unit investigations (Tsao, Freiwald, Tootell & Livingstone, 2006, Tsao, Vanduffel, Sasaki, Fize, Knutsen, Mandeville, Wald, Dale, Rosen, Van Essen, Livingstone, Orban & Tootell, 2003).

Just as there are reasons to understand the relationship between human and nonhuman primate maps, there are also reasons to pursue human measurements independently. Humans are a cooperative and intelligent subject population so that fMRI can be applied to perceptual problems that are difficult to approach in animals, particularly problems that involve judgments of appearance and learning. Furthermore, the differences in size and anatomical structure between human and nonhuman primates are substantial (Figure 9), so that coordination should depend on independent measurements in the separate species rather than assume that it is possible to generalize by simple size scaling.

Figure 9. Comparison of the human and macaque cortical surface.

(A) The image compares a macaque and human brain in post-mortem. From Sincich et al. (2003). (B) The renderings compare smoothed representations of macaque and human cortical surfaces. The shading indicates gyri (light) and sulci (dark). The surfaces are smoothed but the ratio of surface area between the two brains is maintained at scale.

In the early years of fMRI, investigators were frequently challenged to confirm that human measurements were consistent with animal models. This is not straightforward because there are a number of inter-species differences, even within nonhuman primates (Rosa & Tweedale, 2005). The comment below, from a leading group of investigators, represents a shift in the thinking of the field:

“The macaque brain is often described as a “model” for the human brain, but this is somewhat misleading. The macaque belongs to a completely different zoological family (Cercopithecidae) than humans (Hominidae), reflecting independent evolution over several million generations. The macaque model brain is not just a miniaturized version of the human brain, like a toy car or a doll. Thus studying human brain function is not just an exercise in confirming what is already known from animal studies. (Tootell et al., 2003)”

The stand taken by Tootell and his colleagues is becoming the norm. As confidence in the human fMRI measurement methods has increased, there has been a more balanced approach to integrating the work on nonhuman primates and human fMRI.

Computational modeling of fMRI responses

The signal-to-noise ratio (SNR) of fMRI responses in human visual cortex is large compared to PET; also the SNR in visual cortex is large compared to responses in anterior parts of the human brain. When SNR is low, investigators may be limited to demonstrating that a signal is present, and in service of this goal they may be forced to average over large extents of cortex and combine data from multiple subjects. The large SNR in visual cortex affords investigators the opportunity to develop quantitative models of the response time course in individual subjects; visual field mapping is an example. Studies in individuals are particularly helpful when considering how research might be applied to clinical applications.

In their pioneering papers, Tootell and colleagues (Tootell et al., 1995, Tootell et al., 1997) immediately took advantage of the high SNR in visual cortex and explored how stimulus properties influence many aspects of the response dynamics and spatial spread of the fMRI signal. For example, using an array of stimulus manipulations and data analysis techniques, they made inferences about the spatial receptive fields of neuronal populations within different field maps. This field continues to advance, so that there are now several reports measuring population receptive field properties across visual field maps and even within a single map at different eccentricities (Amano et al., 2009, Dumoulin & Wandell, 2008, Kay et al., 2008, Smith et al., 2001, Thirion et al., 2006, Winawer et al., 2010).

An example of the power of computational modeling comes from Kay et al. (2008). These authors derived models of the BOLD response in the early visual field maps in individual subjects. They used these models to predict the fMRI responses to a new set of images. They then measured the fMRI responses to these images and showed that they could predict the observed responses; further, from the responses they could infer which of the images the subject was viewing.

There have also been advances in data analysis tools. Early experiments often set statistical conditions using the subtraction methodology and strong statistical methods, whose main goal was to declare a significant difference in the response between two conditions. To augment the statistical reliability of a response, a difference would be declared reliable only if a set of contiguous voxels respond to a change in stimulus in synchrony (cluster size threshold). In recent years investigators have explored a different approach, using classifiers from machine learning to evaluate whether there is reliable information in the spatial pattern of activity in an array of voxels, say the voxels in a specific field map or a region of cortex (Haxby, Gobbini, Furey, Ishai, Schouten & Pietrini, 2001). These multiple voxel pattern analysis (MVPA) methods evaluate information in the responses about stimulus characteristics, or about the observer’s state of mind. Using MVPA techniques, it is possible to evaluate whether the fMRI response contains enough information to discriminate stimulus orientation, color or motion direction from the responses in individual visual field maps (Brouwer & Heeger, 2009, Haynes & Rees, 2005, Haynes & Rees, 2006, Kamitani & Tong, 2005, Serences & Boynton, 2007a, Serences & Boynton, 2007b). Such analyses may prove helpful in improving measurement sensitivity and in characterizing the spatial representation of information.

The information present - or the information absent - in the fMRI response within a map can provide guidance about the functional purpose of the neurons within a map. Investigators are considering how to best reason about the outputs of such classifiers (Norman, Polyn, Detre & Haxby, 2006). An example of the complexity is the following: Knowing that information necessary for a discrimination is present in the signal does not imply that information is used. For example, Haynes and Rees (Haynes & Rees, 2005) show that V1 signals contain information about grating orientation, even though the subject’s performance in discriminating the orientation was at chance (see also (Williams, Dang & Kanwisher, 2007)). Conversely, the MVPA may not detect the presence of critical information because of limitations in fMRI methods or the classification algorithm.

All of these methods, from statistical thresholding to receptive field modeling to MVPA, will benefit from a better understanding of the relationship between the neuronal signals and the fMRI response (Logothetis & Wandell, 2004). The mechanisms relating neuronal signals and fMRI responses is an active area of research, and as our understanding advances investigators are likely to improve both their data acquisition and analysis methods (Whittingstall & Logothetis, 2009).

To the future and beyond

White matter connections and visual field maps

The information processed within a visual field map arrives from somewhere and is sent to somewhere. To understand more about information processing within the maps, we must learn more about these inputs and outputs. The study of anatomical connections is fundamental to anatomy broadly, and there is a considerable body of knowledge about connections in nonhuman primates (Schmahmann & Pandya, 2006). But with a few exceptions (Burkhalter & Bernardo, 1989, Clarke & Miklossy, 1990) detailed information about the human connections is limited.

The limits in our knowledge are glaring, and there has been a great deal of recent interest in identifying anatomical connections. Experimental advances are proceeding in parallel at two scales. At small length scales, investigators are seeking to characterize the full set of connections within a cubic millimeter of a model brain, often the rodent (Lichtman, Livet & Sanes, 2008, Lichtman & Sanes, 2008). The full set of local connections is called the connectome. Investigators hope that measurement of the connectome will provide insights that generalize across mammalian cortex and specify a ‘canonical cortical microcircuit’ (Douglas, Martin & Whitteridge, 1989, Nelson, 2002).

At larger length scales, MR methods based on diffusion weighted imaging are being developed to identify connections. These methods measure the directional diffusion of water within the white matter (Basser & Jones, 2002, Moseley, Bammer & Illes, 2002) and then apply algorithms to estimate the tracts connecting different parts of the human brain (Hagmann, Cammoun, Gigandet, Meuli, Honey, Wedeen & Sporns, 2008, Sporns, Tononi & Kotter, 2005). There are a number of methods of measuring diffusion (Stejskal & Tanner, 1965, Tofts, 2003 ) as well as a variety of methods for tracing connections, which are called tractography algorithms (Behrens, Johansen-Berg, Woolrich, Smith, Wheeler-Kingshott, Boulby, Barker, Sillery, Sheehan, Ciccarelli, Thompson, Brady & Matthews, 2003, Conturo, Lori, Cull, Akbudak, Snyder, Shimony, McKinstry, Burton & Raichle, 1999, Mori, Crain, Chacko & van Zijl, 1999, Sherbondy, Dougherty, Ben-Shachar, Napel & Wandell, 2008a). Estimates at the larger length scale are also sometimes called the connectome. To distinguish the two concepts, some have proposed referring to the large-scale measurements of cortical projections as the projectome (Kasthuri & Lichtman, 2007, Sherbondy, Dougherty, Ananthanarayanan, Modha & Wandell, 2009).

A number of groups use tractography to measure pathways, such as the optic radiation, in the occipital lobe (Dougherty, Ben-Shachar, Bammer, Brewer & Wandell, 2005, Kim et al., 2006, Levin, Dumoulin, Winawer, Dougherty & Wandell, 2010, Powell, Parker, Alexander, Symms, Boulby, Wheeler-Kingshott, Barker, Koepp & Duncan, 2005, Sherbondy, Dougherty, Napel & Wandell, 2008b, Yamamoto, Miki, Urayama, Fushimi, Okada, Hanakawa, Fukuyama & Togashi, 2007) (see Figure 10). These measurements can be used to better understand visual disorders including the consequences of retinal dysfunction and optic neuritis and other clinical applications (Ciccarelli, Toosy, Hickman, Parker, Wheeler-Kingshott, Miller & Thompson, 2005, Taoka, Sakamoto, Iwasaki, Nakagawa, Fukusumi, Hirohashi, Taoka, Kichikawa, Hoshida & Sakaki, 2005, Toosy, Ciccarelli, Parker, Wheeler-Kingshott, Miller & Thompson, 2004, Trip, Wheeler-Kingshott, Jones, Li, Barker, Thompson, Plant & Miller, 2006). As the spatial resolution of the diffusion weighted data increases, and the sophistication of the algorithms for processing the measurements improve, we can hope to have a thorough characterization of the projections between retinotopic maps in the living human brain. At present, tractography has been useful in identifying (a) fibers that connect known areas, and (b) the retinotopic organization of fibers in the splenium (Dougherty et al., 2005, Saenz and Fine, 2010). With further advances in spatial resolution and precision it may be possible to use these methods as a tool for discovery of connections between visual field maps (Kim et al., 2006), or even the discovery of new maps.

Figure 10. Optic radiation in the human brain.

(A) The optic radiation prepared using Klingler’s fiber dissection technique in a post-mortem brain, as seen from below. The anterior extension of the fibers into the temporal lobe, Meyer’s loop, is indicated by the blue arrows. (B) Using diffusion-weighted imaging and fiber tractography, it is possible to identify the optic radiation in living subjects. The fibers are shown against a background of a T1 anatomical image. The fibers connect the lateral geniculate nucleus of the thalamus (red) and the calcarine sulcus. After Sherbondy et al. (2008a,Figure 4).

Dynamic measurements from identified maps

A weakness of fMRI is its inability to make temporally resolved measurements; the response of the vasculature is on the order of seconds while the response of the nervous system is on the order of milliseconds. Since the inception of fMRI, investigators have worked to find ways to integrate data from time-resolved modalities, including electro-encephalography (EEG), magneto-encephalography (MEG), and implanted electrodes (electro-corticography now called eCog) (Dale & Halgren, 2001, Murphey, Maunsell, Beauchamp & Yoshor, 2009, Sharon, Hamalainen, Tootell, Halgren & Belliveau, 2007).

There has been good progress in integrating visual field map measures with EEG recordings (Appelbaum, Wade, Vildavski, Pettet & Norcia, 2006). A limitation of EEG measurements is the large number of potential cortical and subcortical sources that contribute to the small number of measured scalp recordings. Knowing that the responses arise in occipital cortex, and that the solutions are likely to be grouped within visual field maps, provides useful constraints to select among the many possible cortical signals that explain the EEG data. Another way to increase the spatial resolution of electrical recordings is through the use of subdural electrical recordings, which are being increasingly carried out during or prior to surgery, both with subdural patch electrodes (Murphey et al., 2009, Murphey, Yoshor & Beauchamp, 2008, Voytek, Secundo, Bidet-Caulet, Scabini, Stiver, Gean, Manley & Knight, 2009, Yoshor, Bosking, Ghose & Maunsell, 2007) and microelectrodes capable of recording from individual neurons (Kreiman, Koch & Fried, 2000, Quiroga, Reddy, Kreiman, Koch & Fried, 2005). Combining knowledge of the fMRI field maps with data obtained from either EEG or implanted electrode measurements helps clarify the spatial location of the neural signals and provides a good method for combining data obtained in different subjects.

Quantitative MRI and molecular imaging within maps

Generally, MR signals are based on interactions between water and the cells and molecules at an extremely fine spatial resolution. Thus, even though the pixels in an MR image measure signals summed from, say, a cubic millimeter, the MR signal probes tissue properties at extremely small length scales. MR is a flexible technology so that experimental procedures can be designed to measure various aspects of how these water molecules are influenced by tissue properties. In certain cases, the MR measurement is quantitative in the sense that the values derived from MR signals have physical units. For example, diffusion yields an apparent diffusion coefficient (ADC) with physical units (m²/s). The ADC summarizes the interactions between water and tissue, and this interaction takes place at a spatial resolution that is much smaller than the 2 mm voxel size of typical MR diffusion data (Basser & Jones, 2002, Le Bihan, Mangin, Poupon, Clark, Pappata, Molko & Chabriat, 2001).

There are many opportunities to create quantitative MR measures to assess tissue properties. An approach to obtaining quantitative MR measurements is to measure the fundamental relaxation time constants, T1 and T2. For homogenous materials the relaxation is a mono-exponential time course, but in the mixed tissue of brain the T1 and T2 relaxations are multi-exponential because they include contributions from water and multiple types of tissues. It is possible to model the signals arising from these interactions, separating the relaxation time into multiple sources within a voxel. This analysis can inform us about tissue properties, such as the myelin density, within different brain regions (Deoni et al., 2008, Meyers et al., 2009).

There are also opportunities to use MR spectroscopy (MRS) to suppress the water signal and measure the concentration of important molecules, such as the inhibitory neurotransmitter GABA. The relatively low concentration of these molecules means that MRS measures aggregate signal over larger regions, say a few cubic centimeters. But even so it is possible to arrange the measurements so that most of the signal arises from a single field map. Recent work shows that the concentration of the inhibitory neurotransmitter, GABA, within V1 correlates with certain aspects of the MEG signal and as well as human perceptual performance (Edden et al., 2009, Muthukumaraswamy et al., 2009). Post-mortem receptor mapping shows that the concentration of GABA receptors is greater in V1 than in adjacent visual field maps (V2, V3), and that the density and laminar pattern of several other receptor binding sites change at the boundaries of visual field maps (Eickhoff et al., 2008). We can expect further advances in understanding the significance of GABA and other molecules, as well as their distributions across the different maps. It also seems likely that we will be able to use MRS measurements to learn about the development of field maps and the relationship between molecules, field maps, and visual function.

Conclusion

No scientist working in 1985 would have predicted the spectacular advances in measuring responses and structures in the living human brain. We have acquired a great deal of information about the number and arrangement of human visual field maps; we know something about the functional signals from different maps; and we are integrating this information with other measures - including neurology, EEG, MEG, quantitative MRI, MR-Spectroscopy, receptor mapping. The data analysis and visualization tools are much more powerful as well, riding the wave of the electronics industry. There is no reason to think that the pace of innovation in MR measurements or software tools will slow in the near future.

The 1980s were a time of great excitement in theoretical vision science. Computational vision was growing (Marr, 1982) and the field of visual perception transformed itself by applying linear systems theory and other mathematical tools to the understanding of pattern, motion and color vision (Wandell, 1995), a set of tools that had achieved prominence in earlier decades (Campbell & Robson, 1968, Graham & Nachmias, 1971, Robson, 1966). The enthusiasm of the era is evident in the summary of a meeting on the localization of cerebral function held in 1984. Phillips et al. (1984) explained that neurobiologists were confident that they would understand

“elusive secrets of the cortex … not in fifty years, but in five because our range of facts is now so great and hence we are able to speculate, as in the (last section, in ways which we could not have done even twenty years ago. This, in turn, allows us to explore new approaches, discount old ideas and pursue new ones, always with the knowledge that the one certain thing about the future of cortical studies is that there will be surprises which will alter radically our way of thinking about the brain (p. 356).”

The range of facts available to the working scientist continued to increase enormously over time, just as Phillips et al. expected. Current technology and measurements would be unrecognizable to the working scientist of the 1980s. Yet, the theoretical treatment of brain function would be quite recognizable to such a scientist. The next twenty-five years might be profitably spent developing theories to explain and integrate the wealth of accumulated data.

The most significant advance in visual field mapping over the last twenty-five years is the ability to measure space-resolved maps in human. The advances in our understanding of cortical maps in the human brain are undeniable, but there are several critiques that are commonly made about the limits of these human measurements, particularly fMRI. For example, the resolution of cellular and molecular measurements currently possible in human is less than what can be achieved in animal models. Also, the responses that the signals one measures from human cortex using MR are not precisely the same as the action potentials favored by electrophysiologists in animal models.

In response to these criticisms, we think it is worth observing that some aspects of cortical organization are better revealed at the spatial resolution of MR - and cortical maps are surely one of these structures. Specifically, we have learned that studies of mouse or monkey retinotopic maps would not have answered the question of the number, position, and size of human retinotopic maps. Also, the assertion that the integrative cortical signals measured by fMRI are not important signals is one side of an opinion that is debated in the physiological literature (Bullock, 1997, Logothetis, 2008). Human measurements should not be rejected on the grounds that fMRI and EEG do not uniquely measure action potentials.

Measurements in human have significance just because they are in human. Twenty-five years ago we already knew that it is impossible to generalize safely from animal models to human. For example, a human V1 lesion has devastating effects on vision, but a cat V1 lesion reduces contrast sensitivity by only 30% and raises orientation discrimination thresholds by a factor of two (Berkley & Sprague, 1979). The mouse is used increasingly as a model vision system, yet removal of mouse V1 only increases contrast threshold at peak frequency (0.20 cycles/deg) from 20% to 30% contrast (Prusky & Douglas, 2004). Such evidence suggests that human vision relies on V1 cortical processing more than other mammalian species which use subcortical signals and alternate pathways.

To understand human vision fully, it is essential to continue improving measurements and theory of human brain function, and to find ways to coordinate these measurements with data obtained from relevant animal models. We hope that these data will be accompanied by theories that advance our understanding of visual signals with enough specificity to guide the repair of damaged or diseased eyes and enable the construction of working artificial systems. The properties of the human retinotopic maps, measured with much painstaking work during the last twenty-five years, is likely to be an important guide in developing such theories.