Visual pathways and psychophysical channels in the primate

Abstract

The main cell systems of the retina that provide input to the striate cortex are now well described, although certain aspects of their anatomy and physiology remain contentious. Under simple stimulus conditions and in a threshold context psychophysical performance can often be assigned to one or other of these systems, and an identification of psychophysical channels with afferent pathways is justifiable. However, results from psychophysical studies using more complex stimulus conditions are more difficult to relate to ‘front end’ channels, and it is more difficult to separate the physiological contributions of afferent pathways from those of cortical mechanisms, in particular the separation of dorsal and ventral streams.

Barry Lee's main interest is the anatomy and physiology of primate retina and the relationship of retinal neurobiology to visual perception of colour and shape. After graduate work at University College in the UK and postdoctoral work at the University of Rochester, New York, he led a laboratory in the Department of Neurobiology at the Max Planck Institute for Biophysical Chemistry in Göttingen, concentrating on the primate visual system. Since 2000 he has been Professor of Biological Sciences at the College of Optometry, State University of New York.

Introduction

General features of the anatomy and physiology of the visual pathways of the primate have become well established in past decades (see Lee & Dacey, 1997; Dacey, 1999, 2000; Lee, 2004 for reviews). This overview first briefly summarises standard views of the retinal anatomy of these afferent pathways, and some of their physiological properties. It mentions issues that are still unclear. In the second section these pathways are discussed in a psychophysical context. There is much classical evidence for the presence of psychophysical channels for colour and luminance under threshold conditions, which may map closely onto the different pathways originating in the retina. However, psychophysical performance in more complex visual contexts, such as natural scenes, now receives much attention and in such contexts these pathways relation to function is much less clear; it may be difficult to distinguish between properties of afferent pathways and those of central mechanisms.

Basic retinal structure in the primate

Figure 1 summarises Old-World primate retinal anatomy; the standard pattern of connectivity of receptor to bipolar cell to ganglion cell within the eye is outlined in Fig. 1A, and in the Nissl-stained section in Fig. 1B the laminar retinal structure is evident. Figure 1C provides a cartoon of retinal wiring, focusing on the three main cell systems that project to cortex through the lateral geniculate nucleus. A more detailed description of primate retinal anatomy can be found elsewhere (Lee et al. 2010).

Figure 1. A standard view of primate retinal structure.

A, a view of the eye; the sketch of primate retinal wiring in C appears complex but is made up of standard retinal elements: receptors, bipolar and ganglion cells as indicated. The lamina structure of the primate retina is clearly delineated in B (scale bar 50 μm). C shows a sketch of primate retinal wiring. Each pathway conforms to the standard pattern (A), with receptors connected to bipolar cells to ganglion cells. The three main pathways sketched (parasol, midget and blue-yellow) are those providing the major input to striate cortex. Photomicrograph kindly provided by U. Grünert.

The cell systems providing the main input to the LGN are (from the top) the magnocellular (MC) pathway, which begins in parasol ganglion cells and projects to the striate cortex through the magnocellular layers of the lateral geniculate nucleus (LGN). There exist on- and off-centre cell types, each receiving input from various classes of diffuse bipolar. These diffuse bipolars almost entirely avoid input from short-wavelength (S) cones (Lee & Grünert, 2007). Input from medium- and long-wavelength (M and L) cones is thought to be derived at random from the underlying cone matrix. These cells have high achromatic contrast sensitivity, and very transient responses.

The second system is the parvocellular (PC) pathway beginning in the midget system. Midget ganglion cell anatomy strongly suggests that a single cone provides input to the centre through the one-to-one connectivity by which a single cone (in and near the fovea) provides input to a single midget bipolar cell which in turn contacts a single midget ganglion cell (Boycott & Dowling, 1969). Again, on- and off-centre varieties are present. Midget ganglion cells project to the PC layers of the LGN and thence to striate cortex. They display either M–L or L–M cone opponency, being excited by short and inhibited by long wavelengths or vice versa; they respond strongly to red–green chromatic modulation in a sustained manner and weakly to achromatic targets. In the sketch of Fig. 1C, these neurones are portrayed with red and green stripes; this reflects the fact that neighbouring ganglion cells may receive centre input from either an M- or L-cone and thus have opposite chromatic properties. This mixture of function within a single ganglion cell class of the same mosaic is unusual.

Lastly, S-cones provide input to several ganglion cell classes, of which the best described is the small bistratified cell (Dacey & Lee, 1994; Crook et al. 2009). Excitatory (on) input is provided from S-cones by the S-cone bipolar and off input from the M- and L-cones mostly through diffuse bipolars, giving +blue–yellow (B–Y) responses. There are other cells with S-cone input, and at least one has inhibitory input from the S-cones. Their retinal connectivity remains uncertain but the class is well established from LGN recordings (Derrington et al. 1984; Valberg et al. 1986; Szmajda et al. 2006). These cell classes project to the koniocellular layers of the LGN (Tailby et al. 2008). Responses to blue–yellow modulation are strong and sustained with little response to achromatic modulation.

These three cell groups may relate to luminance and chromatic channels demonstrated psychophysically. How far this assumption is justified, is summarized in a subsequent section.

Issues of interest

The previous section presents a textbook view, but several unresolved issues have important functional implications. This section briefly points out some that have received recent attention.

Parasol ganglion cells

Dendritic trees cover a field six to eight cones across in the central retina; this is smaller than often assumed in the psychophysical literature. Measured centre sizes for midget and parasol ganglion cells are quite similar (Lee, 2004). Central parasol cells in the macaque are estimated to respond to 20–30 cycles deg⁻¹ (cpd), close to the resolution limit for this species.

Parasol cells show a response minimum with equal luminance modulation (see below) but there is a residual response at twice the modulation frequency (Lee et al. 1989). This may be related to the effects of chromatic motion on luminance motion mechanisms (Cavanagh & Favreau, 1985; Mullen et al. 2003; Lee & Sun, 2009).

Recent recordings from arrays of parasol cells have indicated that there may be some degree of correlation of their activities (Shlens et al. 2006). Since parasol ganglion cells are implicated in the hyperacuities (Lee et al. 1994; Rüttiger et al. 2002), how these correlations may affect fine spatial processing is an interesting unresolved question.

Midget ganglion cells

The anatomy suggests that the centres of midget ganglion cells near the fovea are derived from a single cone. In most ganglion cells, centre diameter is defined by the spatial extent of the dendritic tree (Wässle et al. 1981), but midget ganglion cells are thus an exception to this rule. Measured receptive field centre sizes are much larger than expected on the basis of a single cone centre in the central retina. The reasons for this are probably partly optical (since the point spread function is larger than a single cone), but some measurements using interference fringes (McMahon et al. 2000) and adaptive optics (Sincich et al. 2009) have indicated a more complex picture. If this is so, current views of midget connectivity may have to be revised.

A second issue concerns the degree of specificity of connectivity in this pathway. Trichromatic colour vision is found only in primates among mammals. It evolved close to the beginning of primate evolution (Mollon, 1991). Although the blue–yellow system is phylogenetically ancient, the new red–green system presumably took over some existing cell class (Shapley & Perry, 1986; Wässle & Boycott, 1991). A centre derived from a single cone automatically confers cone specificity on the centre and some degree of red–green cone opponency, but whether or not surrounds are cone selective has generated much debate, with no clear conclusion (Lennie et al. 1991; Reid & Shapley, 1992, 2002). On evolution of this pathway, presumably surrounds were non-specific, but any selectivity would increase the strength of the chromatic signal (Lee, 2008), so some intermediate degree of specificity is plausible. A related question is cone specific connectivity in the retinal periphery. Midget morphology changes in peripheral retina. Although the one-to-one connection between cones and midget bipolar cells is maintained to high eccentricity, convergence of midget bipolar cells onto midget ganglion cells becomes pronounced (Boycott & Dowling, 1969). Nevertheless, strongly cone opponent responses are maintained in a substantial percentage of cells in the mid-periphery (Martin et al. 2001; Solomon et al. 2005), implying some selectivity in this convergence; there is no known anatomical substrate for this.

The blue–yellow pathway

Physiology of the small bistratified cell is now well described (Crook et al. 2009). Other cell classes are less well defined; in particular the blue-off/yellow-on cell is less well understood (Dacey et al. 2002). There is also an additional blue-on/yellow-off class (Dacey & Packer, 2003), and the functional relationship between these two classes with apparently similar chromatic properties is obscure. Lastly, the photosensitive, melanopsin-containing ganglion cell also receives S-cone input (Dacey et al. 2005).

Visual pathways and channels

Beginning with Barlow and others (Barlow & Levick, 1969; Barlow, 1972), there have been attempts to rigorously link retinal physiology and psychophysical performance. In primate vision, there was early evidence for independent luminance and chromatic channels with different temporal properties (Kelly & Norren, 1977). A luminance channel was thought to underlie psychophysical performance on photometric tasks, such as heterochromatic flicker photometry (HFP) and the minimally distinct border (MDB). There is strong evidence that the MC pathway provides a physiological substrate for these tasks (Lee et al. 1988; Kaiser et al. 1990). The recent suggestion (Chatterjee & Callaway, 2002) that the MC pathway has significant S-cone input (which does not contribute to luminance) has been refuted (Sun et al. 2006). Also, psychophysical detection of luminance modulation (flicker) closely matches the properties of this pathway (Lee et al. 1990, 2007). On the other hand, detection of chromatic changes in a S-, M- and L-cone space can be partitioned into detection by S-cone or |M-L| cone mechanisms (Cole et al. 1993) and it is plausible that these map onto the S-cone-driven ganglion cells and the |M-L| cells of the PC pathway.

It is remarkable that psychophysical performance at threshold is constrained at a retinal level. There are a lot of synapses between the retina and a behavioural response, yet links between retinal physiology and psychophysics can be robust; the ‘nothing mucks it up’ principle (Teller, 1980).

However, there are a number of complications to this simple picture. Firstly, grating acuity (Pokorny et al. 1968) and the hyperacuities (Morgan & Aiba, 1985; Rüttiger & Lee, 2000) demonstrate a luminance-like spectral sensitivity. It has been proposed that the |M-L| cone opponent pathways does ‘double duty’ and, though underlying red–green vision at low spatial frequencies, at high spatial frequencies it underlies achromatic spatial vision (Ingling & Martinez-Uriegas, 1983; Lennie & D’Zmura, 1988). This view originated with the observations of Wiesel and Hubel (1966) that PC pathway cells in the LGN displayed centre-surround structure, and so large-field stimuli evoked |M-L| cone opponent responses while opponency is less with small targets. The alternative is that the MC pathway supports an achromatic channel of spatial vision, a view at first discounted since MC pathway cells were thought to have large centres and thus poor acuity (de Monasterio & Gouras, 1975). As noted above, however, recent data have shown that the difference in centre size is much smaller than first thought. This issue is beyond the scope of this brief review.

A peculiarity in the link between ganglion cell physiology and psychophysical performance is that chromatic pathways (S-cone and |M-L|) give responses to stimuli that are not perceived. Figure 2 shows three conditions for which this is the case. In Fig. 2A the temporal frequency tuning of |M-L| ganglion cells to red–green chromatic modulation is compared to psychophysical sensitivity to similar stimuli. Psychophysical sensitivity falls off above 4 Hz and above 10–12 Hz the chromatic alternation cannot be seen. Yet ganglion cells respond vigorously to at least 30 Hz. A similar result holds for blue–yellow modulation and S-cone cells. Figure 2B shows another example; chromatic sensitivity to red–green modulation decreases rapidly with eccentricity, even with appropriately scaled stimuli, well before the one-to-one midget connectivity is lost. Midget ganglion cells at equivalent eccentricities have red–green chromatic sensitivity comparable to cells near the fovea. Lastly, grating visual resolution is higher with luminance gratings than with red–green equiluminant gratings (Mullen, 1985). However, resolution of PC pathways is similar to both grating types; independent of grating type, visual resolution is largely determined by centre size (Peichl & Wassle, 1979). Figure 2C is an example from the parafovea in which responses to both types of grating cut off near 10 cpd. This is much higher than parafoveal chromatic grating acuity. Again, it appears that central sites do not utilize a PC pathway signal. All these effects indicate that some aspects of the retinal signal are not utilized cortically, which substantially complicates the task of linking retinal physiology with psychophysical performance.

Figure 2. Three examples in which retinal signals are not perceptually utilized.

A, temporal frequency tuning for red–green modulation in psychophysics and PC pathway retinal ganglion cells, replotted from Lee et al. (2007 with permission from ARVO as the copyright holder). Psychophysical sensitivity (derived from Swanson et al. (1987)) and converted to cone contrast sensitivity. Physiological data are based on cell firing rates required to reach a response criterion. B, psychophysical and PC pathway sensitivity as a function of retinal eccentricity. Replotted from Martin et al. (2001). C, PC cell (+M-L cell, 4.8 deg eccentricity) responsivity to luminance (40% contrast) and red–green chromatic (100% modulation contrast) gratings (2 Hz drift rate) as a function of spatial frequency. Visual resolution is similar for both conditions.

Distribution of function between the PC and MC pathways is often considered in terms of their spatial and temporal properties complementing one another to cover a broader range of frequencies than either alone (Kulikowski & Tolhurst, 1973; Silveira, 1996). Part of this distinction rests on the fact that PC pathway cells (and S-cone cells) show very sustained responses. This can provide information about brightness and chromaticity of object surfaces; it is possible to derive psychological scaling of colour and brightness based on the activities of these neurons (Valberg & Seim, 1991). This differs somewhat from the textbook view of ganglion cell receptive fields as contrast or edge detectors, based on their centre–surround structure. A simple view of how PC and MC pathways deliver spatially distinct signals with complex patterns (in this case, colour Mondrians) was provided by Nothdurft and Lee (1982). Object boundaries derive from both luminance and colour, but shadows and shading tend to make chromatic properties more reliable indicators of material boundaries than luminance changes (Switkes et al. 1988; Kingdom et al. 2004). Algorithms for visual segmentation have found determination of surface characteristics (including chromatic properties) a useful approach (Li & Lennie, 2001; Maxwell et al. 2008).

There is now abundant evidence that colour in complex scenes provides a wealth of information about texture, structure and depth (see Shevell & Kingdom (2008) for a review from a psychophysical perspective). These data are sometimes difficult to relate to the apparent segregation of chromatic and luminance signals in the afferent pathways and in simple psychophysical detection tasks. Neurophysiologically, beginning in area 17 (Lennie et al. 1990), neurons are found which show chromatic properties intermediate between the characteristic chromatic signatures of the PC and KC pathways; also many neurons receive combined PC and MC input (Johnson et al. 2004). Given certain assumptions, linear combination of different pathways in cortex need not invalidate the ‘nothing mucks it up’ principle cited earlier, where psychophysical thresholds to simple stimuli map closely onto the properties of retinal neurons (or onto the cardinal axes of colour space; Krauskopf et al. 1982; Derrington et al. 1984). However, complex stimulus configurations appear to reveal ‘higher order’ chromatic mechanisms that might be generated at a cortical level (Krauskopf et al. 1986; Li & Lennie, 1997) but interpretation of these data is not straightforward (Eskew, 2009).

Nevertheless, complex spatial processing such as texture segmentation and pop-out can be based on chromatic cues; as pointed out by Shevell and Kingdom (2008), congruent (and incongruent) changes in luminance and chromaticity are an implicit aspect of normal viewing. For example, binocular vision, as a hyperacuity based on disparity, may be impaired with isoluminant patterns (Livingstone & Hubel, 1987), but binocular depth can readily be achieved with chromatic cues (Kingdom, 2003; Zaidi & Li, 2006). Various aspects of complex spatial processing may differentially rely on afferent pathways, but also psychophysical performance is likely to be heavily dependent on the separation of information in different cortical streams (see below). There have been numerous recent attempts to define cortical cells’ properties in these complex contexts (e.g. Nothdurft et al. 1999; Roe et al. 2005), although links between physiology and behaviour tend to be less robust in the cortex than at earlier stages in the visual pathway.

Concluding remarks

The identification of psychophysical luminance and chromatic channels with afferent visual pathways may remain valid for simple detection tasks, but in more complex spatial contexts the way in which central pathways handle afferent input make such correlations less clear. In particular, dorsal (parietal) and ventral (temporal) processing streams with different functional properties receive differential input, with dorsal pathways dominated by the MC pathway and the ventral pathway receiving both luminance and chromatic input (Merigan & Maunsell, 1993). This ventral pathway processing must involve mechanisms that both assess surface chromatic properties of objects as well as using such information in spatial contexts; whether there are different mechanisms for these two functions, or whether they are two aspects of a unitary mechanism is obscure. Relating psychophysical performance to retinal mechanisms will be complicated by physiological properties of different cortical areas. For example, recent functional magnetic resonance imaging (fMRI) measurements suggest the V4 complex may have a sluggish temporal response (Liu & Wandell, 2005), perhaps related to the filtering of high temporal frequency chromatic information mentioned above. On the other hand, higher temporal frequency signals in the dorsal, motion systems appear better preserved. Lastly, the spectral sensitivities of chromatic mechanisms in detection tasks map closely onto that expected of retinal mechanism, but are inconsistent with psychophysical unique hues. This has been termed ‘one of the central mysteries of colour science’ (Mollon & Jordan, 1997) and is a further indication that cortical processing may modify afferent input in unexpected ways.