COMPLEX RECEPTIVE FIELDS IN PRIMARY VISUAL CORTEX

Abstract

In the early sixties, Hubel and Wiesel reported the first physiological description of cells in cat primary visual cortex. Hubel and Wiesel distinguished two main cell types – simple cells and complex cells. Based on their distinct response properties, they suggested that the two cell types could represent two consecutive stages in receptive field construction. Since the sixties, new experimental and computational evidences provided serious alternatives to this hierarchical model. Parallel models put forward the idea that simple and complex receptive fields could both be built in parallel by direct geniculate inputs. More recently, recurrent models suggested that simple cells and complex cells may not be different cell types after all, and be just two functional states of the same cortical circuit. To this day, a consensus among hierarchical, parallel and recurrent models has been difficult to attain, however, the circuitry used by all models is becoming increasingly similar. Here, we review theoretical and experimental evidence for each line of models emphasizing their strengths and weaknesses. At the end we attempt to underline the difficulties that keep them apart and propose a possible common ground for all models to converge.

INTRODUCTION

In 1938, Hartline introduced the term “receptive field” to name a region of the retina where a change in light brightness modified the firing rate of a retinal ganglion cell (Hartline, 1938). Kuffler (1953) would later demonstrate that Hartline’s receptive field had a specific spatial structure known as center-surround and, in a series of studies to follow, Hubel and Wiesel (1959, 1961, 1962, 1965) would extend the term “receptive-field” to other cells in visual cortex and would demonstrate that receptive field structures become increasingly complex at successive stages of the visual pathway.

In comparison with the retina and the lateral geniculate nucleus, the primary visual cortex turned out to have a remarkable variety of receptive fields. Hubel and Wiesel classified cortical receptive fields into two main categories – simple cells and complex cells – after admitting that ‘new varieties [of receptive fields] are continually appearing, and it is unlikely that the ones we have listed give anything like a complete picture of the striate cortex (Hubel and Wiesel, 1962; pag. 109). Hubel-and-Wiesel simple receptive fields had a very characteristic spatial structure. Like cells in the lateral geniculate nucleus, they had separate on- and off-subregions that could be mapped with small spots of light. Unlike geniculate cells, the on- and off-subregions were elongated and parallel instead of circular and concentric (Fig. 1). Simple receptive fields were identified based on four criteria:

Figure 1.

Simple receptive fields (colored version of figure 2 from Hubel and Wiesel, 1962). Red crosses represent on-subregions and blue triangles off-subregions.

1. ‘they were subdivided into distinct excitatory and inhibitory regions’

2. ‘there was summation within the separate excitatory and inhibitory parts’

3. ‘there was antagonism between excitatory and inhibitory regions’

4. ‘it was possible to predict responses to stationary or moving spots of various shapes from a map of the excitatory and inhibitory areas’

In contrast, complex receptive fields formed a much more diverse population and were identified by exclusion. Figure 2 shows three different types of complex receptive fields as illustrated in Hubel and Wiesel (1962). Those three examples share very little in common. Cell A generates on-off responses throughout the entire receptive field. Cell B responds exclusively to a black horizontal bar. Cell C has partially separated on- and off-regions but, unlike a simple cell, its receptive field cannot be mapped with light spots.

Figure 2.

Three different types of complex receptive fields (modified version of figures 4, 6 and 7 of Hubel and Wiesel, 1962). Complex cells are a very heterogeneous population.. Cell A generates on-off responses throughout the entire receptive field. Cell B responds exclusively to a black horizontal bar. Cell C has partially separated on- and off-regions but the receptive field cannot be mapped with small spots of light. The green icons on the left represent the complex receptive field with the stimuli (flashed bars or borders) overlaid.

Hubel and Wiesel (1962) description did not provide a quantitative test to clearly distinguish between simple and complex receptive fields. The lack of such a test would later become a serious problem and eventually would lead to the proposal of many different classification criteria (Palmer and Rosenquist, 1974; Schiller et al., 1976; Henry, 1977, et al., 1983; Tanaka, 1983; Toyama et al., 1981a, 1981b; see Orban 1984 for review). Simple cells and complex cells have been classified based on the presence and degree of overlap of on- and off-subregions (Fig. 3A), spontaneous activity level, response amplitude, length summation, responses to patterns of random dots, responses to moving light and dark bars, responses to moving edges, responses to drifting or contrast-reversal gratings, responses to flashed bars and reverse correlation maps (Fig. 3B; see Skottun et al., 1991 and Mechler and Ringach, 2002, for review). To give and example of the confusion, cell B of figure 2 could probably be classified by different authors as complex (e.g. Hubel and Wiesel, 1962), simple (e.g. Skottun et al, 1991), S1 (e.g. Schiller et al., 1976; Martin and Whitteridge, 1984; Jones and Palmer, 1987; see Orban, 1984, for review) or Eoff (e.g. Tanaka, 1983). Similarly, cell C, could probably be classified either as complex (e.g. Hubel and Wiesel, 1962) or simple (e.g. Debanne et al., 1998).

Figure 3.

Simple and complex cells differ in their receptive-field structure and their responses to static and moving stimuli. A, Idealized receptive-field maps of a simple cell and a complex cell. B, Cartoon of the intracellular responses from a simple cell to small spots (light/dark) presented on each subregion (left and middle, the traces show a push-pull arrangement of excitatory and inhibitory inputs in the simple receptive field). Complex cells lack segregated on and off regions, small spots (light/dark) evoke on and off (push-push) responses throughout the entire receptive field (right). C, Cartoon representing the responses of a simple cell and a complex cell to drifting sinusoidal gratings.

A successful quantitative test to classify simple and complex cells was introduced by De Valois and colleagues (1982) and further refined by Skottun et al. (1991). The so called “response modulation” method provided a mixed-quantification of the 2^nd and 3^rd criteria of Hubel and Wiesel (1962) and rendered two cell populations that roughly corresponded to simple and complex cells (Fig. 3C). This new classification was based on the different response modulation of simple cells and complex cells to drifting sinusoidal gratings. While simple cells tend to modulate their firing rate in phase with the stimulus, complex cells elevate their firing with little or no modulation. Thus, the ratio between the amplitude of the first Fourier harmonic and the mean spike rate can be used as a quantitative index of ‘response non-linearity’ or ‘receptive field complexity’ (Fig. 3C). Unfortunately, this test is not free of limitations. The ‘response modulation’ does not provide any information on the receptive field geometry (e.g. number and shape of subregions) and therefore it cannot distinguish an X-geniculate cell from a simple cell even if thalamic and cortical cells are clearly different populations. Moreover, granted a non-linear relationship between synaptic current and firing rate, cells with identical synaptic inputs and intrinsic properties can still show a bimodal distribution of ‘response modulation’ (Mechler and Ringach, 2002).

The difficulty in classifying receptive fields is not simply a problem of semantics. It is closely related to fundamental questions such as: how simple and complex receptive fields are generated? And what is the role of simple and complex receptive fields in visual processing? Possible answers to the first question have been intensively investigated over the last decades by experimental and computational neuroscientists. In this article, we review the history of ideas and receptive field models that, for didactic purposes, we divide into three categories: Hierarchical, Parallel and Recurrent. Each of these models has made important contributions to our current understanding of how complex receptive fields are generated and, as recently noticed by Kevan Martin (2002), all models have become progressively similar. In this review we attempt to identify the barriers that keep them apart driving us away from consensus.

What makes a Complex Cell complex?

Before we review the different models it is important to further clarify what we mean by ‘complex cell’. While simple cells (as defined by Hubel and Wiesel, 1962) have relatively similar receptive field structures (Fig. 1), complex cells are very diverse and the term clearly embraces different populations (Fig. 2). Here, a simple cell is defined by the four Hubel-and-Wiesel criteria (see above) and a complex cell is defined by exclusion: any cortical neuron that does not have a simple receptive field.

In our view, a large body of evidence indicates that simple cells are a separate population from the rest of cortical cells in cat visual cortex.

1. Simple cells are segregated in specific cortical layers (e.g. the overwhelming majority of cells in layer 4 have simple receptive fields; Hubel and Wiesel, 1962; Gilbert, 1977; Gilbert and Wiesel, 1979; Hirsch et al., 1998, 2002; Martinez et al., 2002; but see Orban, 1984).

2. The overwhelming majority of spiny stellate cells in cat area 17 are simple cells (Kelly and Van Essen, 1974; Gilbert and Wiesel, 1979; Martin and Whitteridge, 1984; Hirsch et al., 1998, 2002; Martinez et al., 1999, 2002).

3. At least in layer 4, all simple cells have receptive-field structures consistent with a push-pull organization (e.g. within each subregion, stimuli of the opposite contrast evoke synaptic responses of the opposite sign; Ferster, 1988; Hirsch et al., 1998, 2002; Martinez et al., 1999;2002)

4. All simple cells receive direct geniculate input, in contrast, only a minority of complex cells does (most complex cells in the superficial layers and layer 5 do not receive direct geniculate input [e.g Ferster and Lindstrom, 1983; Martin and Whitteridge, 1984; Alonso and Martinez, 1998]).

5. The majority of simple cells generate responses that are roughly linear (Hubel and Wiesel, 1962; Movshon et al., 1978b; Skottun et al., 1991; Ferster, 1994; Carandini et al., 1997; Lampl et al., 2001).

6. Mechler and Ringach (2002) have recently suggested that it is necessary to show a bimodal distribution of cortical response properties to prove that simple cells are a separate class from complex cells. Although their argument is valid, statistical tools cannot be taken as the only possible source of proof. Neuroscientists do not discuss whether pyramidal cells and spiny stellate cells are different classes and, to our knowledge, nobody has shown that soma shapes are bimodally distributed.

It should be also emphasized that most of the evidence cited above comes from studies in cat visual cortex (using Hubel and Wiesel criteria for classification). Over the last two decades, we have accumulated enormous knowledge on the anatomy and physiology of cat area 17 while essential data in primate are still missing. Below there are a few examples of unknowns in primate:

1. There are very few studies that measured the receptive field structure of cells that receive direct geniculate input (as identified by electrical stimulation or cross-correlation analysis, Bullier and Henry, 1980).

2. We have almost no data on intracellular physiology in vivo (Anderson et al., 1993).

3. There are very few studies correlating neuronal morphology and receptive field properties (McGuire et al., 1991; Anderson et al., 1993).

4. There is still no agreement on whether there is a laminar segregation of simple cells and complex cells in layer 4C [e.g. layer 4C has mostly simple cells and un-oriented cells (Hubel and Wiesel, 1968; Bullier and Henry, 1980); layer 4C has simple cells and complex cells in similar proportions (Ringach et al., 2002)].

Consistent with this reasoning, most of the work revised here is based on studies of cat visual cortex or computational models based on the physiology of the cat visual cortex.

COMPLEX RECEPTIVE-FIELD MODELS

Hierarchical models

The hierarchical model was inspired by a systematic study and comparison of the receptive field properties in the lateral geniculate nucleus (LGN) and primary visual cortex (Hubel and Wiesel, 1959, 1961, 1962, 1965). According to this model, simple receptive fields are constructed from the convergence of geniculate inputs with receptive fields aligned in visual space. In turn, complex receptive fields originate from the convergence of simple cells with similar orientation preferences (Fig. 4A).

Figure 4.

Hierarchical models put forth the idea that simple cells and complex cells represent two successive stages of cortical processing. A, Simple receptive fields are generated in layer 4 from the convergent input of geniculate neurons with receptive fields aligned in visual space. Complex receptive fields are, in turn, generated in the superficial layers by pooling the input from layer 4 simple cells with similar orientation preferences. B, Newer versions of the model incorporate more elaborated intracortical circuitry. E and I stand for excitatory and inhibitory neurons respectively. θ, orientation; k, spatial frequency, φ, spatial phase. Full details are given in the text.

Hubel and Wiesel’s proposal was summarized in a drawing instead of a computational model. Consequently, many of the circuitry details were not precisely described. In their drawing, the possibility that both excitatory and inhibitory inputs could be involved in the construction of cortical receptive fields was left open by writing ‘the suppression of firing … is presumed to be the result of withdrawal of tonic excitation … one should, however, consider the possibility of direct inhibitory connections … Up to the present the two mechanisms have not been distinguished, but there is no reason to think that both do not occur’ (Hubel and Wiesel, 1962, pages 142–143). Moreover, it was emphasized that ‘Proposals such as those of Text-figs. 19 and 20 [model drawing] are obviously tentative and should not be interpreted literally’ (Hubel and Wiesel, 1962, page 144). Accordingly, the model has evolved in the past few years to account for new experimental discoveries that, mostly, emphasize the role played by local inhibitory inputs in shaping functional response properties in the cortex (Sillito, 1975, 1977, 1979; Pei et al., 1994; Allison et al, 1995; Crook et al., 1998; Borg-Graham et al., 1998; Hirsch et al., 1998; Murthy and Humphrey, 1999; Anderson et al., 2000; Martinez et al., 2002; see Ferster and Miller, 2000, for review)

It is important to make a clear distinction between Conceptual models (general hypothesis) and Computational models (computer simulations) since they greatly differ in the specifics and the scope of their predictions. In this review we make this distinction to avoid unnecessary confusion that could make models seem more different than they really are.

Conceptual model

The idea of the hierarchical model was proposed four decades ago but is far from being outdated (Kandel, Schwartz and Jessell, 2000). New findings over the years have kept the model alive and strong. First, as predicted by Hubel and Wiesel (1962), simple cells were found to receive direct geniculate inputs (Bullier and Henry, 1979; Mustari et al., 1982; Ferster and Lindstrom, 1983; Tanaka, 1983; Martin and Whitteridge, 1984; Reid and Alonso, 1995; Alonso and Martinez, 1998) and be overwhelmingly present in cortical layer 4 (Kelly and Van Essen, 1974; Gilbert and Wiesel., 1979; Martin and Whitteridge, 1984; Hirsch et al., 1998, 2002; Martinez et al., 1999, 2002). Second, in vivo recordings from neurons that were intracellularly labeled demonstrated that most layer 4 simple cells project to the superficial layers of the cortex (Gilbert and Wiesel., 1979; Martin and Whitteridge, 1984; Hirsch et al., 1998; Martinez et al., 1999, 2002) where most cells are complex (but see Orban, 1984).

The combination of morphological and physiological data provided strong support for the hierarchical model, however, some of the very same authors that provided the data remained skeptical. For example, Martin and Whitteridge wrote in their classical 1984 paper ‘Thus, although we agree that the superficial layer cells may use as their substrate the layer 4 cell receptive fields, this does not necessarily imply an S-to-C progression.’ [e.g. simple cells in layer 4 could target simple cells in the superficial layers].

This skepticism was based on several findings that seriously challenged the hierarchical model. First, some complex cells were found to receive direct geniculate input (Hoffmann and Stone, 1971; Bullier and Henry, 1979; Ferster and Lindstrom, 1983; Tanaka, 1983; Martin and Whitteridge, 1984) indicating that complex cells were ‘not so different’ from simple cells after all (see Parallel models below). Second, several studies failed to find evidence for excitatory connections from simple cells to complex cells (Toyama et al., 1981a, 1981b; Ghose et al., 1994; Freeman, 1996). Finally, by making injections of Cobalt Chloride in the lateral geniculate nucleus, Malpeli and colleagues concluded that it was possible to inactivate layer 4 simple cells without affecting the response of complex cells in the superficial layers (Malpeli, 1983; Malpeli et al., 1986; Mignard and Malpeli, 1991).

Some of the criticisms to the hierarchical model have been addressed over the years. By recording from vertically aligned layer 4 and layers 2+3 cells within the cortex, evidence for excitatory connections from simple cells to complex cells was found (Alonso and Martinez, 1998). Further, by making injections of GABA in the lateral geniculate nucleus, the complex cells described by Malpeli were found to be a minority; most layers 2+3 complex cells are not visually driven when the thalamic input to layer 4 is inactivated (Martinez and Alonso, 2001; Callaway, 2001). Finally, additional support for the hierarchical model came from another discipline that would become increasingly important over the years: Computational neuroscience.

Computational models

Most theoretical approaches to complex receptive-field generation were open up by the elegant “two-bar interaction” experiments of Tony Movshon and colleagues (1978a). Movshon et al. (1978a) argued that if complex cells respond to stationary or moving forms in a way that can not be predicted from their first-order responses (e.g. responses to single bars or small spots) must be because complex-cell responses depend on nonlinear interactions between at least two positions in space and time. Movshon and colleagues showed that when sets of two-bars were applied to complex cells their responses displayed on and off linear subunits resembling simple-cell subregions (see also Baker and Cynader, 1986; Gaska et al., 1994; Szulborski and Palmer, 1990; Heggelund, 1981). Emerson et al. (1992) extended this finding by demonstrating that the subunits of directionally selective complex-cells were also directional selective. Similarly, Ohzawa and Freeman demonstrated that the subunits of binocular complex cells were also binocular and had the same optimal disparity than the complex cell (Ohzawa and Freeman, 1986; see also Ohzawa et al., 1990, 1997; Anzai et al., 1999).

Based on these findings, many authors modeled complex cell’s responses as a square sum of simple cells with similar orientation and spatial frequency but with phases that differed in 90 degrees (e.g. Emerson et al., 1992; Fleet et al., 1995; Ohzawa et al., 1990, 1997; Pollen et al. 1989; Qian and Zhu, 1997; Sakai and Tanaka, 2000; Okajima and Imaoka, 2001; Shams and Malsburg, 2002). Mathematically, this can be described as a square sum of two linear operators, each characterized by a Gabor function of the same frequency but with phases 90 degrees apart from each other. Since a linear filter followed by a squaring device and then an integrator is considered an energy detector (Green and Swets, 1996) these models are collectively known as energy models (Adelson and Bergen, 1985). Recently, Okakima and Imaoka (2001) demonstrated that energy models render complex cells that are optimally designed from an information-theory point of view. However, as an interesting alternative, Lampl et al (2001) suggested that the pooling of simple cell inputs, rather than relaying on a linear summation, may be better described by a MAX operation (Riesenhuber and Poggio, 1999, 2002) where the strongest input determines the response of the complex cell. Linear or non-linear, both proposals rest on the assumption that simple and complex receptive fields represent two successive stages of cortical processing. A cartoon version of the core circuitry predicted by energy models is shown in figure 4B.

Although energy models seem to be good news for the hierarchical hypothesis, a key question remains: how is this precise synaptic connectivity achieved during development? Recently, it has been shown that a two-layer network fed with natural images can learn the phase-invariance characteristic of complex cells (Einhauser et al., 2002). The model uses layers of neurons that learn their properties from competitive Hebbian algorithms (based on the relative timing of pre- and postsynaptic spikes). Intexrestingly, neurons in the first layer learn to respond like simple cells and then, simple cell responses are used by neurons in the second layer to learn to respond as complex cells. Similar approaches have been used in the past to model the emergence of ocular dominance and orientation columns (e.g. Miller, 1996; Miller et al., 1999). Hyvarinen and Hoyer (2001) also used natural images to make neurons learn to respond as complex-cells in a network organization that is very similar to cat area 17 (cells clustered according to retinotopy, spatial frequency and orientation but independently of spatial phase; see DeAngelis et al., 1999). Again, in this model simple cell responses emerge in the first layer (see also Olshausen and Field, 1996) and complex cells in the second layer.

The hierarchical model has been very successful in great part because of its appealing simplicity. The essential idea that simple cells and complex cells represent two stages in receptive field construction is likely to remain correct even if newer models incorporate increasingly more realistic circuits (see below). The hierarchical hypothesis has also encountered criticisms that could not address. One of these criticisms is based on an experimental finding that is now unanimously accepted: some complex cells do receive direct geniculate input. This criticism makes it necessary for the hierarchical model to embrace other ideas and evolve.

Parallel models

Conceptual model

Historically, the first strong evidence against the hierarchical model was the discovery that some complex cells, like simple cells, receive monosynaptic input from the thalamus (Hoffman and Stone, 1971). Based on this discovery, Hoffman and Stone proposed that both cell types, simple and complex, were generated in parallel by separate thalamocortical pathways (Hoffman and Stone, 1971; Stone et al., 1979; see figure 5A). But how complex cells could become non-linear without pooling simple cell inputs? It was already known that there were linear and nonlinear cells in the retina and the LGN (X- and Y-cells, respectively [Shapley and Hochstein, 1975]) and that the two cell types formed separate parallel channels: X-cells in the retina projected to X-cells in LGN and Y-retinal-cells projected to Y-geniculate-cells (see Stone et al., 1979, for review). Hoffman and Stone used this evidence to propose that linear simple cells were built from the convergence of linear X-cell inputs and nonlinear complex cells from the convergence of nonlinear Y-cell inputs.

Figure 5.

A, Parallel models advocate that simple cells and complex cells are both created in parallel from different thalamocortical pathways. B, The most recent versions of parallel models come in two sorts that exploit the preponderance of the single cells over the circuit (Mel et al’s) or vice versa (Tao et al’s). Mel et al’s proposal generates complex-cell responses that are orientation selective through specific intradendritic computations. Tao et al’s scheme obtains simple cells and complex cells by modulating the gain of the thalamocortical and corticocortical inputs. The line thickness represents the strength of the connection. E and I stand for excitatory and inhibitory neurons respectively. See the text for details.

A striking experimental result soon came in support of the new parallel model: some complex cells seemed to respond to visual stimuli that were not effective in driving simple cells (Hammond and Mackay, 1975, 1977). This result was reinforced by a series of remarkable studies done by Malpeli and colleagues (Malpeli, 1983; Malpeli et al., 1986; Mignard and Malpeli, 1991). Malpeli et al. found that injections of Cobalt Chloride in layer A of LGN inactivated layer 4 simple cells but not layer 2+3 complex cells (Malpeli, 1983); responses at layers 2+3 cells were only affected when the inactivation of LGN (layer A) was combined with large lesions in area 18 (Mignard and Malpeli, 1991).

The results of Malpeli and colleagues fitted very nicely with the idea that simple and complex receptive fields were built in parallel and independently. However, parallel models soon found serious criticisms also. First, the idea of two cleanly segregated parallel channels (X → simple cell, Y → complex cell) was discarded by a large number of studies (Singer et al., 1975; Bullier and Henry, 1979; Tanaka, 1983; Ferster and Lindstrom, 1983; Martin and Whitteridge, 1984). Second, some authors showed that the impact of the entire Y-pathway in cat area-17 was weak (Spitzer and Hochstein, 1987; Ferster, 1990a, 1990b; Burke et al., 1992). Third, Skottun and colleagues (1988, 1991) presented evidence against Hammond-and-Mackay findings showing that most simple cells do respond to the stimuli that drive complex cells (but see Hammond, 1991). Finally, with the availability of new techniques, Malpeli’s results were found to be correct only for a minority of cells in the superficial layers (Martinez and Alonso, 2001; Callaway, 2001) –most layers 2+3 complex cells cannot be driven from within the classical receptive field when the main thalamic input to layer 4 is inactivated (see also Rivadulla and Sur, 2000).

Regardless of the specifics, the main idea of parallel models remained valid: at least some complex receptive fields could be constructed from direct geniculate inputs. However, an important question was still without answer. How can a complex cell be orientation selective if its main inputs are non-oriented and its receptive field does not have elongated subregions? Once again computational neuroscience came to the rescue (Mel et al., 1998).

Computational models

Mel and colleagues (1998) clearly showed that a cortical neuron could receive input from geniculate cells with overlapping on- and off-centers and still produce phase-invariant orientation tuning. In this model, the arrangement of geniculate inputs, originated by a Hebbian developmental rule, produces functional subunits scattered across the dendrites. These functional subunits respond more efficiently to specific line orientations by activating local voltage-dependent excitatory currents (see also Spitzer and Hochstein, 1985, 1988; see figure 5B, left).

In a similar line of thinking, a more recent parallel model combined thalamic and cortical inputs to generate linear and non-linear visual responses (Tao et al., 2001; Tao et al., 2002; see also Wielaard et al., 2001). In Tao et al.’s model, non-linear responses originate in cells that receive weak geniculate input and linear responses in cells that receive strong geniculate input (figure 5B, right). In agreement with Tao et al.’s model, complex cells (that generate non-linear responses) are abundant in cortical layers that receive the weakest thalamic input and the converse is true for simple cells that generate linear responses. Also consistent with this model, a large number of complex cells in cat primary visual cortex do not receive measurable direct geniculate input (e.g. Ferster and Lindstrom, 1983; Martin and Whitteridge, 1984; Alonso and Martinez, 1998).

Simple cells and complex cells are far from being two parallel cortical pathways in the same way as X and Y cells are parallel thalamic pathways. However, the idea that some complex receptive fields can be generated at least in part by direct thalamic inputs is likely to be correct. In support of this idea, recent computational models have provided plausible mechanisms to generate complex receptive fields directly from geniculate inputs (Spitzer and Hochstein, 1988; Mel et al., 1998; Tao et al., 2001). Some of these models (Tao et al., 2001; Tao et al., 2002) also emphasize another possible key player in the construction of complex receptive fields: the intracortical circuitry.

Recurrent Models

Conceptual model

Perhaps the most widely used argument against any thalamocortical model (hierarchical or parallel) is the nature of the cortical circuit itself. The number of geniculate synapses is only a small fraction of the total excitatory synapses made onto cortical cells (LeVay and Gilbert, 1976; Kisvarday et al., 1986; Peters and Payne, 1993; Ahmed et al., 1994). Therefore, cortical responses should be determined mostly by cortical inputs and not by thalamic inputs (Martin, 2002).

Douglas and Martin (Douglas et al., 1989, Douglas and Martin, 1991; see also Douglas et al., 1995) developed the conceptual frame for a new type of models that are known as recurrent models (figure 6A). Their proposal rests on three key assumptions:

Figure 6.

Recurrent models changed the focus of attention from single cells to networks of cortical connections. A, Most models are inspired by Douglas and Martin’s canonical microcircuit that emphasized the predominance of the input from intracortical, connections over the thalamocortical input (bottom). The most recent addenda to the family of recurrent models are the so-called state-dependent models. State-dependent models propose that simple and complex cells originate from the same cortical circuit operating at different amplification gains (top). B, Debanne et al’s model (left), emphasizes the balance between excitatory and inhibitory inputs as the origin of different cortical receptive fields. Chance et al’s model (right), emphasizes the strength of local excitatory connections within the superficial layers of the cortex. The line thickness represents the strength of the connection. E and I stand for excitatory and inhibitory neurons respectively. A more detailed explanation is given in the text.

1. Thalamic excitatory inputs are weak because geniculate synapses account for less than 10% of the total excitatory synapses onto layer 4 cells (Kisvarday et al., 1986; Peters and Payne, 1993; Ahmed et al., 1994).

2. Cortical excitation from neighboring neurons is very strong and serves to amplify the weak thalamic input.

3. Weak cortical inhibition controls the gain of the ‘cortical amplifier’ preventing runaway excitation and adjusting the network performance to changes in visual stimulation.

In initial versions of recurrent models, the predominance of intracortical excitation over intracortical inhibition was emphasized because 80% of the synapses on cortical cells are excitatory (Braitenberg and Schuz, 1991) and because intracellular measurements failed to find evidence for strong inhibition in response to visual stimuli (Berman et al., 1991; Douglas et al., 1991; Ferster, 1988). However, our view of cortical inhibition has changed over the years (Sillito, 1975, 1977, 1979; Pei et al., 1994; Allison et al, 1995; Crook et al., 1998; Borg-Graham et al., 1998; Hirsch et al., 1998; Murthy and Humphrey, 1999; Anderson et al., 2000; Martinez et al., 2002) and, accordingly, new recurrent models are incorporating stronger inhibitory inputs (Somers et al., 1995; Ben-Yishai et al., 1995; reviewed in Sompolinski and Shapley, 1997).

Recurrent models claim to be more faithful to the structure of the cortical network than hierarchical models and, to some extent, they are. Cortical neurons (simple or complex) receive abundant input from other cortical neurons both inhibitory and excitatory (e.g. Ahmed et al., 1994; Callaway, 1998; Fitzpatrick, 1996; Braitenberg and Schuz, 1998). Therefore, any model that incorporates more elaborate intracortical circuitry is in some way closer to the reality than models that use primarily one type of input. That being said, the idea that geniculate inputs are much weaker than cortical inputs is unlikely to be correct. While the number of geniculate synapses is a small percentage of the total excitatory synapses made on cortical cells (5% to 25% depending on which study is cited; e.g. Peters and Payne, 1993; Levay and Gilbert, 1976), geniculocortical connections have many other features that make them strong.

1. Geniculate synapses are bigger than cortical synapses (e.g. Ahmed et al., 1994).

2. Geniculate synapses are located proximally in the dendrites while excitatory cortical synapses tend to be located more distally (e.g. Ahmed et al., 1994).

3. Geniculate synapses have more release sites than cortical synapses (Gil et al., 1999).

4. Thalamocortical EPSPs are likely to be larger than cortico-cortical EPSPs (Gil et al., 1999; Stratford et al., 1996).

5. Geniculate cells have generally higher firing rates than cortical cells (e.g. Bullier et al., 1982).

6. Many of the geniculate cells that converge on the same cortical cell generate precise and strong synchronous firing (Alonso et al., 1996; such precise and strong synchrony is not found within the cortex)

7. The inactivation of a tiny region of LGN is enough to silence the activity of most cortical cells in layers 4 and 2+3 (Martinez and Alonso, 2001; see also Malpeli, 1983). The inactivation of a similarly tiny region in the cortex has only a subtle effect on the response of a cortical cell (Bolz and Gilbert, 1986; Grieve and Sillito, 1995).

In general, it is risky to estimate the strength of a given pathway based solely on the number of synaptic contacts. Indeed, if this type of assumption were correct the geniculate receptive fields should resemble more cortical receptive fields than retinal receptive fields and this is clearly not the case (Hubel and Wiesel, 1961; Cleland et al., 1971; Usrey et al., 1998; geniculate cells receive 7% of excitatory synapses from retina and about 40% from cortex; Van Horn et al., 2000).

While the relative contribution of thalamic and cortical inputs to cortical receptive field generation is still matter of debate (Ferster and Miller, 2000; Martin, 2002), recent models have chosen to focus on a different question: how recurrent networks generate simple and complex receptive fields? This focus is the seed of a new bold idea: simple cells and complex cells may not be different cell types after all (e.g. Mechler and Ringach, 2002; Abbott and Chance, 2002). Supporting this proposal, the work of Debanne et al. (1998) showed that the relative strength of cortical on- and off-responses could be modified by pairing visual stimuli with current injections. In theory, if we can modify the strength of on- and off-responses we should also be able to transform a simple cell into a complex cell and vice versa. In reality, changes in the strength of on- and off-responses are rare in the adult (4 cases in Debanne et al., 1998) and they are usually subtle. More pronounced changes are generated when the precise balance between cortical excitation and inhibition is manipulated (Sillito, 1975; Nelson et al., 1994; Rivadulla et al., 2001). However, these changes are not inconsistent with a hierarchical model. In a hierarchical model, a stimulus presented in an on-subregion of a simple cell should generate on-responses because it activates mostly the receptive field centers from on-center geniculate cells. In the absence of cortical inhibition, the same stimulus should also activate off-responses that originate from two main different sources:

1. The off-surrounds from on-center geniculate cells that overlap the on-subregion.

2. The off-center borders from off-center geniculate cells that overlap the adjacent off-subregion (see Alonso et al., 2001 for multiple examples).

The fact that on-off responses are observed in simple cells after blocking inhibition is exactly what we would expect from a hierarchical model and the properties of thalamocortical connections (Reid and Alonso, 1995; Alonso et al., 2001). Also as expected from a hierarchical model, when on- and off-responses are quantitatively mapped and averaged, simple receptive fields still look ‘very simple’ even if cortical inhibition is blocked (Murthy and Humphrey, 1999; see also Rivadulla et al., 2001).

The idea that simple cells and complex cells are the same cell type (or generated by the same basic circuit) seems hard to accept based on a large body of literature in cat visual cortex (see introduction of this review). Recent anatomical and physiological data also suggests that thalamic inputs are not weaker than cortical inputs (see above). In spite of these criticisms, the ‘intracortical emphasis’ that emerged from recurrent models is important and is leading to powerful computational models.

Computational models

Recurrent models moved the focus of attention from thalamo-cortical connections to networks of cortical neurons that are reciprocally connected (Martin, 2002; Nelson, 2002). With differences regarding mainly the role of inhibitory inputs, recurrent models used networks of cortical neurons to explain the emergence of orientation and direction selectivity in the cortex (Somers et al., 1995; Ben-Yishai et al., 1995; reviewed in Sompolinski and Shapley, 1997). Somers et al. (1995) developed a recurrent model that was especially successful at explaining a large body of experimental data. In this model, cortical excitation links cells with similar orientation preferences while cortical inhibition links cells with a broader range of orientations. The model was challenged by recent data demonstrating that the tuning for excitatory and inhibitory inputs in layers 4 and 2+3 is similar (Nelson et al., 1994; Anderson et al., 2000; Martinez et al., 2002). Also, against the model predictions, local excitatory connections have been found to extend farther laterally than inhibitory connections (Roerig and Chen, 2001; Yousef et al., 1999, 2001; Buzas et al., 2001; see Carandini and Ringach, 1997 for other criticisms). Another recurrent model was proposed by Ben Yishai et al. (1995) to generate contrast invariant orientation tuning. In this model feedback connections are so strong that the cortical network becomes an orientation attractor (i.e., the output of the network is determined simply by the balance between excitation and inhibition independently of the properties of the input). This type of orientation attractors (see also Tsodyks et al., 1999) have been challenged by data showing that cortical responses to oriented gratings do depend on the spatial frequency of the input (Vidyasagar and Siguenza, 1985; Webster and De Valois, 1985; Jones et al., 1987; Hammond and Pomfrett, 1990).

The models of Somers et al. (1995) and Ben-Yishai et al. (1995) did not address specifically the generation of complex receptive fields, however, they were very influential in setting the basis for future recurrent models that did address the issue. Debanne et al. (1998) used recurrent circuits to generate simple and complex cells. In their model, both cell types receive input from geniculate cells with overlapping on- and off-receptive-field centers (and other cortical cells). Simple cells are generated when the cortical inhibitory inputs are strong enough to impose a bias towards either on or off responses. Complex cells are generated when inhibition is reduced to unmask on-off responses. Debanne et al.’s model explains why simple cells generate on-off responses when cortical inhibition is pharmacologically blocked (Sillito, 1975; Ramoa et al., 1988; Nelson et al., 1994; Shulz et al., 1993). However, as discussed earlier, these findings can also be explained with a hierarchical model (see above page 20). Moreover, some of the details of Debanne et al.’s model are at odds with experimental data. For example, not all smooth inhibitory neurons in layer 4 have S1 receptive fields (e.g. Gilbert and Wiesel, 1979; Martin et al., 1983; Kisvarday et al., 1985, 1987; Azouz et al., 1997; Hirsch et al., 2000) and the model fails to explain why many layer 4 cells have receptive fields with push-pull organization (Ferster, 1988; Hirsch et al, 1998; Martinez et al., 1999; Anderson et al., 2000).

While Debanne et al.’s model (1998) made a bridge between parallel and recurrent models (Tao et al., 2001, 2002 is also in this category), the model of Chance et al. (1998) abandoned this bridge to adopt elements from a hierarchical organization (Chance et al., 1998). As in a hierarchical model, Chance et al. uses a first layer of simple cells (equivalent to layer 4) that feeds into another layer of cells with similar orientation preferences (equivalent to layers 2+3). Chance et al.’s model departs from a hierarchical organization by making the connections from layer 4 to layers 2+3 weaker than the connections within layers 2+3 and by making the vertical connections link cells with similar spatial phase while strong recurrent connections link cells with different spatial phase (figure 6B, right). While previous recurrent models used cortical amplification to generate orientation tuning and direction selectivity (Douglas et al., 1995; Ben-Yishai et al., 1995; Somers et al., 1995), Chance et al. used cortical amplification to generate nonlinear responses. In support of Chance et al.’s model, it has been suggested that simple cells and complex cells cannot be separated into two different populations (Mechler and Ringach, 2002; Abbott and Chance, 2002) and that complex cells can behave like simple cells when the balance between excitation and inhibition changes (Sillito, 1975; Nelson, 1994; Shulz et al., 1993; Rivadulla et al. 2001). We have already criticized these two suggestions in detail (see above page 20). Here, we could also add that the specificity of vertical connections (layer 4 → layers 2+3) for spatial phase is difficult to reconcile with the lack of clustering for this property (DeAngelis et al., 1999) and with the finding that some layer 4 simple cells with different spatial-phases do converge on the same superficial complex cells (Alonso and Martinez, unpublished observations). Furthermore, it has been recently shown that the synaptic connections between layer-4 and layers-2+3 are among the strongest connections in the cortical network (stronger than local horizontal connections between cells in the superficial layers; Feldmeyer et al., 2002).

As with any model, the circuitry predicted by recurrent models will have to pass the test of time and experimental data. In any case, the idea that response linearity could be modulated by the gain of the cortical network is interesting and may prove to be correct in some extent. New experimental data will tell us whether this modulation exists, how strong it is and which cortical layers involves (see Rivadulla et al., 2001).

SEARCHING FOR A CONSENSUS

Of all the criticisms that the hierarchical model has received over the years two have remained sufficiently strong as to promote the birth of new conceptual frameworks (figure 7). First, as suggested by parallel models, some complex receptive fields can be generated at least in part from direct geniculate inputs. Second, as suggested by recurrent models, cortical cells are heavily interconnected and their responses are likely to be modulated by recurrent connections. In spite of these criticisms, the idea that simple cells and complex cells represent two stages in receptive field construction is likely to remain correct even if models evolve and become more precise. Eventually, all models will mix to adopt elements from each other and finally converge into one. For example, Troyer et al (1998) introduced strong inhibitory inputs and more elaborate local connectivity in their hierarchical model of orientation selectivity in cat layer 4 (see also Miller, 2001 for review). Chance et al. (1998) adopted a hierarchical organization by using a first layer of simple cells that feeds into a second layer of cells with more elaborated responses (simple or complex depending on network gain). Also, Martin (2002) has recently described his recurrent model as an ‘alternative hierarchy’ [the word hierarchy was not used in the original description of recurrent or canonical microcircuits (e.g. Douglas and Martin, 1991)]. Finally, Tao et al. (2001, 2002) used strong/weak geniculate inputs to model simple cells/complex cells, making simple cells ‘closer’ to their thalamic inputs than complex cells (in previous parallel models complex cells received strong geniculate inputs, e.g. Mel et al. 1998).

Figure 7.

Hubel-and-Wiesel’s hierarchical model (left) has been extensively challenged over the years. Two main criticisms (middle) have propelled the emergence of alternative frameworks that accounted for the new experimental findings. Here, we propose a new circuit diagram for cat primary visual cortex (right) that embraces all current ideas of hierarchical, parallel and recurrent models. Layer 4 simple-cells and superficial complex-cells form two successive stages in receptive field construction. In addition, some cells in layer 4 have complex receptive fields derived, in parallel, from direct geniculate inputs. At each layer, local intracortical circuits (both excitatory and inhibitory) modulate the gain of the network. See the text for a detailed explanation.

It seems clear that each line of models –hierarchical, parallel or recurrent— can no longer be considered in isolation. Here we propose a circuit diagram to integrate the main ideas/models discussed above (Fig. 7). The diagram cannot be easily ascribed to any of the main models although some could see it as clearly hierarchical, others as clearly parallel and still others as clearly recurrent. The diagram gives a general framework of the main connections as a function of cortical layers (layer 4, 2+3) and receptive field properties (simple/complex). Layer 4 simple-cells and layers 2+3 complex-cells are two stages in receptive field construction as proposed by the hierarchical model (Hubel and Wiesel, 1962; Adelson and Bergen, 1985) and some recurrent models (Chance et al., 1998). At the same time, a subpopulation of complex-cells has their receptive fields constructed from direct geniculate inputs (Spitzer and Hochstein, 1988; Mel et al., 1998; Tao et al., 2001, 2002). Finally, at each layer, local intracortical circuits (both excitatory and inhibitory) modulate the gain of the cortical network (Debanne et al., 1998; Chance et al., 1998; Tao et al., 2001, 2002) generating different results depending on the layer. Within layers 2+3, response linearity would be modulated in some extent by local connections as proposed by recurrent models; within layer 4 (simple cells with push-pull receptive fields) response linearity would not be so easily modulated. Forty years had gone by since Hubel and Wiesel proposed the hierarchical model, one wonders how many times will the pendulum swing back and forth before it finally sets at a standstill and we reach a consensus on how complex receptive fields are generated.