Role of Feedback Connections in Central Visual Processing

Abstract

The physiological response properties of neurons in the visual system are inherited mainly from feedforward inputs. Interestingly, feedback inputs often outnumber feedforward inputs. Although they are numerous, feedback connections are weaker, slower, and considered to be modulatory, in contrast to fast, high-efficacy feedforward connections. Accordingly, the functional role of feedback in visual processing has remained a fundamental mystery in vision science. At the core of this mystery are questions about whether feedback circuits regulate spatial receptive field properties versus temporal responses among target neurons, or whether feedback serves a more global role in arousal or attention. These proposed functions are not mutually exclusive, and there is compelling evidence to support multiple functional roles for feedback. In this review, the role of feedback in vision will be explored mainly from the perspective of corticothalamic feedback. Further generalized principles of feedback applicable to corticocortical connections will also be considered.

1. INTRODUCTION

Humans rely heavily on our sense of sight to navigate the physical and social environment. Our fascination with our visual experience is represented throughout human history, beginning with paintings in dwellings and vivid descriptions of visual imagery passed on through oral tradition. Perhaps unsurprisingly, a major focus of clinical and systems neuroscience research is on understanding and restoring visual perception. Accordingly, more is known about the structure and function of visual brain areas than those of any other portion of the mammalian brain. However, our current knowledge is heavily biased toward feedforward connections, which constitute the well-characterized visual processing hierarchy spanning the retina, the visual thalamus, and myriad visual cortical areas. Far less is known about feedback connections, which are, in many cases, more numerous than their feedforward counterparts. The objective of this review is to highlight past and current work on the organization and function of feedback circuits in the visual system. Feedback must always be considered in the context of feedforward connections, so a summary of feedforward visual pathways and their emergent physiological properties is provided. Emphasis is then placed on the first feedback pathway in the visual system, made up of corticothalamic neurons connecting the primary visual cortex (V1) with the first-order visual sensory thalamus, the dorsal lateral geniculate nucleus (LGN). These corticogeniculate circuits have been studied extensively, and recent work provides important clues about their role in vision. Importantly, functions attributed to the first feedback neurons in the visual system may also apply to higher-order cortical feedback circuits. Thus, general principles that may govern the function of cortical feedback connections throughout the visual system are also considered.

Focus is placed on studies of early visual pathways in nonhuman primates and carnivores because these have proven to be the best animal model approximations for visual processing in humans. Other species, such as rodents, are also discussed because they provide an important comparison across species from different environmental niches and with different behavioral goals. Because most experimental data collected in animals with good visual acuity (e.g., primates and carnivores) come from visual brain regions corresponding to parafoveal eccentricities, this review discusses principles governing feedforward and feedback pathways in visual processing of the central visual field.

The objective of this review is to address the following characteristics and hypotheses regarding the function of cortical feedback in vision. (a) Based on the anatomy and organization of feedback connections, their functional role is proposed to be more modulatory than driving. Comparisons between feedforward retinal inputs and feedback corticogeniculate inputs onto LGN neurons have established the driver versus modulator classification as a guiding premise. Nonetheless, feedback inputs are numerous and retinotopically restricted in their extent, providing additional clues about functionality. (b) Physiological characterization of corticogeniculate neurons reveals multiple distinct cell types defined based on axon conduction speed and visual physiological responses. In highly visual mammals, corticogeniculate cell types match those of the established feedforward parallel processing streams. In less visual mammals, relationships between axon conduction speed and visual responses are also evident. These results suggest that feedback operates on different timescales to convey feature-specific information. (c) Causal manipulations of cortical feedback have been employed for decades to elucidate its functional role in vision. Most of these efforts have focused on the effects of feedback manipulations on spatial properties of target neurons; however, compelling evidence supports a role for cortical feedback in regulating the timing of neuronal responses. The role of cortical feedback in enhancing spatial and temporal precision of feedforward signals is explored below. (d) Some of the oldest proposed functions for cortical feedback, especially to the thalamus, involve modulating among sleep, alertness, and arousal. Additional extra-visual roles for feedback have been proposed, including communicating top-down signals about context, attention, or task-relevant goals. Whether and how cortical feedback circuits could multiplex visual and extra-visual signals remain significant and open questions.

2. ANATOMY AND PHYSIOLOGY OF FEEDFORWARD CONNECTIONS

The structure and organization of cortical feedback must be understood within the context of feedforward circuitry in the visual system. A hallmark of mammalian visual systems is parallel processing of discrete visual feature information (for reviews, see Briggs 2017, Kaplan 2004, Nassi & Callaway 2009, Niell 2015, Seabrook et al. 2017, Sherman & Guillery 2006). It should be noted that these streams are not strictly homologous across species in terms of their cellular components and physiology and the visual feature information that they convey. Parallel streams begin in the retina with distinct classes of bipolar cells that sample inputs directly from cone photoreceptors and provide the inputs to correspondingly distinct classes of retinal ganglion cells (RGCs), the only neurons in the retina with axons that leave the retina (for reviews, see Dacey et al. 2014, Field & Chichilnisky 2007).

In trichromatic primate species, there are three parallel visual processing streams, the parvocellular, magnocellular, and koniocellular streams, that separately convey information relevant for form, acuity, and red–green color vision; motion and low-contrast detection; and blue–yellow color vision, respectively (Figure 1). Midget RGCs have center–surround, red–green color-opponent receptive fields due to separate long-wavelength (L)- and middle-wavelength (M)-sensitive cone inputs to center and surround components of their receptive fields and project to the parvocellular layers of the LGN (Field & Chichilnisky 2007). Parasol RGCs also have center–surround receptive fields, but because they receive a mixture of L and M cone inputs to the center and surround, they are luminance (On–Off) rather than color opponent; parasol RGCs project to the magnocellular LGN layers (Field & Chichilnisky 2007). Midget and parasol RGCs make up the majority of primate RGCs (Field & Chichilnisky 2007). Small bistratified RGCs receive short-wavelength (S)-sensitive cone input opposed by L and M cone inputs and convey blue–yellow color-opponent signals to the koniocellular layers of the LGN (Dacey et al. 2014). LGN neuronal receptive fields mimic their RGC inputs: LGN neurons in the dorsal parvocellular layers have center–surround, red–green opponent receptive fields due to their midget RGC inputs; neurons in the ventral magnocellular layers have center–surround luminance On–Off receptive fields due to their parasol RGC inputs; and neurons in the intercalating koniocellular layers that receive small bistratified RGC input have blue–yellow opponent receptive fields (Derrington et al. 1984, Hendry & Reid 2000). Primate RGC and LGN neurons in the parvocellular, magnocellular, and koniocellular streams can be differentiated based on additional physiological response properties including contrast sensitivity, spatial and temporal frequency selectivity, and extraclassical surround suppression. Parvocellular neurons have linear contrast response curves and poorer contrast sensitivity, higher spatial frequency preferences, lower temporal frequency preferences, and less extraclassical surround suppression, whereas magnocellular neurons have saturating contrast response curves and higher contrast sensitivity, lower spatial frequency preferences, higher temporal frequency preferences, and stronger extraclassical surround suppression, and koniocellular neurons have intermediate properties and are likely to be physiologically heterogeneous (Alitto & Usrey 2008, Derrington & Lennie 1984, Eiber et al. 2018). The parvocellular, magnocellular, and koniocellular streams remain anatomically segregated from the retina, through the LGN, and to the geniculocortical afferent termination zones in V1. Parvocellular LGN neurons project to layer 4Cβ with collateral axons in layer 6; magnocellular LGN neurons project to layer 4Cα with collaterals in layer 6; and koniocellular LGN neurons project to the cytochrome oxidase–rich blobs in layer 2/3 and to layer 1 (Blasdel & Lund 1983, Freund et al. 1989, Hendrickson et al. 1978, Hendry & Yoshioka 1994, Hubel & Wiesel 1972). In V1, several more advanced visual physiological properties emerge through combination of LGN afferent inputs and/or local circuitry within V1. These emergent properties include orientation tuning, which first appears at the magnocellular geniculocortical synapses onto layer 4Cα Simple cells (Hubel & Wiesel 1968); direction selectivity (Gur et al. 2005, Hubel & Wiesel 1968); color selectivity (Conway & Livingstone 2006, Garg et al. 2019, Johnson et al. 2001); and binocular disparity (Cumming & Parker 1997, Hubel & Wiesel 1968). Feedforward corticocortical projection neurons in V1 reside mainly in the superficial layers (layers 2/3 and 4B) and project axons to the extrastriate visual cortical areas V2, V3, V4, middle temporal (MT), and V6 (Nassi & Callaway 2007, 2009; Sincich & Horton 2005; Yukie & Iwai 1985). Neurons in the deepest cortical layers in V1, layers 5 and 6, can also project to the extrastriate cortex, but the majority of projection neurons in the deep layers target subcortical structures, including the LGN and the visual portion of the pulvinar (Briggs 2017, Kaas & Lyon 2007). In general, corticocortical circuits in primates mostly connect neighboring visual cortical areas that are also proximal on the visual cortical hierarchy, although sparse connections are found across many visual cortical areas (Felleman & Van Essen 1991, Markov et al. 2014). As a general rule throughout the primate visual cortex, feedforward corticocortical connections originate in the superficial (and, to a lesser extent, deep) cortical layers and project axons to layer 4 of the target higher cortical area, while feedback corticocortical connections originate in the superficial and deep cortical layers and target superficial and deep cortical layers in the target lower area (Felleman & Van Essen 1991, Rockland & Virga 1989).

Figure 1.

Feedforward and feedback parallel pathways in the early visual system of primates. Retinal ganglion cells conveying visual information to the dorsal lateral geniculate nucleus (LGN) of the thalamus are separated into three major parallel processing streams: the parvocellular (red), magnocellular (black), and koniocellular (blue) streams. These retinal ganglion cells (filled colored circles at bottom) project axons to separated parvocellular (pink), magnocellular (gray), and intercalated koniocellular (blue) layers of the LGN. LGN relay neurons (filled colored circles in LGN) display distinctive receptive field properties driven by their retinal inputs: red–green center–surround organization among parvocellular LGN neurons, On–Off (black–white) center–surround organization among magnocellular LGN neurons, and blue–yellow opponent organization among koniocellular neurons. The geniculocortical axons of LGN relay neurons target segregated layers within the primary visual cortex (V1): Parvocellular geniculocortical axons (red thin line) mainly target layer 4Cβ (pink); magnocellular geniculocortical axons (black thin line) mainly target layer 4Cα (gray); and koniocellular geniculocortical axons (blue thin line) target layer 1, the blobs in layer 2/3, and layer 4A. Geniculocortical recipient neurons in these V1 layers also display distinctive receptive field properties driven by their geniculocortical inputs: Parvocellular-recipient V1 neurons are simple cells with separable red–green receptive field subregions, magnocellular-recipient V1 neurons are simple and complex cells with orientation tuning and direction selectivity, and koniocellular-recipient V1 neurons are responsive to blue color. Corticogeniculate neurons located in layer 6 of V1 provide feedback from V1 to the LGN. Corticogeniculate neurons are also separated into the same parallel streams, and their receptive field properties reflect their feedforward stream-specific inputs. There are parvocellular-projecting corticogeniculate neurons (red pyramidal cell in upper layer 6 of V1), magnocellular-projecting corticogeniculate neurons (black pyramidal cell in lower layer 6), and koniocellular-projecting corticogeniculate neurons (blue tilted cell at the bottom of layer 6). The distributions of corticogeniculate axon terminals in the LGN (red, black, and blue downward cones) are broader compared to the distributions of retinogeniculate axon terminals in the LGN (red, black, blue upward cones). Similarly, the distribution of terminals from corticocortical feedback axons (top black cone) are also more broadly distributed relative to feedforward and local axon terminal distributions within V1. Corticocortical feedback originates in extrastriate visual areas such as the secondary visual cortex (V2) and the middle temporal area (MT), in which neurons are responsive to more complex visual features including texture and stimulus motion.

Although it is not strict, there is some homology between the parvocellular, magnocellular, and koniocellular streams in primates and the X, Y, and W streams in carnivores. Carnivores are dichromats and therefore lack color-opponent neurons, but X and Y RGCs and LGN neurons have center–surround receptive fields that are On–Off opponent (Hubel 1960, Kuffler 1953). Additionally, the neuronal circuits that make up the X, Y, and W streams remain separated from the retina through the LGN and to the afferent termination zones in layers 4 (X and Y inputs) and 2/3 (W inputs) in V1 (area 17), even though X and Y neurons are mixed in the A LGN layers, and their geniculocortical afferent terminations overlap in layer 4; W neurons are restricted to the C LGN layers and project to layer 2/3 (Anderson et al. 2009, Bullier & Norton 1979, Cleland et al. 1971, Freund et al. 1985, Humphrey et al. 1985, Sur & Sherman 1982). X and Y RGC and LGN neurons were originally distinguished based on linear versus nonlinear spatial summation and sustained versus transient responses to stationary gratings, respectively (Cleland et al. 1971, Enroth-Cugell & Robson 1966, Hoffmann et al. 1972). Based on these physiological response properties, most parvocellular and magnocellular neurons in primates are more X-like (Derrington & Lennie 1984, Kaplan & Shapley 1982). However, X and Y LGN neurons can also be somewhat differentiated based on contrast sensitivity and spatial and temporal frequency selectivity: X neurons have linear contrast response curves and prefer higher spatial and lower temporal frequencies, while Y neurons have saturating contrast response curves and prefer lower spatial and higher temporal frequencies (Derrington & Fuchs 1979). Furthermore, transcription factors have been identified that are selectively expressed in parvocellular and X LGN neurons (FoxP2) or magnocellular and Y LGN neurons (PCP4) in macaques and ferrets (Iwai et al. 2013, Kawasaki et al. 2004). Similar to primates, center–surround geniculocortical inputs to V1 combine to produce orientation tuning in layer 4 Simple cells (Hubel & Wiesel 1962, Reid & Alonso 1995), and more advanced visual response properties emerge through local circuitry within V1 (Hirsch et al. 2002). Furthermore, a hierarchy of visual cortical areas exists in carnivores with patterns of feedforward and feedback corticocortical connectivity similar to those observed in the primate visual cortical hierarchy (Hilgetag & Grant 2010).

Parallel streams from the retina to the cortex have been identified in rodents and rabbits (Hei et al. 2014, Seabrooket al. 2017); however, the homology between these and the parallel visual processing streams of primates and carnivores is not clear. In mice and rabbits, most RGCs and LGN neurons display orientation and/or direction selectivity, response properties that are largely absent in carnivores and primates (Scholl et al. 2013). The circuits conveying orientation- and direction-selective inputs from the LGN to V1 in mice and rabbits are distinct from the geniculocortical afferent circuits in carnivores and primates (Hei et al. 2014, Scholl et al. 2013). Furthermore, in rodents, neurons in V1 and other identified visual cortical areas have multisensory responses (Niell & Stryker 2010), reflecting a high level of interconnectivity with multiple nonvisual cortical areas (Choi et al. 2018) that is quite different from the hierarchical visual cortical connectivity patterns observed in highly visual mammals.

3. ANATOMY OF FEEDBACK CONNECTIONS

Having established the organization of feedforward visual pathways and the visual physiological responses emerging through these connections, I turn next to the structural organization and morphology of feedback connections in the visual system. There is no visual feedback to the retina. Strictly speaking, the first feedback circuit in the visual system is the positive inhibitory loop between LGN relay neurons and inhibitory neurons in the thalamic reticular nucleus (TRN; perigeniculate nucleus in carnivores). Geniculocortical axons targeting V1 send a collateral branch to the TRN, and GABAergic TRN neurons then inhibit LGN relay neurons (Sherman & Guillery 2006). TRN neurons are retinotopically organized and have receptive fields similar to those of LGN neurons (Soto-Sanchez et al. 2017), although their burst spiking patterns have led some to propose that the TRN plays a role in regulating sleep–wake rhythms (Fuentealba & Steriade 2005). Because TRN neurons receive copies of LGN relay cell output and feedback from the visual cortex, and they project solely to LGN relay neurons, it may be more useful to think of TRN neurons as a local thalamic inhibitory neuron population, rather than a feedback circuit. With that caveat, the first true feedback circuit in the visual system is the corticogeniculate pathway. Additional feedback circuits in the visual system include corticocortical feedback connections and other corticothalamic connections targeting the superior colliculus, pulvinar, claustrum, and other areas, although the extent to which some of these corticothalamic connections are strictly feedback is open to debate (Usrey & Sherman 2018). Among all of these feedback connections, the most is known about corticogeniculate circuits, so these can serve as a model for exploring the structure and function of cortical feedback in the visual system.

Corticogeniculate neurons are a subset of corticothalamic neurons, specific to the feedback connection between the visual cortex and the LGN, that have been observed in all mammalian visual systems studied to date. Corticogeniculate neurons are mostly pyramidal neurons (some are spiny stellates), are always excitatory neurons that release glutamate, and always have cell bodies exclusively in layer 6 of the visual cortex (Briggs et al. 2016, Brumberg et al. 2003, Conley & Raczkowski 1990, Fitzpatrick et al. 1994, Gilbert & Kelly 1975, Hasse et al. 2019, Ichida et al. 2014, Jiang et al. 1993, Katz 1987, Kim et al. 2014, Lin & Kaas 1977, Lund et al. 1975, Swadlow & Weyand 1981, Usrey & Fitzpatrick 1996). Although the majority of corticogeniculate neurons originate in V1 (or area 17 in carnivores), a sparser population of corticogeniculate neurons have also been observed in V2 (area 18) of primates and carnivores (Briggs et al. 2016, Hasse et al. 2019, Lin & Kaas 1977). In primate V1, corticogeniculate neurons projecting to the parvocellular layers of the LGN have cell bodies in the upper third of layer 6, while those projecting to the ventral magnocellular and koniocellular layers of the LGN have cell bodies in the lower third of layer 6 (Fitzpatrick et al. 1994).

Corticogeniculate neurons can receive direct geniculocortical afferent input (Blasdel & Lund 1983, Briggs & Usrey 2007, Bullier & Henry 1980, Da Costa & Martin 2009, Ferster & Lindstrom 1983, Hendrickson et al. 1978), in addition to local inputs from cortical neurons (Briggs & Callaway 2001, Briggs et al. 2016, Hasse et al. 2019, Katz 1987, Kim et al. 2014, Lund & Boothe 1975, Wiser & Callaway 1996, Zarrinpar & Callaway 2006). There are species differences in the local circuitry of corticogeniculate neurons within the visual cortex. Corticogeniculate neurons in primates and carnivores are strongly and reciprocally connected with neurons in layer 4 but have dendrites and axons in the superficial layers and often receive superficial layer input (Briggs & Callaway 2001, Briggs et al. 2016, Katz 1987, Wiser & Callaway 1996). In contrast, corticogeniculate neurons in rodents lack processes in the superficial layers and receive minimal or no input from superficial layer neurons; instead, rodent corticogeniculate neurons are strongly connected to layer 5 and can also mediate broad inhibition across the cortical layers (Bortone et al. 2014, Kim et al. 2014, Olsen et al. 2012, Zarrinpar & Callaway 2006).

Most, if not all, corticogeniculate neurons have local axons that target local neurons within V1, as described above. However, the functional purpose of the local axon collaterals that convey a copy of the message back to the LGN is not known. More focus has been placed on corticogeniculate axons to the LGN. In both primates and carnivores, corticogeniculate axon terminals within the LGN are retinotopic, mainly restricted to single layers, and more spread relative to RGC axonal terminations but consistent with the size of V1 neuronal receptive fields (Angelucci & Sainsbury 2006, Claps & Casagrande 1990, Ichida & Casagrande 2002, Ichida et al. 2014, Murphy & Sillito 1996, Robson 1983). Aside from local axons within the cortex, the only known axon collateral of corticogeniculate neurons in primates and carnivores terminates in the TRN (Sherman & Guillery 2006). In rodents, corticogeniculate neurons can send collaterals to additional thalamic nuclei, including the lateral posterior nucleus (Olsen et al. 2012). Taken together, corticogeniculate dendritic morphology, cell body position in layer 6, axonal termination patterns, and local circuit inputs all indicate that there are distinct subclasses of primate and carnivore corticogeniculate neurons that target LGN neurons in separate parallel streams (for a review, see Hasse & Briggs 2017b). In this framework, tall pyramidal corticogeniculate neurons with dendrites in upper layer 4 receive biased magnocellular/Y-stream inputs and project to magnocellular/Y neurons in the LGN, shorter pyramidal corticogeniculate neurons with dendrites in lower layer 4 receive biased parvocellular/X-stream inputs and project to parvocellular/X LGN neurons, and tilted corticogeniculate neurons project to koniocellular/W LGN neurons (Figure 1). Morphological evidence and local circuit connectivity suggest that corticogeniculate neurons in rodents are more homogeneous than those in primates or carnivores (Brumberg et al. 2003, Olsen et al. 2012), suggesting that corticogeniculate feedback in rodents may not be strictly organized into parallel streams.

Although corticogeniculate neurons make up a minority of pyramidal neurons in layer 6 of the primate and carnivore V1 (approximately 15% in primates and approximately 50% in carnivores) (Ferster & Lindstrom 1985, Fitzpatrick et al. 1994, Gilbert & Kelly 1975), their anatomical input onto LGN relay neurons is robust. Corticogeniculate neurons contribute the largest proportion of synapses onto LGN relay neurons, sometimes outnumbering RGC synapses by almost 10 to 1 (Erisir et al. 1997a,b; Guillery 1969). In spite of the greater number of corticogeniculate synapses onto LGN relay neurons, corticogeniculate influence over LGN neurons is likely to be modulatory rather than driving. Corticogeniculate synapses are small, are located on distal dendrites of LGN relay neurons, have a low probability of neurotransmitter release, activate a combination of ionotropic and metabotropic glutamate receptors, and generate small postsynaptic currents, in contrast to RGC synapses, which are large, are located on proximal dendrites, have a high release probability, activate only ionotropic receptors, and generate quite large postsynaptic currents (Bickford 2016; Sherman & Guillery 1998, 2006). The driver–modulator framework explains why LGN relay neurons have visual physiology matching their RGC inputs rather than their corticogeniculate inputs, in spite of the greater number of the latter. However, the anatomy may not paint a complete picture. Studies in LGN slices have revealed multiple amplification mechanisms mediated by NMDA receptors and T-type calcium channels within LGN relay neuron dendrites that could increase the drive of corticogeniculate inputs, especially when those inputs are temporally synchronous (for details, see Connelly et al. 2016).

In focusing on synaptic connections between corticogeniculate neurons and LGN relay neurons, the driver–modulator framework does not account for another important component in overall corticogeniculate feedback circuit organization: disynaptic inhibitory connections. In addition to direct excitatory synapses onto LGN relay neurons, corticogeniculate axons synapse onto inhibitory interneurons within the LGN, which make up approximately 20% of LGN neurons, as well as inhibitory neurons in the TRN (Sherman & Guillery 2006). TRN neurons can exert different impacts on LGN neurons, depending on context (Cruikshank et al. 2010, Lam & Sherman 2010), and the overall balance of excitation and inhibition in LGN/TRN neurons depends on dynamic interplays among the spiking modes of corticogeniculate, LGN, and TRN neurons (Alexander & Godwin 2005, Cudiero et al. 2000). For example, chronic elimination of corticogeniculate feedback results in prolonged depolarization among LGN neurons during synchronized electroencephalogram states, likely due to a loss of inhibition from TRN and LGN interneurons (Eyding et al. 2003). Stimulating corticogeniculate neurons at different frequencies can also induce sleep-like spindles or slow oscillations in LGN neurons through activation of rhythmic TRN inhibition (Bal et al. 2000, Destexhe 2000). Additionally, stimulation of corticothalamic circuits in the somatosensory and auditory systems with low- or high-frequency patterns can generate net thalamic suppression or facilitation and bias detection versus discrimination ability (Crandall et al. 2015, Guo et al. 2017). A thorough understanding of the interplay between direct excitatory and indirect inhibitory inputs onto LGN neurons requires further in vivo investigation using methods to selectively activate and record from excitatory and inhibitory neurons in these circuits.

There are several similarities between the termination patterns of corticogeniculate and corticocortical feedback axons. Morphological studies of corticocortical feedback axon termination patterns in the visual cortex reveal multiple arborization patterns, including axon arbors that extend over large cortical distances with distributed or clustered endings (Rockland & Knutson 2000, Rockland & Virga 1989). For both corticogeniculate and corticocortical pathways, feedback axon termination patterns are wider than those of the feedforward inputs impinging on the same target area (Figure 1), and the distribution of feedback terminals is equivalent to the receptive field size of the feedback neurons (for a review, see Angelucci & Bressloff 2006). Furthermore, extrastriate feedback terminals within V1 are retinotopically coextensive with V1 projection neurons, suggesting alignment of feedforward and feedback connections (Angelucci et al. 2002). Evidence suggests that, while feedforward cortical inputs drive visual responses, corticocortical feedback inputs are more modulatory (Crick & Koch 1998, Hupe et al. 2001), consistent with the driver–modulator framework proposed for corticothalamic connections. Along similar lines, corticocortical feedback neurons may outnumber corticocortical feedforward neurons (Rockland 1997). Finally, like corticogeniculate feedback, corticocortical feedback is mediated by excitatory neurons that may make synaptic connections with both excitatory and inhibitory neurons within their target region (Anderson & Martin 2009, Angelucci & Bressloff 2006, Angelucci et al. 2017).

4. PHYSIOLOGY OF FEEDBACK CONNECTIONS

The anatomy of cortical feedback provides some useful clues about possible physiological purposes. In visual mammals, feedback terminal distributions suggest that the influence of feedback is restricted to the receptive field of the feedback neurons. Where further morphological details are available, as for corticogeniculate neurons, evidence suggests that feedback is made up of multiple distinct cell classes, each projecting to a unique target neuronal population. Physiological data from primates and carnivores strongly support this theory and provide additional evidence that corticogeniculate circuits are organized into parallel processing streams that match the feedforward streams. The extent to which corticocortical feedback pathways are made up of unique feature-selective neuronal subtypes is yet to be determined.

The first indication that corticogeniculate neurons are a heterogeneous group came from antidromic stimulation studies in which corticogeniculate neurons were identified by recording backward-propagating spikes generated through current injection at their terminals in the LGN. In primates and carnivores, corticogeniculate neurons with the fastest conducting axons are Complex cells; those with intermediate conduction speeds are Simple cells; and a third type with very slowly conducting, likely unmyelinated, axons are Complex cells or neurons that are minimally responsive to visual stimulation (Briggs & Usrey 2005, 2007; Grieve & Sillito 1995; Harvey 1978; Tsumoto & Suda 1980). Further examination of visual physiology in primate corticogeniculate neurons revealed that fast-conducting Complex corticogeniculate neurons had saturating contrast response curves, preferred higher temporal frequencies, and demonstrated extraclassical surround suppression; medium-conducting Simple corticogeniculate neurons had linear contrast response curves, preferred lower temporal frequencies, had little extraclassical suppression, and were preferentially driven by L and M cone–isolating stimuli; and the slowest-conducting Complex corticogeniculate neurons had intermediate tuning preferences and were the only neurons responsive to S cone–isolating stimuli (Briggs & Usrey 2009). These results illustrate the striking alignment of primate corticogeniculate cell types defined by axon conduction speed with the feedforward magnocellular, parvocellular, and koniocellular streams.

The fastest-conducting corticogeniculate neurons in primates and carnivores, the putative magnocellular- and Y-targeting neurons, have conduction latencies that are similar to feedforward geniculocortical latencies (Bullier & Henry 1979, 1980). Interestingly, fast-conducting corticogeniculate neurons are the only neurons to also receive suprathreshold direct geniculocortical feedforward inputs (Briggs & Usrey 2007). This means that, in primates at least, visual information can traverse a reciprocal loop from the LGN to V1 and back to the LGN in approximately 5 ms. The majority of corticogeniculate neurons, however, convey feedback signals to the LGN much more slowly than their feedforward counterparts. The range of conduction times across corticogeniculate cell types is quite large (from approximately 1 ms to >30 ms latency in primates, up to approximately 60 ms latency in carnivores). This range in conduction speeds among corticogeniculate neurons is not limited to highly visual mammals. Corticogeniculate neurons in rabbits also display a range of conduction latencies, and there are some relationships between conduction speed and physiological response properties, even though rabbit corticogeniculate physiology appears to fall along a continuum (Stoelzel et al. 2017, Swadlow & Weyand 1987). As in carnivores, the slowest-conducting corticogeniculate neurons in rabbits are difficult to drive with visual stimuli (Stoelzel et al. 2017, Tsumoto & Suda 1980), raising the issue of their function.

If there is one concrete take-away from anatomical and physiological data regarding corticogeniculate neurons, it is that these neurons are a heterogeneous population in highly visual mammals. Accordingly, the corticogeniculate feedback pathway is not one circuit, but rather a collection of distinct circuits conveying unique visual feature information and operating on a corresponding variety of timescales. This organizing rule is most pronounced in primates and carnivores, but evidence from rabbits suggests some diversity among corticogeniculate neurons, even if that diversity is organized along a continuum. Is there evidence for multiple subtypes of corticocortical feedback neurons within each areal corticocortical projection? Although anatomical data can be challenging to interpret in this context, injections of small amounts of tracers have revealed patchy patterns of label in V1 following injections into V2 in primates, suggesting that unique corticocortical feedback neurons target distinct populations of V1 neurons (Angelucci et al. 2002). Some evidence suggests that V2 cortical feedback axonal terminations in V1 are located in orientation domains matching the orientation preference of the V2 feedback neurons and are generally anisotropically distributed along the axis of V2 neuronal orientation preference (Angelucci & Bressloff 2006, Shmuel et al. 2005), although this notion is still controversial (Stettler et al. 2002). A recent two-photon imaging study of V2 corticocortical feedback terminals in V1 in tree shrews showed that V2 feedback boutons were spatially (and retinotopically) organized within the V1 orientation map and tuned to the same orientation as their target V1 neurons (Zhang et al. 2018). Interestingly, dissimilar results were reported in mice, where corticocortical feedback to V1 was coarsely retinotopically organized with some displacement orthogonal to the preferred orientation or direction axis of the feedback neurons (Marques et al. 2018). Corticocortical feedback conduction latencies may also vary, with some being quite fast, on the order of corticocortical feedforward latencies (Girard et al. 2001, Lamme et al. 1998).

If cortical feedback pathways are made up of heterogeneous neuronal subclasses as a rule, then this raises several important general questions. Why is it important to have multiple distinct subpopulations of feedback neurons, each conveying unique feature information? Presumably, preservation of feature-specific information prevents dilution of response properties among target neurons. A corollary question is: How highly processed can a cortical feedback signal be before it disrupts the response properties of its target neurons? Corticogeniculate neurons in primates share multiple tuning preferences with their putative LGN target neurons within each parallel stream. This could ensure that stream-specific signal transmission is preserved through the LGN but also seems redundant. Is there a secondary purpose for stream-specific feedback that perhaps does not involve alteration of spatial receptive field properties? Lastly, why is it important that these subpopulations of feedback neurons each integrate and transmit visual signals on such different timescales? Hints at the answers to some of these questions are provided by studies that manipulate feedback circuits, but many of these general questions are still unanswered.

5. FEEDBACK FOR SPATIAL AND TEMPORAL PRECISION

Anatomical and physiological findings point toward the idea that cortical feedback neurons included within a projection pathway are heterogeneous, with each neuronal subtype conveying feature-specific information that is aligned with the feature selectivity of its target neurons. Evidence supporting this notion is strongest for corticogeniculate circuits in highly visual mammals, but similar organizing rules could apply to corticocortical feedback circuits and/or feedback circuits in other mammals as well. As proposed above, one reason for a feature- or stream-specific organization of feedback is to prevent dilution of feedforward visual signals that are themselves feature- or stream-specific. Another reason is that feedback may refine or produce an emergent spatial receptive field property within its target neurons that is aligned with the feature selectivity of the feedback neurons, akin to the egocentric selection described for corticothalamic feedback circuits in other sensory modalities (for a review, see Briggs & Usrey 2008). A third reason, which is perhaps most in line with current data, is that cortical feedback may regulate the timing and precision of feedforward visual information transmission, and it may do this separately for information within each stream.

The spatial classical receptive field properties for most visual neurons are derived from their feedforward inputs, which means that the influence of feedback on spatial receptive field properties is modulatory at best. A possible candidate emergent spatial receptive field property generated through feedback is extraclassical surround suppression because, by definition, the extraclassical surround is larger than the classical receptive field, and feedback axon terminations extend into regions corresponding to the surround (Angelucci et al. 2002). It is clear that extraclassical surround suppression emerges through a combination of feedforward, local, and feedback connections (Alitto & Usrey 2008, Angelucci et al. 2017), and there is evidence that both corticogeniculate (Andolina et al. 2013; Geisert et al. 1981; Jones et al. 2000, 2012; Murphy & Sillito 1987; Webb et al. 2002) and corticocortical feedback (Nassi et al. 2013, Nurminen et al. 2018) modulate classical and extraclassical receptive field interactions in their target neurons. However, many visual neurons lack extraclassical surround suppression, so feedback must perform additional functions beyond refining classical and extraclassical receptive fields.

Interestingly, it has been quite challenging to pinpoint consistent effects of cortical feedback on spatial receptive field properties in target neurons. Studies in multiple species reveal variable effects of corticogeniculate feedback manipulation on LGN spatiotemporal receptive field properties (Denman & Contreras 2015, Geisert et al. 1981, Marrocco et al. 1996). Changes in gain were often observed without accompanying changes in tuning. For example, removing corticogeniculate feedback or enhancing it optogenetically led to decreased or increased responses, respectively, to moving stimuli without altering LGN tuning preferences, and this effect was specific to Y LGN neurons in the latter case (Gulyas et al. 1990, Hasse & Briggs 2017a, Marrocco et al. 1996). Similarly, cooling or lesioning V1 led to reduced contrast gain and adaptation, mostly in parvocellular and Y LGN neurons (Li et al. 2011, Przybyszewski et al. 2000). While these results do not provide a clear picture of how corticogeniculate feedback modulates LGN spatial receptive field properties, differential effects across LGN cell types provide support for the notion that corticogeniculate feedback modulates LGN neurons in a stream-specific manner.

It is possible that cortical feedback does not alter the spatial receptive field properties of individual target neurons but does coordinate the activity of specific ensembles of target neurons to produce an emergent spatial response pattern among that ensemble. Support for this theory comes from studies showing that corticogeniculate feedback facilitates the activity of, and synchronizes, LGN neurons whose receptive fields are aligned with the orientation axis of the feedback neurons (Andolina et al. 2007; Sillito et al. 1993, 1994; Wang et al. 2006). Furthermore, the precise alignment of LGN neuronal receptive fields relative to the preferred orientation axis of the corticogeniculate neurons matters: LGN neurons with aligned receptive fields are enhanced, while those with misaligned receptive fields are suppressed or shifted, when corticogeniculate feedback is stimulated (Tsumoto et al. 1978, Wang et al. 2018). Coordinating the responses of groups of neurons encoding relevant visual features, individually or as an ensemble, seems like a viable function for cortical feedback. Further experimental work will need to explore how feedback generates ensemble response patterns and whether coordinated signals among neurons in these ensembles boost feedforward visual signal transmission.

While the effects of cortical feedback on spatial receptive fields are subtle and have been difficult to pinpoint, several interesting observations suggest that cortical feedback plays a significant role in regulating temporal aspects of target neuronal responses. When corticogeniculate feedback is suppressed, variability in LGN responses to visual stimuli increases, LGN interspike-interval distributions are significantly broadened, and responses to a variety of visual stimuli become more similar, all of which suggest that corticogeniculate feedback increases response precision, reliability, and information about unique stimulus features for each spike (Andolina et al. 2007, Funke et al. 1996,McClurkin et al. 1994). Using optogenetics to enhance corticogeniculate activity, Hasse & Briggs (2017a) found that visually evoked responses in both X and Y LGN neurons were faster, less variable, and more precise. A possible mechanism by which corticogeniculate feedback could regulate LGN spike timing is through control of conductance noise in LGN neurons, enabling efficient transmission of visual information regardless of the membrane potential state or spiking mode of LGN neurons (Wolfart et al. 2005). Increasing the temporal precision of LGN visual responses could have a large impact downstream. When spikes from multiple LGN neurons converging on a V1 layer 4 neuron are tightly correlated, arriving within a few milliseconds of each other, they are more effective at driving a response in the V1 neuron (Alonso et al. 1996, Usrey et al. 2000).

The idea that cortical feedback primarily regulates response timing and precision among target neurons is appealing because it explains several puzzling findings, including subtle effects of feedback on gain and spatial receptive field properties, as described above. Timing control of target neurons could also be implemented through relatively simple connections such as monosynaptic excitation and fast, disynaptic inhibition (e.g., corticogeniculate inputs onto LGN relay neurons and inhibitory neurons in the TRN and LGN). Furthermore, it makes sense that feedback would regulate spike timing and precision in a stream- or feature-specific way and on different timescales to ensure that synchronous spiking is restricted to target neuronal ensembles conveying unique visual feature information. While these ideas are based on recent compelling data, they are still speculative and will require experimental confirmation.

6. FEEDBACK FOR CONTEXTUAL MODULATION

In the discussions above, emphasis is placed on visual functions for cortical feedback, such as refining the spatial and temporal responses of target neurons. However, there is a strong possibility that cortical feedback communicates extra-visual signals, in addition to purely visual signals, within the same pathway. In this case, the term extra-visual could refer to any aspect of cognitive function that is not strictly driven by feedforward visual sensory signals, including top-down information about context, attention, reward, task goals, or levels of arousal. Perhaps the best example of extra-visual information being conveyed by cortical feedback is the involvement of corticothalamic circuits in cycling between sleep and wakefulness through interactions among corticothalamic neurons, thalamic excitatory neurons, and TRN inhibitory neurons (for a review, see Steriade 2003). Corticothalamic circuits in many sensory systems and in all mammalian species studied display sleep–wake dynamics, suggesting that sensory corticothalamic circuits might multiplex sensory and arousal information. It is possible that corticothalamic circuits are completely overridden during sleep or under anesthesia, but this seems unlikely given that sensory responses are measurable in corticothalamic neurons during sleep and anesthesia. How then could cortical feedback circuits multiplex visual and arousal, or other extra-visual, signals? One possibility is that distinct subtypes of cortical feedback neurons handle visual or arousal signals. For example, the slowly conducting, non-visually responsive corticogeniculate neurons that have been identified in rabbits could mediate arousal, while faster-conducting corticogeniculate neurons could relay visual signals. Counterintuitively, arousal strongly modulates faster-conducting corticogeniculate neuronal activity and does not change the activity of slow, non-visual corticogeniculate neurons (Stoelzel et al. 2017). Additionally, all corticogeniculate subtypes recorded in alert or anesthetized primates are strongly visually responsive, irrespective of axon conduction speed (Briggs & Usrey 2007,2009). These data suggest that parallel streams of cortical feedback are present for the purpose of communicating unique visual information. Additionally, arousal signals are present in corticogeniculate neurons that communicate visual information in species with parallel streams of feedback (e.g., primates and carnivores) and in species with less evidence for strict parallel streams of feedback (e.g., rabbits and rodents). An intriguing hypothesis is that corticogeniculate circuits across a wide variety of species multiplex visual and arousal information, suggesting an evolutionary precedent for sensory and arousal signal multiplexing.

Direct evidence for visual and extra-visual signal multiplexing in cortical feedback circuits is lacking. However, there is ample evidence suggesting that cortical feedback circuits relay extra-visual signals. Given that corticothalamic circuits are involved in arousal, it is not a big leap to infer that they may play a role in selective attention as well. Indeed, attentional modulation in LGN neurons has been measured using functional magnetic resonance imaging (fMRI) in humans, as well as using single-unit neurophysiology in primates (Ling et al. 2015; McAlonan et al. 2006, 2008; O’Connor et al. 2002; Vanduffel et al. 2000), suggesting that corticogeniculate circuits convey attention signals from the visual cortex to the TRN and LGN. Along these lines, attention significantly enhances directed communication in the gamma frequency band between neurons in the deep layers of V1 and the LGN, but attention does not modulate interactions between neurons in the granular or superficial cortical layers and the LGN (Mock et al. 2018). Corticocortical feedback circuits also likely convey attention signals from higher to lower visual cortical areas (Buffalo et al. 2010, Gregoriou et al. 2009, Michalareas et al. 2016, Noudoost et al. 2010, van Kerkoerle et al. 2014). Whether attention is a separate signal that can be parsed from visual signals, or is instead an effect, like an amplification of visual signals, is open to debate. Accordingly, it could be that all circuits, both feedforward and feedback, can convey information about attention. These are key questions in the field of attention that need to be resolved with carefully designed experimental approaches.

Cortical feedback could also relay signals about the broader context in which relevant visual sensory information is embedded. One example of this is extraclassical surround suppression, discussed above. Another example is figure–ground interactions, which are globally important for perception but often involve integration of visual information from far outside the boundaries of the receptive fields of early visual neurons. A recent study showed figure–ground enhancements in the human LGN measured with fMRI even when the figure and ground elements were presented separately to each eye (Poltoratski et al. 2019), suggesting that the perceptual enhancement measured must be mediated by feedback from binocular cortical neurons. In the visual cortex, corticocortical feedback interacts with local horizontal connections to provide the spatial context for contour grouping and detection of textures (Klink et al. 2017, Liang et al. 2017).

More recently, many have proposed that cortical feedback is instrumental in probabilistic inference (for a review, see Cumming & Nienborg 2016) or predictive coding (Edwards et al. 2017, Pennartz et al. 2019, Rao & Ballard 1999). In these scenarios, top-down signals encapsulating information about subjects’ prior knowledge of a task, choices on preceding trials, or cognitive state are relayed through corticocortical feedback and combined with feedforward visual sensory information within the target neuronal population. Cortical feedback signals could regulate correlated variability across target neurons, depending on task conditions (Bondy et al. 2018), that could then instruct subjects’ choices or focus of attention. Some of these hypotheses have been tested experimentally, and others are based on theoretical predictions, but in both cases, the exact role of feedback circuits and how feedforward and feedback signals are integrated are still unclear.

7. CONCLUDING REMARKS

The specific functional contributions of cortical feedback to visual perception have been difficult to determine for many reasons. Until recently, technological limitations often prevented selective and reversible manipulations of feedback neurons without altering the activity of other cortical neurons. In addition, the majority of studies of cortical feedback have been conducted in anesthetized animals, and while visual responses are usually maintained, anesthesia can alter the dynamics of activities in cortical feedback and thalamic circuits. Perhaps the biggest barrier to understanding cortical feedback arises from the very nature of feedback as a modulator rather than a driver of target neuronal responses. In a general sense, feedback must be modulatory rather than driving to preserve the hierarchical organization of the visual pathways. If feedback, rather than feedforward, inputs were driving, then all visual receptive fields would be large and encode complex visual features regardless of their position in the visual processing hierarchy. It is intuitive to study spatial receptive field properties of visual neurons because these are the most salient and easiest to measure. However, if cortical feedback exerts minimal or subtle influence over spatial receptive field properties, then our focus must broaden to include non-spatial response properties, and extra-visual responses as well, to elucidate the function of feedback.

Anatomical and physiological characteristics of corticogeniculate and corticocortical neurons suggest that cortical feedback pathways are made up of multiple distinct neuronal subtypes, each relaying unique information about visual features. Cortical feedback is retinotopically and spatially organized, with axonal termination patterns corresponding to the size and orientation preference of the feedback neurons. Furthermore, the visual response preferences of cortical feedback neurons match those of their target neurons. For example, corticogeniculate subtypes in highly visual mammals display visual response properties that align with the three parallel processing streams in the retino-geniculocortical pathways. Additionally, corticogeniculate subtypes also operate on timescales that can differ by an order of magnitude. Functional manipulations of corticogeniculate feedback reveal subtle effects of feedback on spatial receptive field properties but more robust modulations of the temporal dynamics and precision of LGN responses. Although it is unclear whether cortical feedback multiplexes visual and extra-visual signals, cortical feedback does play a role in conveying information about arousal, attention, context, and possibly prior knowledge of task or goals. Although significant progress has been made in characterizing the anatomy, physiology, organization, and functional contributions of cortical feedback circuits in visual perception, many important questions remain unresolved and are therefore ripe for further experimentation.

SUMMARY POINTS.

1. Cortical feedback in the visual system is anatomically robust but does not drive physiological responses of target neurons.

2. Visual cortical feedback pathways are made up of multiple distinct neuronal subtypes, each of which projects to unique target neuronal populations.

3. Corticogeniculate subtypes in highly visual mammals have visual physiological response properties that match those of neurons in the feedforward parallel retino-geniculocortical streams.

4. Corticogeniculate subtypes operate on timescales that can differ by an order of magnitude from very fast (approximately 1 ms conduction time) to very slow (approximately 30 ms conduction time).

5. Visual cortical feedback has a subtle impact on spatial receptive field properties of target neurons, including some modulation of center and extraclassical surround interactions.

6. Corticogeniculate feedback regulates the timing and precision of LGN responses, perhaps to synchronize specific ensembles of feature-selective neurons.

7. Cortical feedback circuits may multiplex visual and extra-visual information including signals related to arousal state, context, or task goals.

FUTURE ISSUES.

1. Are all visual cortical feedback pathways comprised of distinct neuronal subtypes, each processing unique information about visual features?

2. What is the precise connectivity of visual cortical feedback neurons with their neuronal targets, how many neurons are contacted, and are they all of the same physiological type? In other words, if visual cortical feedback pathways are comprised of distinct neuronal subtypes as a rule, then how precisely feature matched are these feedback circuits at the level of individual neurons?

3. What is the purpose of branching axon collaterals of cortical feedback neurons that synapse onto local neurons within the cortex and/or subcortical structures?

4. How many visual brain areas are typically targeted by cortical feedback axon collaterals in different species?

5. Do corticocortical feedback circuits regulate the timing and precision of target neuronal responses to visual stimuli in a manner similar to the action of corticogeniculate circuits?

6. How do visual cortical feedback circuits multiplex visual and extra-visual information?

Glossary

Corticothalamic

general term for feedback circuits from the cortex to the thalamus

V1

primary visual cortex, also called area 17, according to Brodmann’s cortical area definitions, or the striate cortex

LGN

dorsal lateral geniculate nucleus of the thalamus, the primary sensory thalamus that processes visual inputs

Corticogeniculate

feedback circuits from the visual cortex (V1 and V2) to the LGN, a subset of corticothalamic pathways

Corticocortical

general term for connections between cortical areas, in this case referring to connections between visual cortical areas

V2

secondary visual cortex in primates, homologous to area 18 in carnivores