How Cortical Circuits Implement Cortical Computations: Mouse Visual Cortex as a Model

Abstract

The mouse, as a model organism to study the brain, gives us unprecedented experimental access to the mammalian cerebral cortex. By determining the cortex’s cellular composition, revealing the interaction between its different components, and systematically perturbing these components, we are obtaining mechanistic insight into some of the most basic properties of cortical function. In this review, we describe recent advances in our understanding of how circuits of cortical neurons implement computations, as revealed by the study of mouse primary visual cortex. Further, we discuss how studying the mouse has broadened our understanding of the range of computations performed by visual cortex. Finally, we address how future approaches will fulfill the promise of the mouse in elucidating fundamental operations of cortex.

INTRODUCTION

What chiefly distinguishes cerebral cortex from other parts of the central nervous system is the great diversity of its cell types and interconnexions. It would be astonishing if such a structure did not profoundly modify the response patterns of fibres coming into it.

—Hubel & Wiesel (1962, p. 106)

These two prescient sentences at the beginning of Hubel & Wiesel’s (1962) landmark publication encapsulate a key question in understanding visual cortex: How do diverse cell types and their connectivity sort out and make sense of the incoming stream of visual information? The study of mouse primary visual cortex (V1) has advanced our understanding of the relationship between cortical structure and function. How so? Progress in biology relies on the discovery of a phenomenon, its quantification and parameterization, and its mechanistic understanding. Research on primates and carnivores led to the discovery of many fundamental response properties of visual cortex to visual stimuli and allowed investigators to precisely quantify these responses relative to the stimulus parameters (Gilbert & Wiesel 1990, Hubel & Wiesel 1968, Movshon et al. 1978, Reid et al. 1991). Furthermore, this quantitative effort has led to models of cortical function that made mechanistic predictions (e.g., Carandini & Heeger 1995, Ferster & Miller 2000). However, due to the complexity of the mammalian cortex, experimental verification of these predictions has been slow and required heroic effort (Malpeli et al. 1981, Priebe & Ferster 2008, Reid & Alonso 1995, Sillito 1975). As a consequence, our mechanistic understanding of cortical vision has lagged substantially relative to the progress made on discovering and parameterizing phenomena. How do the different types of cortical neurons each contribute to the response of V1 to visual stimuli? What is the role of local recurrent connectivity among cortical neurons? How are the results of computations distributed to downstream targets to enable visually guided behavior? By using the mouse, we can harness the power of molecular biology to selectively record and perturb the activity of the individual cellular components of the cortex and provide insight into the cellular mechanisms that enable V1 to see.

How appropriate is mouse V1 for studying cortical computations? A number of response properties of mouse V1 to visual stimuli and nonvisual variables have been well established and parameterized (Ayaz et al. 2013; Niell & Stryker 2008, 2010; Self et al. 2014; Van den Bergh et al. 2010). A mechanistic understanding of how the underlying computations are implemented provides profound insight into the biology of the mammalian cortex in general. Can these findings generalize to V1 of other species? Many fundamental properties of visual cortical function originally reported in primates and carnivores are similar in the mouse (Niell & Stryker 2008, Van den Bergh et al. 2010). The question of whether those computations are implemented by the same mechanisms in mice as they are in the cortex of carnivores and primates will have to wait until we can experimentally access the cortex of other model organisms as thoroughly as we can access that of the mouse today. However, some examples already suggest that at least some of these mechanisms are indeed shared across species (Lien & Scanziani 2013, Liu et al. 2011, Priebe & Ferster 2008, Reid & Alonso 1995). Clearly, the mouse visual system differs from that of the primate in several ways, including low acuity, lack of a fovea, the natural visual environment within which it operates, and the repertoire of visually guided behaviors it subserves (Huberman & Niell 2011, Seabrook et al. 2017). However, even in foveate animals, much of the cortex is dedicated to processing vision outside the fovea, and mouse vision shares a strong similarity with primate peripheral vision, from the low acuity and rod dominance to behavioral roles such as detecting salient stimuli and guiding navigation. We expect that studies of mouse V1 will reveal canonical principles of visual cortical function and may demonstrate that primate specializations represent detailed implementation rather than fundamental differences.

Finally, the striking similarity between the microcircuit organization across cortical areas has long suggested the possibility that the cortical circuit performs a canonical computation (Douglas et al. 1989, Miller 2016) shared across modalities and species but that differs based on the nature of the input received and the detailed local connectivity of individual neurons. Thus, determining how the circuits of mouse V1 implement visual processing could lead to a general understanding of how cortex computes.

COMPUTATION OF VISUAL RESPONSE PROPERTIES BY THE CORTICAL CIRCUIT

The amount of available data in mouse visual cortex on cell types, including morphology, electrophysiological properties, molecular identity, and local and long-range connectivity patterns, exceeds that of any other area of mammalian cortex (Gouwens et al. 2019, Harris et al. 2019, Jiang et al. 2015, Pfeffer et al. 2013, Tasic et al. 2016). This list of parts and their wiring provides the basis for understanding the cellular mechanisms of cortical computations and constraining models. In the Supplemental Appendix, we present a very brief overview of the anatomical organization of visual cortex (Figure 1), including cell types and their connectivity, to provide context for understanding the computations performed. Below, we describe how these circuit elements generate a range of visual response properties, from orientation selectivity to contextual modulation, beginning with the transformation that occurs from the thalamic input to cortical responses in layer 4 (L4).

Figure 1.

Canonical circuits of mouse visual cortex. (a) Overlay of V1 and higher visual areas on the mouse brain showing location and relative size. (b) Summary of layer-specific excitatory connectivity. V1 outputs project to higher visual areas and other cortical regions as well as many subcortical targets, including structures involved in behavioral output such as superior colliculus and basal ganglia. Panel b adapted from Ji et al. (2016), Jiang et al. (2015), Morgenstern et al. (2016), and Seeman et al. (2018). (c) Summary of inhibitory connectivity motifs. Panel c adapted from Karnani et al. (2016), Lee et al. (2010), and Pfeffer et al. (2013). (d) Thalamocortical input targets both excitatory neurons and PV-positive inhibitory neurons. Panel d adapted from Ji et al. (2016), Jiang et al. (2015), and Seeman et al. (2018). See the Supplemental Appendix for further overview of anatomical circuit organization. Abbreviations: dLGN, dorsal lateral geniculate nucleus; E, excitatory; L, layer; PV, parvalbumin; SOM, somatostatin; V1, primary visual cortex; VIP, vasoactive intestinal peptide.

The Thalamocortical Transformation

As in other mammalian species, most dorsal lateral geniculate nucleus (dLGN) neurons have a standard center-surround organization (Figure 2a), with either ON or OFF polarity (i.e., responding to increments or decrements of light, respectively) and transient or sustained temporal dynamics. A minority of neurons have more diverse response properties (Marshel et al. 2012, Piscopo et al. 2013, Zhao et al. 2013), including neurons that prefer stimuli moving in a certain direction (direction selectivity). Although such noncanonical responses are present in the dLGN of other species, including cat and primate, their proportion is greater in the mouse, though still a small fraction of the total dLGN population (Scholl et al. 2013). Since the discovery that receptive field (RF) structure in V1 differs from that in dLGN, a large effort has gone into understanding the logic of this transformation.

Figure 2.

Overview of classical visual response properties. (a) Response of ON sustained (top) and OFF transient (bottom) center-surround neurons. (b) Response of an orientation-selective neuron preferring vertical orientation. (c) Response of a direction-selective neuron preferring rightward motion.

Thalamic convergence and unitary amplitude.

It takes many thalamic afferents to fire a cortical neuron. A single thalamic afferent impinging onto L4 excitatory neurons triggers a so-called unitary excitatory postsynaptic potential averaging 0.8 mV (Lien & Scanziani 2018) [ranging from0.1 to 3.4 mV; similar to that observed in rat somatosensory cortex (Bruno & Sakmann 2006) and cat visual cortex (Sedigh-Sarvestani et al. 2017)], thus too small to depolarize the membrane enough to reach threshold for action potential generation. As a consequence, the activation of L4 excitatory neurons by a visual stimulus must rely on the summed activity of many dLGN neurons. This is an important property because it means that a L4 neuron can selectively respond to features of the visual environment that are not represented by any of its individual dLGN afferents but are instead captured by a conjunction of features in the activity of the dLGN afferents from which it receives input. In other words, L4 neurons can extract features of the visual environment that are not explicitly represented by individual neurons in upstream stages of visual processing. It has been estimated that, in response to a visual stimulus, L4 excitatory neurons receive the convergent activity of approximately 80 dLGN inputs (Lien & Scanziani 2018, Bruno & Sakmann 2006). However, a much smaller number of dLGN inputs, as low as two to six, may be sufficient to account for the majority of a cortical neuron’s response (Ringach 2021).

Orientation selectivity.

Neurons in V1 preferentially respond to edges of luminance of a particular orientation, a property referred to as orientation selectivity (OS) (Figure 2b). This property, originally discovered in cat (Ferster & Miller 2000, Hubel & Wiesel 1962), has been confirmed across mammalian species, including mouse V1 (Niell & Stryker 2008), and represents one of the most salient differences between the response of dLGN and cortical neurons. Because most dLGN neurons do not show a preferential response to edges of any particular orientation (Piscopo et al. 2013), most OS responses are most likely generated in cortex. Is OS in mouse V1 a property that emerges through the interaction between cortical neurons or through the convergence of multiple dLGN afferents onto cortical neurons? A number of models have been proposed (Ferster & Miller 2000, Hubel & Wiesel 1962), and evidence for the convergence of appropriately aligned dLGN inputs was provided by heroic paired recordings in dLGN and cortex of cat (Reid & Alonso 1995). However, to directly address this question, one needs to isolate thalamic excitation from recurrent cortical excitation, an approach that was pioneered in the cat (Ferster et al. 1996) and recently optimized using genetic approaches in the mouse. By performing whole-cell recordings from L4 excitatory neurons while optogenetically silencing V1, it is possible to directly record thalamic synaptic excitation in isolation (Li et al. 2013, Lien & Scanziani 2013). This approach revealed that the RF structure generated by the ensemble of dLGN afferents converging onto individual L4 excitatory neurons is made of spatially separated yet overlapping ON and OFF subregions (Lien & Scanziani 2013). This RF structure likely results from the fact that ON- and OFF-centered dLGN neurons with spatially offset RFs converge onto individual L4 neurons (Figure 3a), thereby imparting OS (Figure 3b,c). Thus, the convergence of dLGN neurons with distinct polarities (either ON or OFF) and distinct RF location imparts L4 neurons with the ability to detect a feature of the visual environment that is not necessarily captured by any individual dLGN neuron from which they receive input.

Figure 3.

The emergence of orientation and direction selectivity in L4. (a) L4 neurons receive spatially offset ON versus OFF input from dLGN, which imparts OS. (b) To optimally activate a L4 neuron, the dark portion of an edge needs to cover the OFF region and the bright portion the ON region, a configuration that occurs if the main axis of the edge is perpendicular to the axis that connects the center of the ON and the OFF subregions of the RF. Any other orientation of the edge would produce a suboptimal excitation of the L4 neuron. As a grating drifts across the RF, it produces alternating optimal (t1) and suboptimal (t2) excitation. (c) Examination of the peak excitation across orientations reveals an orientation tuning curve. Panels a–c adapted from Lien & Scanziani (2013). (d) Direction-selective L4 neurons receive spatially offset transient versus sustained input from dLGN. (e, left) A visual stimulus moving in the preferred direction will first cross the RF of the dLGN neuron with a sustained response and then the RF of the dLGN neuron with a transient response. By doing so, the excitation produced by these two inputs will sum optimally. (Right) When the stimulus moves in the opposite direction, the transient response will have already largely subsided by the time the stimulus crosses the RF of the dLGN neuron with a sustained response, leading to less summation. Panels d and e adapted from Lien & Scanziani (2018). Abbreviations: dLGN, dorsal lateral geniculate nucleus; DS, direction selectivity; L, layer; OS, orientation selectivity; RF, receptive field; S, sustained; T, transient.

Pharmacological and genetic manipulations in several mammals, including mice (Sarnaik et al. 2014), demonstrate that OS can persist even in the absence of ON inputs from the retina, implying that other mechanisms could also contribute to OS, for example, the convergence of aligned dLGN inputs of like polarity along the axis of preferred orientation (Chapman et al. 1991, Li et al. 2013).

Direction selectivity.

Moving stimuli are particularly salient. The ability of V1 neurons to preferentially respond not only to edges of luminance of a specific orientation but also to the motion of those edges in a specific direction is another prominent feature of cortical responses to visual stimuli (Hubel & Wiesel 1959, 1962). This property is referred to as direction selectivity (DS) (Figure 2c), and the underlying mechanisms have intrigued scores of scholars. Like V1 neurons of many mammals, neurons in mouse V1 also display DS (Niell & Stryker 2008). How does DS emerge in V1? Because motion is a process that occurs across space and time, DS requires a comparison across those two dimensions (Albrecht & Geisler 1991, DeAngelis et al. 1993, Livingstone 1998, McLean & Palmer 1989, Reid et al. 1987, Saul & Humphrey 1992). Using the approach described above, namely the optogenetic silencing of V1 to isolate thalamic excitation, it could be demonstrated that, in L4 neurons, DS emerges through the convergence of dLGN neurons with distinct spatial and temporal RFs (Lien & Scanziani 2018) (Figure 3d). The RF of dLGN neurons is characterized not only by its spatial coordinates and polarity (ON or OFF) but also by its temporal properties. dLGN neurons can have different response dynamics, with some responding transiently to a visual stimulus while others respond in a more sustained manner (Lien & Scanziani 2018, Piscopo et al. 2013). The convergence of dLGN neurons with spatially offset RFs and with different response dynamics to visual stimuli onto individual L4 neurons imparts DS (Figure 3e). This is another clear example of how, by combining the response diversity of dLGN neurons, L4 neurons can extract features of the environment (e.g., direction of motion) that are not necessarily represented in the activity of any of the individual dLGN neurons from which they receive input. DS is further enhanced in V1 through the spatial separation of the RFs of excitatory and inhibitory synaptic conductances. Electrophysiological recordings from L4 neurons show that, in direction-selective neurons, the spatial position of a stimulus that triggers maximal excitation is offset relative to the position that triggers maximal inhibition (Li et al. 2015). This offset results in a delay of inhibition relative to excitation specifically for stimuli moving in the preferred direction, thus contributing to the DS of the neuron. Similarly, the spatial position (in terms of retinotopic coordinates) of inhibitory neurons that are presynaptic to a direction-selective neuron is offset relative to that of its presynaptic excitatory neurons. This spatial offset predicts the preferred direction of the direction-selective neuron (Rossi et al. 2020).

A small fraction of dLGN neurons inherit DS from the retina (Piscopo et al. 2013) and project across cortical layers (Sun et al. 2016). Do they contribute to DS in V1? Disruption of DS in the retina does not affect the overall distribution of direction-selective neurons in V1, implying that most DS in V1 is not inherited from the retina (Hillier et al. 2017). However, after impairing retinal DS, one observes a reduction in the fraction of L2/3 neurons whose preferred direction is posterior motion (i.e., the direction of lateral visual flow experienced by the mouse as it moves forward in its environment) (Hillier et al. 2017). Thus, while the direction preference for most motion directions is computed de novo in V1, the sensitivity to posterior motion in L2/3 is in part inherited from the retina (Cruz-Martín et al. 2014, Rasmussen et al. 2020).

Computations by Recurrent Excitation

Even though dLGN afferents are the main input from the visual periphery into V1, they represent only a minority of the excitatory drive within cortex. Recurrent synaptic connections among excitatory neurons are a defining characteristic of the cortical circuit. Below we discuss how recurrent excitation shapes the stimulus selectivity and response dynamics.

Functional connectivity among V1 excitatory neurons.

Beyond the general intra- and interlaminar connectivity principles (Figure 1b; Supplemental Appendix), what are the rules that guide specific connectivity among excitatory neurons in V1, and how do they relate to their functional properties? A general principle of connectivity that has emerged through many studies is like-to-like (Figure 4a). In L2/3, for example, the probability of connection (Cossell et al. 2015, Ko et al. 2011, Lee et al. 2016, Wertz et al. 2015), the synapse size (Lee et al. 2016), and the unitary amplitude of connections (Cossell et al. 2015, Ko et al. 2011) are all greater among excitatory neurons with similar orientation preference. Consistent with this finding, a L2/3 excitatory neuron that receives excitatory input from another L2/3 excitatory neuron is more likely to project back to that L2/3 excitatory neuron than chance, and these two neurons are more likely than chance to receive a common input from a L4 excitatory neuron (Yoshimura et al. 2005). Like-to-like connectivity also occurs across layers, with L2/3 neurons being more likely to receive input from L4 neurons tuned to the same orientation (Rossi et al. 2020), although some interlaminar input populations are tuned for mismatched orientations (Wertz et al. 2015). This rule of connectivity is also reflected at the entry point of visual information into V1 where the tuning of the dLGN input matches that of recurrent excitation in L4 neurons (Li et al. 2013, Lien & Scanziani 2013), and connected L4 neurons are more likely to receive shared input from dLGN (Morgenstern et al. 2016). The rules of connectivity also obey general principles across cortical space, such that neurons with spatially offset RFs tend to be connected if their preferred orientations align with the axis connecting their RFs (Iacaruso et al. 2017, Rossi et al. 2020) (Figure 4b), consistent with tuning for extended contours. Thus, the visual response properties of V1 excitatory neurons are a key variable in predicting who talks to who.

Figure 4.

Recurrent excitatory connectivity. (a) Connectivity within the cortical circuit is primarily like to like, with preferential (though not exclusive) connections between neurons tuned to similar orientations. This applies to both feedforward connectivity from L4 to L2/3 and recurrent connectivity within L2/3. Panel a adapted from Ko et al. (2011), Lee et al. (2016), Lien & Scanziani (2013), Morgenstern et al. (2016), Rossi et al. (2020), Wertz et al. (2015), and Yoshimura et al. (2005). (b) The spatial organization of connectivity is coaxial, whereby a neuron (red) tends to receive input from neurons (green) with similar orientation preference and whose location, in retinotopic coordinates, is along the axis of their preferred orientation. Panel b adapted from Iacaruso et al. (2017) and Rossi et al. (2020). (c) Recurrent cortical excitation is larger than incoming thalamic excitation but is matched in tuning, resulting in cortical amplification (total excitation). Panel c adapted from Li et al. (2013) and Lien & Scanziani (2013, 2018). Abbreviations: dLGN, dorsal lateral geniculate nucleus; L, layer.

Dynamics of the cortical response: amplification.

While there are no exact numbers for the mouse, in other mammals only about 5–20% of excitatory synapses onto L4 neurons are of dLGN origin (Ahmed et al. 1994, Garcia-Marin et al. 2019). Consistent with this, in mice only 30% or so of visually evoked synaptic excitation of L4 excitatory neurons directly originates from dLGN afferents (Li et al. 2013, Lien & Scanziani 2013). The remainder of visually evoked excitation (~70%) is mediated by other cortical neurons (Li et al. 2013, Lien & Scanziani 2013), most likely L4 neurons, since they excite one another through recurrent connections (Seeman et al. 2018) (Figure 4c). The relative fraction of afferent thalamic versus recurrent excitation evolves in time after the onset of the visual stimulus (Reinhold et al. 2015). Excitation is predominantly thalamic at the beginning of the stimulus and shifts toward recurrent after a few tens of milliseconds as L4 excitatory neurons begin to fire action potentials in response to the stimulus (Reinhold et al. 2015). Visually evoked responses are thus amplified by recurrent connections. Importantly, this cortical amplification does not degrade the orientation and direction preference imparted by the dLGN input onto L4 neurons (Li et al. 2013, Lien & Scanziani 2013) because, as described above, the connectivity pattern among V1 neurons is biased toward neurons with similar orientation and direction preferences. Such amplification was one of the first proposed canonical cortical computations (Douglas et al. 1989).

Dynamics of the cortical response: decay time course.

Can recurrent excitation sustain visually evoked activity without ongoing thalamic input? Given that shortly after the onset of the visual stimulus, recurrent excitation makes up most of the excitation received by L4 neurons, one could assume that visually evoked activity may persist for some time even without ongoing dLGN input. This is, however, not the case. No matter how strong the visual stimulus is and, accordingly, how large the response of V1 to that visual stimulus is, silencing of the dLGN input to V1 leads to a fast decay (~10 ms) of the visual response in V1 (Reinhold et al. 2015). This fast decay is on the order of magnitude of the membrane time constant of a neuron. Thus, despite the strong amplification by recurrent excitation, the response of V1 remains tightly linked to the dLGN input. This ensures that temporal fluctuations in the response of dLGN neurons to visual stimuli are closely followed by temporal fluctuations in V1 activity (Reinhold et al. 2015). Selective amplification thereby achieves two functions: amplifying specific features and allowing high temporal fidelity (Murphy & Miller 2009). Intracortical inhibition likely plays a key role in the rapid decay of visually evoked activity in V1 following the interruption of dLGN input.

Computations Through Cortical Inhibition

Approximately 20% of cortical neurons are inhibitory (Meinecke & Peters 1987), and their integration into the cortical circuit ensures that during normal cortical function, excitation and inhibition are inseparable—they walk hand in hand. As discussed below, the combination of synaptic excitation and inhibition underlies several fundamental cortical computations. For an overview of inhibitory circuitry, see the Supplemental Appendix.

Computing with two opposing forces.

Visually evoked activity elicits both excitation and inhibition in V1. The recruitment of these two opposing forces already occurs in the first steps of cortical processing, because dLGN afferents contact both excitatory and PV-expressing inhibitory neurons in V1 (Ji et al. 2016) (Figure 1d). Furthermore, excitation by dLGN afferents is stronger onto PV neurons than onto excitatory neurons (Ji et al. 2016). As a consequence, even the weakest visual stimuli generate, on average, both excitation and inhibition in V1 neurons (Adesnik 2017) (Figure 5a), and as stimulus contrast is increased, excitation and inhibition grow approximately proportionally (Adesnik 2017) (Figure 5b). Why should the dLGN input to V1 push on both the accelerator and the brake at the same time? The functional consequences of the proportionality between excitation and inhibition in cortex have been the focus of extensive modeling (Ahmadian et al. 2013, Brunel 2000, Sadeh & Clopath 2021, van Vreeswijk & Sompolinsky 1996) and have been discussed in more detail elsewhere (Isaacson & Scanziani 2011, Miller 2016). Briefly, a proportional increase of inhibition with excitation enables V1 to respond over a wide range of stimulus intensities (Liu et al. 2011), remaining sensitive to weak stimuli (in which weak excitation is counteracted by only weak inhibition) yet not saturating in response to strong stimuli (because strong excitation is counteracted by strong inhibition). Through this proportional increase, for example, orientation tuning of a V1 neuron changes little or becomes even sharper as stimulus contrast increases (Li et al. 2012) and the inputs of the two eyes sum sublinearly in binocular neurons (Longordo et al. 2013). Furthermore, the ability to operate with strong excitation without the risk that recurrent excitatory synapses could lead to runaway excitation enables cortex to have fast dynamics (Reinhold et al. 2015); to use the analogy of a car, if you want to go fast, you need brakes to stay in control. The divergence of feedforward excitation onto excitatory and PV neurons also occurs at the next stage of processing, from L4 to L2/3, and here, too, L4 afferents provide stronger excitation onto PV neurons as compared to L2/3 excitatory neurons (Adesnik et al. 2012). Furthermore, L2/3 excitatory neurons homeostatically regulate the strength of PV-mediated inhibition as a function of the magnitude of excitation they receive from L4 (Xue et al. 2014). Thus, the divergence of afferent axons onto excitatory and inhibitory neurons ensures proportionality between excitation and inhibition and may be a general property of each stage of cortical processing (Miller 2016).

Figure 5.

Gain control through feedforward inhibition. (a) Stimuli of increasing contrast elicit larger synaptic depolarization of the membrane potential (Vm) and higher spike frequencies, yet the underlying synaptic excitatory (E) and inhibitory (I) currents increase proportionally, and even the weakest contrasts elicit both excitation and inhibition. (b) The change in spike rate (top) and excitatory and inhibitory currents (bottom) as a function of contrast. The proportional increase of excitation and inhibition prevents runaway excitation as stimulus contrast increases. Panels a and b adapted from Adesnik (2017). (c) Increasing stimulus contrast results in a multiplicative increase in response of excitatory neurons (E response) across the orientation tuning curve while leaving selectivity unchanged, i.e., a gain change. (d) Parvalbumin (PV) inhibitory neurons pool excitatory inputs tuned to different orientations, resulting in nonselective orientation tuning. Panel d adapted from Bock et al. (2011). (e) The response of excitatory neurons to stimuli of different orientations is modulated by the activity of PV neurons. Because PV neurons are reciprocally connected with the local excitatory population, their activity can provide multiplicative gain control that mimics the effects of varying contrast. Panel e adapted from Atallah et al. (2012).

In contrast to excitatory neurons, in mouse V1, PV inhibitory neurons tend to be broadly tuned for orientation (Kerlin et al. 2010, Liu et al. 2009, Niell & Stryker 2008, Runyan et al. 2010), reflecting the fact that they pool inputs relatively nonselectively from the local excitatory populations (Bock et al. 2011, Hofer et al. 2011). The result is that PV neurons provide untuned inhibition to their targets. PV-mediated inhibition thus reflects population activity rather than specific features of the stimulus, going up and down in tandem with excitatory neurons. This untuned inhibition is ideally suited to control the gain of V1 responses relative to the stimulus (Atallah et al. 2012, Lee et al. 2012) (Figure 5c–e), that is, to change the magnitude of the response of excitatory neurons without impacting their tuning preferences, like turning the volume knob on a stereo. Some PV neurons located in L6 extend their axons throughout the depth of the cortex (Bortone et al. 2014), which enables them to control the gain of V1 responses across all cortical layers (Atallah et al. 2012, Lee et al. 2012, Olsen et al. 2012). Untuned inhibition may also mediate divisive normalization (Wilson et al. 2012), a proposed canonical computation of cortex (Carandini & Heeger 2011). In contrast to PV neurons, other inhibitory neurons are more selective to the orientation of the stimulus (Ayzenshtat et al. 2016, Lee et al. 2012, Ma et al. 2010), but the mechanism and implications of this are less well understood.

Contextual modulation.

Context is a fundamental attribute to our perception of any sensory stimulus, providing meaning in the sensory scene. In the visual world, context refers to the visual environment surrounding a stimulus. Psychophysical experiments demonstrate that the context of a stimulus influences our perception of that stimulus, including its size, color, or contrast (Albright & Stoner 2002). Indeed, much of the computational power of cortex in visual processing may arise from such contextual modulation.

Several laboratories have found clear physiological signatures of such perceptual phenomena in V1 (Nurminen & Angelucci 2014), and experiments in mouse are contributing to our understanding of some of the underlying cellular mechanisms. Stimuli outside of the RF of a neuron, i.e., stimuli presented in its surround, while generally unable to elicit a response alone (but see below), can, however, modulate the response of the neuron to a stimulus placed in its RF. The nature of this modulation is often suppressive, as shown when the size of a stimulus is increased to cover both the RF and the surrounding regions, and is referred to as surround suppression (Nurminen & Angelucci 2014, Van den Bergh et al. 2010) (Figure 6a). What accounts for this suppression? Recordings in L2/3 of mouse V1 have shown that the relationship between excitation and inhibition changes with the size of the stimulus or with the position of the stimulus relative to the center of the RF. Inhibition becomes progressively more prominent relative to excitation as the stimulus size increases or as the stimulus is shifted toward the periphery of the RF (Adesnik 2017, Haider et al. 2013). What is the cellular basis for this phenomenon? Surround suppression is not only observed in excitatory neurons but also in two types of inhibitory neurons: the PV and vasoactive intestinal peptide (VIP) cells (Adesnik et al. 2012, Keller et al. 2020a). Remarkably, the other main type of inhibitory neuron, the somatostatin (SOM) cell, shows much less surround suppression, and the responses of these neurons instead often continue to increase with increasing stimulus size (Adesnik et al. 2012, Keller et al. 2020a) (Figure 6a). This property of SOM cells, and the fact that SOM cells inhibit all other cell types (Figure 1c), suggests that SOM cells may actually be a key contributor to surround suppression. Indeed, optogenetic silencing of SOM cells strongly diminishes surround suppression, at least in L2/3 excitatory neurons (Adesnik et al. 2012). Thus, SOM cells, by lacking surround suppression and hence robustly responding to large stimuli, suppress neighboring neurons, contributing to their surround suppression (Figure 6b).

Figure 6.

Inhibitory circuits for contextual modulation. (a) In contrast to excitatory (E) neurons, the response of somatostatin (SOM) neurons continues to increase up to a plateau with stimulus size. (b) SOM neurons receive input across a large area and do not inhibit each other. As a consequence, their response increases, leading to increasing responses with stimulus size. By inhibiting nearby excitatory neurons when large stimuli are presented, SOM neurons mediate surround suppression. Panels a and b adapted from Adesnik et al. (2012). (c–e) Contextual modulation depends on properties of the stimulus in the surround. An iso-oriented surround suppresses the response, while a cross-oriented surround increases the response. This gives rise to the illusory perception that the central grating contrast in panel d is greater than in panel c. The disinhibitory circuit from vasoactive intestinal peptide (VIP) neurons onto SOM neurons mediates this context dependence, as the activation of VIP neurons in the cross-oriented condition inhibits the surround suppression that would otherwise be provided by SOM neurons. Panels c–e adapted from Keller et al. (2020a).

It takes time for surround suppression to kick in. The presentation of a large stimulus, which covers both the RF and its surround, initially triggers a response in V1 excitatory neurons that is similar in magnitude to that of a stimulus that covers just the RF. Several tens of milliseconds later, however, the response starts to get suppressed (Self et al. 2014). Consistent with the role of SOM cells in surround suppression, their activation by a visual stimulus is also delayed (Ma et al. 2010). This delay likely results from the fact that SOM cells need repeated activity from their excitatory synaptic input in order to respond because their excitatory inputs are facilitating; that is, synaptic excitation starts small and increases progressively with repeated activity of the afferents (Karnani et al. 2016). Thus, synaptic dynamics of a specific component of the cortical circuit can affect the time course of the sensory response in V1.

Why do L2/3 SOM cells have little or no surround suppression? Unlike other L2/3 cell types, SOM cells do not receive afferent excitatory input from L4, the main input layer to L2/3 (Adesnik et al. 2012). The excitation of SOM cells is in large part provided by recurrent axons within L2/3. This property, together with the fact that SOM cells do not inhibit one another (Adesnik et al. 2012, Pfeffer et al. 2013) (Figure 1c; see also the section on inhibitory circuitry in the Supplemental Appendix), may underlie their ability to integrate stimuli that cover large portions of visual space (Figure 6b).

Not all stimuli in the surround are suppressive (Nurminen & Angelucci 2014). A grating in the surround of a neuron’s RF whose orientation is orthogonal relative to the orientation of the grating in the neuron’s RF (cross-oriented surround) triggers much less surround suppression than if its orientation were the same as that in the neuron’s RF (iso-oriented surround; as discussed above). This phenomenon may represent the physiological basis for the fact that the perception of a stimulus with a cross-oriented surround is more salient than the same stimulus with an iso-oriented surround, as demonstrated by many visual illusions (Figure 6c). Work in L2/3 of mouse V1 suggests that this contextual modulation relies, at least in part, on the reciprocal interaction between two types of inhibitory neurons, namely VIP and SOM cells (Keller et al. 2020a). These two neurons have complementary responses to iso- and cross-oriented surrounds. On one hand, iso-oriented stimuli elicit strong responses in SOM cells (as mentioned above) but only weak ones in VIP cells (Keller et al. 2020a) (Figure 6c). On the other hand, cross-oriented stimuli trigger strong responses in VIP cells but poorly stimulate SOM cells (Keller et al. 2020a). Importantly, VIP cells are integrated in the cortical circuit in a very particular manner: They preferentially inhibit inhibitory neurons rather than excitatory neurons (Pfeffer et al. 2013) (Figure 1c; see also the section on cell types and circuit organization in the Supplemental Appendix). Their activity thus has a disinhibitory impact on excitatory neurons in V1. Recent experimental and modeling approaches suggest that the activation of VIP cells by a cross-oriented surround suppresses SOM cells, thereby leading to the relief of excitatory neurons from suppression (Keller et al. 2020a). This same disinhibitory circuit has been shown to regulate the degree of surround suppression based on the contrast of the stimulus (Millman et al. 2019) as well as to enhance visual responses based on attentional inputs from cingulate cortex (Zhang et al. 2014). Thus, this canonical disinhibitory circuit may represent one of the key mechanisms for how context impacts the perception of a stimulus.

Finally, stimuli presented in the surround that would otherwise suppress the response to a stimulus in the RF can become excitatory when presented in the absence of a stimulus in the RF (Fiorani Júnior et al. 1992, Jones et al. 2001, Rossi et al. 2001, Schnabel et al. 2018, von der Heydt et al. 1984). These responses to the surround stimulus alone may represent the physiological signature of perceptual completion, that is, they may allow the visual system to use context to estimate the nature of a stimulus in the RF when the latter is poorly visible or occluded. Work in mouse has shown that the excitation of V1 neurons to stimuli presented in the surround alone is mediated by feedback projections originating in higher visual areas (HVAs) (Keller et al. 2020b). This is a clear example of how the response of V1 neurons is shaped not only by the feedforward pathway ascending from the retina and the local V1 circuitry but also by feedback projections descending from HVAs.

Sharpening the Receptive Field

Recurrent synaptic excitation among V1 neurons is biased toward neurons with similar orientation preferences (see the section titled Functional Connectivity Among V1 Excitatory Neurons), yet this bias is by no means absolute. Excitatory neurons in V1 also receive many synaptic inputs from excitatory neurons that have different orientation preferences. As a consequence, the orientation tuning of synaptic excitation is very broad, in fact much broader than the spiking response of the neurons themselves. What accounts for this sharpening of the spike response? Whole-cell recordings from mouse V1 neurons have demonstrated that at least two factors, spike threshold and broad inhibition, contribute to the sharpening (Figure 7). Due to the membrane potential threshold for action potential generation, there is a supra-linear relationship between the amplitude of the postsynaptic potential and spike frequency. As was shown originally in the cat (Priebe & Ferster 2008), this supralinear input-output relationship enables the largest visually evoked excitatory postsynaptic potentials to trigger spikes while the smaller ones remain sub-threshold, thereby sharpening the orientation tuning of the spike response (Liu et al. 2011) (Figure 7a). Another important factor in the sharpening of the spike response is the fact that the orientation tuning of synaptic inhibition is even broader than that of synaptic excitation. This is likely a consequence of the fact that, as discussed above, PV neurons, a main source of inhibition to excitatory neurons, are largely untuned to orientation. With synaptic inhibition more broadly tuned than synaptic excitation, the relationship between excitation and inhibition changes with the orientation of the stimulus, being biased toward excitation for the preferred orientation as compared to flanking orientations (Liu et al. 2011) (Figure 7b). Finally, active dendritic conductances also contribute to the sharpening of the orientation preferences of V1 neurons. Bursts of local dendritic sodium spikes generated in response to stimuli presented at the preferred orientation, but not at flanking orientations, sharpen the orientation tuning of the somatic depolarization (Smith et al. 2013) (Figure 7c). Local dendritic spikes may be a consequence of the correlated activity of synaptic inputs clustering on the dendrites of L2/3 neurons (Iacaruso et al. 2017, Lee et al. 2019). Indeed, inputs with overlapping RFs are more likely than chance to be close neighbors on a dendritic branch (Iacaruso et al. 2017). Furthermore, while the preferred orientation of inputs does not predict their spatial relationship on a dendrite (Chen et al. 2013, Iacaruso et al. 2017), in neurons receiving callosal projection from contralateral V1, the orientation preference of those callosal inputs correlates with the orientation preference of their noncallosal neighboring inputs (Lee et al. 2019). Thus, both synaptic and intrinsic voltage-dependent conductances profoundly shape the orientation tuning of the neuron.

Figure 7.

Mechanisms for sharpening response selectivity. (a) The membrane potential threshold for spike generation implies that even stimuli that elicit a significant depolarization may not elicit spiking, thereby sharpening the response to optimal stimuli. This is demonstrated in the temporal response to drifting gratings of varying orientation (left) and the corresponding orientation tuning curves (right) for membrane potential (top) and spikes (bottom). Panel a adapted from Liu et al. (2011). (b) The tuning of inhibitory synaptic conductances is broader than that of excitatory (E) synaptic conductances (top). The relative dominance of inhibition at nonpreferred orientations suppresses responses at these orientations and sharpens selectivity (bottom). Panel b adapted from Liu et al. (2011). (c) Visual input at the preferred orientation elicits dendritic bursts (top). This leads to greater orientation selectivity relative to when active conductances underlying dendritic bursts are suppressed (bottom). Panel c adapted from Smith et al. (2013).

Representation of Cortical Computations at the Population Level

The cortical response to visual stimuli has classically been characterized by tuning curves or RFs of individual neurons, as discussed in this section. However, the firing of one neuron cannot unambiguously convey information about the stimulus. As a simple example, a change in firing rate of an orientation-selective cell could result from either a change in stimulus contrast or a change in stimulus orientation, thus precluding readout of either parameter independently. Information encoding in individual neurons is further confounded by the fact that neuronal responses to a given stimulus can be highly variable. However, these challenges can be resolved by considering the representation across many neurons in the population code. Recent methods for recording large numbers of neurons have thus begun to reveal how sensory information is encoded across large populations of neurons (Fairhall 2014, Panzeri et al. 2015, Whiteway & Butts 2019).

One important aspect of a population code is how information is distributed across the activity of many neurons, in terms of the diversity and overlap of response properties. Stringer et al. (2019b) quantitatively assessed the response of tens of thousands of neurons to a large battery of natural scene images using two-photon calcium imaging in mouse V1. The results demonstrated that responses are distributed across the population according to a power law that optimizes efficiency (ability to represent as many stimuli as possible) while maintaining smoothness (similar stimuli evoke similar patterns of neural activity). Another key question in population coding is the impact of correlations on pooling population activity to overcome variability inherent to individual neurons. It has been known for some time that the variability in response to a stimulus is often correlated across neurons, as a result of multiple factors, including shared input that is not directly related to the stimulus (such as behavioral state variables) or fluctuations resulting from local network dynamics (Kohn et al. 2016, Engel & Steinmetz 2019, Zohary et al. 1994). These noise correlations may or may not limit the amount of information that large populations of neurons can encode, depending on how the correlations are aligned with stimulus coding. From a simple perspective, if the correlated variability between neurons results in a pattern of population activity that resembles the pattern evoked by a visual feature, then this creates a confound in pooling information from those neurons to decode that feature. However, determining whether this is in fact the case depends on recording from large numbers of neurons, in order to determine how decoding accuracy increases with the number of neurons. This is now feasible with two-photon calcium imaging. Rumyantsev et al. (2020) used this approach to demonstrate that the ability to decode stimulus orientation increases with pooling neurons up to at least 1,000 neurons or beyond, as only 10% of the noise correlation overlapped with stimulus coding. Stringer et al. (2019a) used a similar approach and found that information does not saturate even up to 20,000 neurons, with the consequence that readout from sufficiently large populations in mouse V1 could allow estimation of stimulus orientation to significantly less than one degree. Thus, it appears that correlations are distributed across neurons in a manner that minimizes the impact on the encoding of visual information and that limits on the accuracy of orientation discrimination as measured behaviorally [approximately 5 degrees (Glickfeld et al. 2013b)] likely arise downstream of V1.

These large-scale aspects of the cortical computation almost certainly depend on specific circuit mechanisms. For example, the distribution of information across neurons in a power law described above (Stringer et al. 2019b) likely results from patterns of synaptic connectivity that determine the diversity and width of tuning properties. Likewise, the correlation structure of population activity may be determined by cell type–specific circuit motifs. Indeed, modeling studies have suggested that local inhibition could play an important role in limiting correlations generated by local network activity, and in fact inhibitory neuron activity is increased in brain states characterized by lower correlations (Huang et al. 2019, Stringer et al. 2016). However, we still have little understanding of how layer- or cell type–specific interactions shape population codes relative to our understanding of single-neuron response properties. Applying the tools available for studying cortical computations in mouse at the level of population dynamics is therefore an important research direction.

NONVISUAL COMPUTATIONS IN V1

In addition to revealing circuit mechanisms underlying classical aspects of visual processing, studies of mouse visual cortex have also led to the discovery of novel aspects of neural coding in V1 and particularly the contribution of a wide range of nonvisual factors, including movement, navigation, arousal, and vestibular signals (Figure 8). These findings suggest that an important property of V1 is the ability to integrate different sources of sensory and nonsensory information in order to generate flexible representations of the sensory environment to drive appropriate behaviors.

Figure 8.

Nonvisual signals in primary visual cortex (V1). (a) The orientation tuning curve measured in head-fixed mice on a spherical treadmill shows a multiplicative increase (gain modulation) when the animal is moving versus stationary. Panel a adapted from Niell & Stryker (2010). (b) L6 neurons respond to vestibular input from rotation (top) and the corresponding rotation of the visual scene (middle), which summate and result in a signal representing head direction (bottom). Panel b adapted from Vélez-Fort et al. (2018). (c) Combining rich quantification of eye and body movements with neural recordings during ethological behavior, for example, cricket hunting, may allow investigation of visual and nonvisual signals in a natural context. Panel c adapted from Meyer et al. (2018) and Michaiel et al. (2020).

Locomotion and Arousal

Locomotion has a profound impact on mouse V1 (Niell & Stryker 2010), roughly doubling the responses to visual stimuli compared to when the animal is stationary (Figure 8a). This effect includes a shift in spatial integration by increasing the drive in the classical RF and decreasing the suppressive surround (Ayaz et al. 2013). Interestingly, locomotion alone can drive neural activity in V1 even in the absence of a visual stimulus (Keller et al. 2012), with a subset of neurons tuned to specific locomotion speeds, as well as differentially combining locomotion speed with visual stimuli (Saleem et al. 2013). Besides locomotion, a wide range of movements, from facial twitches to operant responses, generate activity in V1 (Musall et al. 2019, Stringer et al. 2019c). In fact, the effects of locomotion on V1 are often paralleled by general changes in arousal, although the effects are dissociable as well (Vinck et al. 2015).

At least some of the mechanisms by which locomotion exerts its impact rely on cholinergic neurons in the basal forebrain that project to V1 (Lee et al. 2014). By receiving input from the mesencephalic motor region, which encodes running speed, the basal forebrain is well poised to send locomotion signals to V1. Through the release of acetylcholine onto VIP neurons, locomotion likely engages a disinhibitory circuit involving the suppression of SOM neurons, as described above in the section titled Contextual Modulation (Fu et al. 2014). Measurement of the activity of three inhibitory neurons subtypes, however, along with computational modeling (Dipoppa et al. 2018), suggests that locomotion likely modulates multiple aspects of the cortical circuit rather than acting through a single mechanism.

Movement-Based Visual Computations

Locomotor signals represent at least two aspects of behavior: (a) the overall behavioral state (e.g., stationary versus moving, unalert versus aroused), which may serve as a global control and act through neuromodulatory mechanisms, and (b) the detailed structure of the movements themselves. The latter may serve to account for the effect of movement on the visual input itself. Indeed, the largest source of visual motion on the retina is generated by self-motion, i.e., the motion of the head and eyes relative to the visual scene, rather than by objects in the visual scene. Incorporating movement information can enable an organism to more accurately reconstruct the visual scene by correcting for self-motion, as well as by enabling the extraction of additional features such as depth through motion parallax (Leopold & Park 2020, Parker et al. 2020). Consistent with these roles, neurons in rat V1 encode a representation of the three-dimensional rotation of the head in space (Guitchounts et al. 2020). Likewise, the speed and direction of movement play predictive roles in processing visual inputs in V1, resulting in mismatch signals when the visual input does not correspond to that expected during locomotion (Leinweber et al. 2017). These predictive signals arise from anterior cingulate and secondary motor cortex, and their impact on V1 is retinotopically specific. Contextual signals, including locomotion and visuo-motor mismatch, are delivered to V1 from pulvinar as well (Roth et al. 2016).

Vestibular signals are another important source of information about head movements relative to the visual scene, and they strongly modulate V1 activity across all layers (Bouvier et al. 2020, Vélez-Fort et al. 2018) (Figure 8b). Furthermore, L6 excitatory neurons sum vestibular activity with motion of the visual scene to create a representation that integrates internal and external motion signals (Vélez-Fort et al. 2018). The impact of vestibular signals on V1 neurons across layers switches polarity depending on ambient luminance (Bouvier et al. 2020), suppressing activity in the dark and increasing activity in the light. Notably, this suppression in darkness is dependent on SOM inhibitory neurons, again implicating this neuron type in subtractively regulating V1 activity (Wilson et al. 2012) depending on context. Although some aspects of these movement and head orientation signals can be probed in head-fixed conditions as described here, fully elucidating their role in vision is likely to require studies under freely moving conditions. This is now facilitated by methods to measure head and eye movements, together with neural activity, in naturally behaving mice (Meyer et al. 2018, Michaiel et al. 2020) (Figure 8c).

Additional Behavioral Context Representations

Multiple other aspects of behavioral context are also represented in mouse V1. These include responses to the timing of an anticipated reward (Shuler & Bear 2006), increased response to task-relevant visual stimuli (Poort et al. 2015), responses to other sensory modalities (Ibrahim et al. 2016, Iurilli et al. 2012), and firing in specific locations of a virtual environment independent of the specific visual input, similar to hippocampal place cells (Fiser et al. 2016, Saleem et al. 2018). The specific source of many of these signals and the circuit mechanisms that integrate them into V1 processing are beginning to be identified (Makino & Komiyama 2015, Roth et al. 2016, Zhang et al. 2014). Furthermore, this diversity suggests that their role extends beyond improving the representation of the visual scene (as in compensating for self-motion) to include, for example, amplifying specific features that are relevant for ongoing behavior.

THE RESULTS OF CORTICAL COMPUTATION: V1 OUTPUTS AND BEHAVIOR

The computations performed in V1 only have an impact to the extent their results get conveyed to other brain regions. Studying the outputs from V1 can therefore provide key insight into its essential functions. V1 neurons project to several cortical and subcortical areas. What visual features are encoded in different output cells of V1, and how do these relate to their downstream targets and to the behavior these targets mediate?

V1 Output to Higher Visual Areas

In primates, cortical visual processing is organized in a parallel, hierarchical structure: V1 projects to a series of higher visual cortical areas that respond successively to more complex visual stimuli such as objects in inferior temporal cortex (Nassi & Callaway 2009). Similarly, mouse V1 projects to approximately ten retinotopically organized HVAs (Wang & Burkhalter 2007). Although these HVAs are not arranged in as clear a hierarchy as they are in primates (Nassi & Callaway 2009, Harris et al. 2019), they show some degree of specialization for certain properties of visual stimuli (Andermann et al. 2011, Glickfeld & Olsen 2017, Juavinett & Callaway 2015, La Chioma et al. 2019, Marshel et al. 2011, Murgas et al. 2020, Roth et al. 2012, Sit & Goard 2020). Projections from V1 are often schematized as dedicated pathways, yet among all V1 neurons projecting to HVAs, only 25% target a single HVA, while 60% target two to three HVAs, and 15% target four or more (Han et al. 2018). Still, for neurons targeting multiple HVAs, certain motifs of shared connectivity are more frequent than others. Thus, given that information from a given V1 neuron is not unspecifically broadcast across all HVAs, we can address how distinct computations performed by distinct V1 neurons contribute to the distinct response properties of their target areas.

Two HVAs, one lateral and the other medial relative to V1, called AL (anterolateral) and PM (posteromedial), respectively, represent a good example of this approach. The populations of V1 neurons that project to AL versus PM are largely distinct, and, interestingly, neurons within each population are connected to each other but not to the other population (Kim et al. 2018). Furthermore, these two populations differ in their transcriptional profile (Kim et al. 2020). Importantly, the response properties of AL- and PM-projecting V1 neurons are also different, with neurons projecting to AL preferring high speeds and neurons projecting to PM preferring lower-speed visual stimuli (Glickfeld et al. 2013a) (Figure 9a). Because the target areas AL and PM as a whole also preferentially respond to high- and low-speed visual stimuli, respectively, the unique tuning properties of these areas may, at least in part, be inherited from distinct populations of V1 output neurons (Glickfeld et al. 2013a). A similar conclusion was made regarding the rostrolateral HVA called RL, which inherits its response to directional visual stimuli from a specific set of L2/3 V1 neurons who, themselves, inherit this property from directionally selective retinal ganglion cells (Rasmussen et al. 2020). Thus, for some response properties, pathway specificity may be maintained all the way from the retina to HVAs.

Figure 9.

Selectivity and impact of V1 outputs. (a) Two populations of V1 neurons (red, blue) projecting to different higher visual areas, AL and PM, have different speed tuning. Panel a adapted from Glickfeld et al. (2013a). (b) L6 neurons projecting to dLGN (CThal) have sparse and highly selective responses to orientation relative to L6 neurons that project within cortex (CC). Panel b adapted from Vélez-Fort et al. (2014). (c) L5 neurons that project to SC boost the amplitude of response to a looming visual stimulus relative to the retinal input alone, suggesting a role in modulating innate behavior. Panel c adapted from Zhao et al. (2014). Abbreviations: AL, anterolateral; CC, corticocortical; CThal, corticothalamic; dLGN, dorsal lateral geniculate nucleus; L, layer; PM, posteromedial; SC, superior colliculus; V1, primary visual cortex.

These outputs to HVAs are reciprocal. Neurons in V1 receive feedback from the areas to which they project to form loops with cellular, laminar, and functional specificity (D’Souza et al. 2016, Marques et al. 2018, Young et al. 2021). These feedback projections modulate responses in V1, including amplification of tuning properties and mediating contextual modulations (Vangeneugden et al. 2019, Huh et al. 2018, Keller et al. 2020b).

V1 Output to Subcortical Areas and Impact on Behavior

Despite the emphasis on V1 output to the cortical hierarchy, as illustrated by the iconic diagram of Felleman & Van Essen (1991), a large fraction of V1 output actually goes to subcortical areas as well. This includes projections from L6 neurons to the dLGN and the reticular thalamic nucleus (TRN), as well as projections from L5 neurons to higher-order thalamus, which may serve as an alternate pathway to the HVAs (Guillery & Sherman 2002), and to structures associated with behavioral output such as superior colliculus (SC), striatum, accessory optic system (AOS), and amygdala. Therefore, even the low-level visual representations of V1 may directly impact behavior.

L6 neurons projecting back to the dLGN and the TRN form a functionally distinct population, with very sparse firing and high stimulus selectivity, while cortically projecting L6 neurons are more broadly responsive (Vélez-Fort et al. 2014) (Figure 9b). Intriguingly, these thalamic-projecting neurons also receive most of their input not from the local cortical circuit but from higher cortical areas. The exact role of this feedback projection to the primary thalamic nucleus from which V1 receives ascending visual information still needs to be elucidated. Recent studies show that it contributes to the sharpening of the RF of dLGN through surround suppression (Born et al. 2021), most likely through the strong disynaptic inhibition mediated by TRN neurons (Olsen et al. 2012).

Little is known regarding what information is conveyed from V1 to the different subcortical targets and how this compares to projections to cortical areas. In L5, neurons projecting to SC have, on average, broader tuning and higher contrast sensitivity to drifting gratings than do neurons projecting to the striatum, while cortically projecting neurons are intermediate between these two (Lur et al. 2016). This suggests that outputs to SC are specialized for detection, while those to striatum are specialized for discrimination. However, these differences represent more of a bias than a true segregation. Most likely, we have not yet determined the stimuli or contexts that best differentiate the response properties between these populations of L5 neurons.

SC thus receives two sources of visual information: one directly ascending from the retina and the other descending from cortex. Projections from V1 strongly impact visual responses in the SC by increasing their gain (Zhao et al. 2014) (Figure 9c). As a consequence, this projection potentiates SC-mediated behavior, like the animal freezing in response to a flashed stimulus (Liang et al. 2015). The impact that V1 exerts on the SC may also explain the impairment in performance of basic tasks, such as detecting the presence of a stimulus or detecting the change in orientation of a grating, upon V1 inactivation or lesion (Glickfeld et al. 2013b, Prusky & Douglas 2004, Ruediger & Scanziani 2020). This is especially the case for low-contrast or high–spatial frequency stimuli, consistent with the modulatory impact of V1 on SC.

Through its corticofugal projections to the AOS in the midbrain, a structure that generates the optokinetic reflex (OKR) and that, like the SC, receives direct retinal input, V1 can directly impact image stabilizing reflexes (Liu et al. 2016). Indeed, V1 not only modulates the gain of the OKR but also contributes to its plasticity (Liu et al. 2016). Thus, through the projections to both SC and AOS, V1 provides a signal that boosts the ongoing function of a retinorecipient subcortical region, allowing for fine-tuning and top-down control of innate behaviors.

The functional role of the corticofugal pathway out of V1 to the striatum (Khibnik et al. 2014) is only beginning to be elucidated. V1 provides a major drive for visual responses in this structure (Peters et al. 2021), and specific lesions of this corticofugal pathway show that it contributes to the learning speed of simple detection tasks (Ruediger & Scanziani 2020).

In addition to modulating the function of subcortical structures and the behaviors encoded in those structures, V1 also likely provides instructive information for behavior. This is the case when the animal is trained to discriminate between stimuli rather than to simply detect the presence of a stimulus or a sudden change in the visual environment. The ability of a mouse to, for example, discriminate between dots moving in different directions (Marques et al. 2018) or gratings of different orientations (Poort et al. 2015, Resulaj et al. 2018, Wekselblatt et al. 2016) is brought down to chance levels upon V1 inactivation. Interestingly, for relatively simple discriminations, like the ability to discriminate between gratings with large differences in orientation, even brief time windows of activity in V1, during which most neurons fire either no or one action potential, are sufficient for the animal to perform well above chance (Resulaj et al. 2018). In addition, activation of appropriate ensembles of neurons in V1, using holographic optogenetic stimulation, is sufficient to elicit appropriate behavioral responses in such orientation tasks (Carrillo-Reid et al. 2019, Marshel et al. 2019). Some of these behavioral roles may be mediated through V1’s projections to higher visual areas in the cortical visual pathway. However, at least in one behavioral paradigm, a specific subcortical output pathway from V1 providing instructive information to downstream targets has been identified: L5 neurons projecting to the pons, but not striatum, are necessary for eyeblink conditioning (Tang & Higley 2020). In contrast to the SC- and AOS-dependent behaviors described above, V1 is the primary source of visual input for eyeblink conditioning, and the L5 neurons encode both the behavior output and sensory information. Thus, V1 subcortical output can actually directly mediate, rather than just modulate, visually driven behavior.

The most essential role of V1 in behavior may be to enable learning and flexible processing of visual information based on experience. Training on visual behaviors induces changes in responses of V1 neurons, including stimulus selectivity (Poort et al. 2015), and association with reward and location (Pakan et al. 2018). Furthermore, training in a reaching task increases glutamate receptor levels in V1 (Roth et al. 2020), particularly when the training was performed in the light and, therefore, was presumably visually guided. These changes may refine the representation of the visual scene to better discriminate relevant stimuli or represent the engram of learned visual stimuli. Alternately, these changes may not encode visual information, per se, but the association of visual stimuli with the context that evoked them.

OUTLOOK

The techniques available for use in mouse have enabled cellular and molecular approaches to be applied to system-level phenomena. This research has been extremely successful in answering long-standing questions and has led to the discovery of general principles, such as how diverse inputs are combined to generate new representations, canonical excitatory and inhibitory circuit motifs, and how information is selectively routed to downstream targets.

However, there are still major gaps in our understanding of cortical function as highlighted by the fact that many structural properties of cortex remain unaccounted for relative to the function they may implement: Why do cortical neurons receive such a large number of synapses? Why does cortex need multiple layers and so many cell types? Why are cortical areas that perform apparently different functions such as sensation, motor actions, or executive functions structurally so similar? Or are they? In these respects, it seems that the potential power of the mouse in addressing the circuit basis of cortical computation has not yet been fulfilled.

Beyond technical limitations, there are also conceptual reasons why progress on the big picture may feel limited. Is there a canonical computation performed by cortex, such as predictive coding or Bayesian inference? We still lack a compelling framework regarding cortical function that can guide future research. Without guiding principles, even the most accurate descriptions of cortical properties, while captivating to a biologist’s mind, lack explanatory power. As stated by Barlow, “A wing would be a most mystifying structure if one did not know that birds flew…. Without knowing this, and without understanding something of the principles of flight, a more detailed examination of the wing itself would probably be unrewarding” (Barlow 1961, p. 217).

The discoveries of Hubel & Wiesel (1962), demonstrating that V1 extracts specific features of the visual scene, e.g., the orientation of edges, have led to the influential hypothesis that subsequent stages iterate this process, leading to high-level representations of ethologically relevant stimuli, e.g., faces. However, visual cortex function is likely more than extracting specific features of the visual world. First, as mentioned above, simple circuits of neurons can extract these features, leaving the role of many elements of the cortical network unaccounted for. Second, neurons receive a large number of inputs that are tuned for features that are not matched to their preferred features (Cossell et al. 2015), and it is not clear how these would contribute in a simple feature extraction framework. Critically, even our knowledge of the specific visual features captured by a given neuron does not predict its responses to a range of visual scenes (de Vries et al. 2020). Finally, repeated presentation of the same visual scene leads to responses that vary from trial to trial due to unaccounted variables beyond the visual stimulus (Busse et al. 2017, Engel & Steinmetz 2019).

The explanatory gap between the feature extraction framework and the organization of cortical circuits may result from the fact that we are only probing a narrow region of the parameter space within which cortex generally operates. A key feature that is missing in simplified stimulus paradigms is context, within both the visual scene and the animal’s interaction with the scene. As discussed above, cortical circuits not only compute the classical RF but also embed that response within the ongoing visual and behavioral context. In this view, the classic RF is a low-level representation of a higher-dimensional response to a range of image structures and behavioral associations. Two neurons may respond similarly when probed by gratings (their classical RF) but dramatically differently when probed by images with that same feature in different contexts, like visual scenes in a natural environment. Furthermore, even passive viewing of natural stimuli lacks much of the richness of what visual cortex experiences when an animal moves through its environment, including motion, vestibular and other sensory cues, and predictive signals based on the animal’s experience and intentions. Current approaches in mouse to study cortical processing in rich visual environments, with both virtual reality and natural behavioral paradigms such as hunting or escape behaviors (Hoy et al. 2016, Vale et al. 2017), while maintaining a detailed characterization of behavioral variables (Meyer et al. 2018, Michaiel et al. 2020) (Figure 8c), may allow us to address these issues and lead to a broader understanding of cortical function.

An alternative way to understand what V1 contributes to brain function may be to determine its impact on subcortical targets. Many vertebrates go on with their lives without much cortex to speak of and have been successful at it for eons, yet the brain of mammals dedicates a lot of space to this structure. What does cortex add to the subcortical lizard brain? The cortex, through its exquisite ability to extract specific features of the sensory world, to learn and to predict, may modulate innate behaviors according to prevailing conditions and experience, as discussed above in the section titled V1 Output to Subcortical Areas and Impact on Behavior. This would expand the behavioral repertoire encoded in subcortical structures, a possibility already recognized more than a century ago by Edinger (1908, pp. 453–54): “In mammals we meet a brain which has so large a neoencephalon (neocortex) that we may well expect a subordination of reflexes and instincts to associative and intelligent actions.” Techniques that are available for use in the mouse to study multiple brain areas, and the cell type–specific connections between them, are providing us with the tools to test this hypothesis directly.

CONCLUSION

A tremendous benefit of studying mouse visual cortex has been a shift in how we approach cortical processing, from spike trains and tuning curves in isolation to the cell types and interconnexions raised in Hubel & Wiesel’s original paper quoted in the Introduction. At the same time, there has been a broadening in the scope of study, from examining V1 and the cortical hierarchy in isolation to exploring connectivity and functional interactions with other brain regions, and from studying low-level visual features in isolation to incorporating a range of contextual factors. As a result of these studies, we have gained detailed insight into how cortical circuits mediate a range of visual computations, but this has also revealed the gaps in our knowledge and opened new directions of inquiry. We are now looking forward to the next decade where the mechanistic understanding of computations performed in V1 can be extended into an integrated view of how cortex functions.

FUTURE ISSUES.

1. What are the similarities and differences in anatomical and functional organization between primary visual cortex (V1) and other cortical areas, as well as between mouse V1 and primate V1? Are there fundamental differences in the computations being performed or primarily specializations within a shared architecture?

2. Why are there so many types of excitatory and inhibitory neurons, even within the broad categories described here? Does each have its own role, such as in the wide diversity of retinal ganglion cells? Or do subtypes represent variations on a theme, such as a violin and a viola, which are functionally similar but tuned to a different parameter range?

3. What is the role of layers? Do layers represent multiple stages of processing, or do they primarily serve to segregate different input and output pathways?

4. Does thalamic input only ignite layer 4? Or is the input to other layers also sufficient to drive activity that can propagate through the circuit?

5. What does recurrent amplification do? Can it be tuned based on context?

6. To what extent do visual context and behavioral (nonvisual) context share common circuit mechanisms?

7. Do nonvisual signals change the tuning of neurons for visual features (e.g., preferred orientation) or simply act as multiplicative or additive factors to dial the response to these features up or down?

8. How separate are the circuits within V1 that eventually lead to distinct output pathways? Are there dedicated pathways through the cortical circuit, or do output neurons selectively integrate responses from a pool of multipurpose representations?

9. V1 receives many inputs beyond the dorsal lateral geniculate nucleus, including higher-order visual thalamus, higher visual cortical areas, and nonvisual cortical areas such as other sensory and associational cortices. How do these inputs contribute to the computations performed by V1?