Multiple components of surround modulation in primary visual cortex: multiple neural circuits with multiple functions?

Abstract

The responses of neurons in primary visual cortex (V1) to stimulation of their receptive field (RF) are modulated by stimuli in the RF surround. This modulation is suppressive when the stimuli in the RF and surround are of similar orientation, but less suppressive or facilitatory when they are cross-oriented. Similarly, in human vision surround stimuli selectively suppress the perceived contrast of a central stimulus. Although the properties of surround modulation have been thoroughly characterized in many species, cortical areas and sensory modalities, its role in perception remains unknown. Here we argue that surround modulation in V1 consists of multiple components having different spatio-temporal and tuning properties, generated by different neural circuits and serving different visual functions. One component arises from LGN afferents, is fast, untuned for orientation, and spatially restricted to the surround region nearest to the RF (the near-surround); its function is to normalize V1 cell responses to local contrast. Intra-V1 horizontal connections contribute a slower, narrowly orientation-tuned component to near-surround modulation, whose function is to increase the coding efficiency of natural images in manner that leads to the extraction of object boundaries. The third component is generated by topdown feedback connections to V1, is fast, broadly orientation-tuned, and extends into the far-surround; its function is to enhance the salience of behaviorally relevant visual features. Far- and near-surround modulation, thus, act as parallel mechanisms: the former quickly detects and guides saccades/attention to salient visual scene locations, the latter segments object boundaries in the scene.

1. Introduction

Surround modulation is the ability of neurons in the visual cortex to change their response to local visual features within their receptive fields (RFs) depending on visual context, i.e. the stimuli simultaneously present in the RF surround. This property, initially attributed by Hubel and Wiesel (1965) to a special class of cells in the primary visual cortex (V1) of cats (which they termed “hypercomplex”), has now been described for most cells in V1 of many species, ranging from mouse (Van den Bergh et al., 2010) to cat (Blakemore & Tobin, 1972; Gilbert, 1977; Maffei & Fiorentini, 1976; Nelson & Frost, 1978; Sengpiel, Sen, & Blakemore, 1997; Walker, Ohzawa, & Freeman, 2000) and monkey (Cavanaugh, Bair, & Movshon, 2002a; Knierim & Van Essen, 1992; Sceniak, Hawken, & Shapley, 2001; Shushruth et al., 2009). Surround modulation and analogous phenomena have also been described throughout the visual system (e.g. Albright & Stoner, 2002; Allman, Miezin, & Mc Guinness, 1985; Born & Bradley, 2005; Desimone & Schein, 1987; Pollen et al., 2002) and across different modalities, including the auditory (Sutter et al., 1999), somatosensory (Sachdev, Krause, & Mazer, 2012; Vega-Bermudez & Johnson, 1999) and olfactory (Olsen & Wilson, 2008) systems. In human visual perception, many studies have demonstrated that spatial context alters the perception of a visual target (Cannon & Fullenkamp, 1991; Chubb, Sperling, & Solomon, 1989; Ejima & Takahashi, 1985; Meese & Hess, 2004; Meese et al., 2007; Nurminen et al., 2009; Olzak & Laurinen, 1999; Snowden & Hammett, 1998). The conservation across such a wide range of species, cortical areas and sensory modalities suggests that surround modulation plays a fundamental role in sensory processing. However, despite a plethora of research that has provided a thorough characterization of the parameter space of surround modulation, its functional role remains a mystery.

In this article we focus on surround modulation in V1. We present our view that surround modulation consists of multiple components that arise from different anatomical circuits and have different spatio-temporal and stimulus tuning properties, and therefore should not be considered as a single entity with a single functional role. We will focus on each of the components separately, and put forward our hypotheses concerning their distinct functional roles in natural vision.

2. Basic properties of surround modulation

The properties of surround modulation in V1 have been quantitatively characterized in many studies, typically using a circular grating patch of increasing radius (Fig. 1A), or a center grating patch confined to the neuron's RF surrounded by an annular grating (Fig. 1B), and varying systematically the grating/s parameters (reviewed in: Angelucci & Shushruth, 2014). There is general agreement among these studies that surround modulation in V1 shows five basic properties. 1. It is predominantly suppressive (stimulation of the surround reduces the neuron's spiking response to an optimal stimulus in its RF – Fig. 1), especially when the center and surround stimuli are of high contrast, and the surround is stimulated with large gratings (Cavanaugh, Bair, & Movshon, 2002a; DeAngelis, Freeman, & Ohzawa, 1994; Levitt & Lund, 1997, 2002; Sceniak, Hawken, & Shapley, 2001; Sengpiel, Sen, & Blakemore, 1997; Walker, Ohzawa, & Freeman, 2000). 2. It is orientation selective, i.e. the strongest suppression is observed when the stimuli in the RF and surround are of the same orientation (DeAngelis, Freeman, & Ohzawa, 1994; Li & Li, 1994; Sengpiel, Sen, & Blakemore, 1997; Walker, Ohzawa, & Freeman, 1999), even when the orientation of the stimulus inside the RF is not the one preferred by the neuron (Cavanaugh, Bair, & Movshon, 2002b; Shushruth et al., 2012). The suppression can turn into facilitation when the stimuli in the RF and surround are cross-oriented, especially when the RF is stimulated with suboptimal orientations for the cell (Shushruth et al., 2012; Sillito et al., 1995). Psychophysical experiments have reported highly similar orientation tuning of surround suppression in human vision as in macaque V1 cells (Cannon & Fullenkamp, 1991; Petrov, Carandini, & McKee, 2005; Solomon, Sperling, & Chubb, 1993), suggesting similar underlying mechanisms for surround modulation in the two species. 3. Surround modulation is also tuned for spatial frequency, so that stimuli of similar spatial frequency in the RF and surround produce the strongest suppressive effects. In human vision the spatial frequency of surround modulation shows band-pass tuning (Chubb, Sperling, & Solomon, 1989), while in V1 cells the tuning is low pass (Webb et al., 2005). 4. Surround modulation is spatially extensive (modulatory effects can be evoked from surround regions up to 12.5° away from the RF center) (Cavanaugh, Bair, & Movshon, 2002a; Levitt & Lund, 2002; Sceniak, Hawken, & Shapley, 2001; Shushruth et al., 2009), and the suppression is strong, but it decreases in strength with increasing distance from the RF center or from a target grating (e.g. Fig. 1B) (Cannon & Fullenkamp, 1991; Nurminen, Peromaa, & Laurinen, 2010; Shushruth et al., 2009). 5. Finally, surround suppression is fast, being delayed on average by as fast as 9 ms relative to the onset of the RF response, in a manner that is nearly independent of the distance of the surround stimulus from the RF (Bair, Cavanaugh, & Movshon, 2003).

Figure 1. Surround suppression in V1 cells.

(A) Surround suppression probed using grating patches of increasing radius centered on the neuron RF. Black and gray curves: responses of an example V1 cell to a grating of high and low contrast, respectively. As the radius of the grating patch increases, the cell's response increases up to a peak (thick arrows), corresponding to the RF size measured at high contrast (RF_high) and at low contrast (RF_low), respectively; the cell's response is then suppressed as the grating extends beyond the RF, into the surround. We call the region between the RF_high and the RF_low “near” surround (gray shaded column in A, and light gray ring in the inset above A); stimulation of this surround region can cause facilitation or suppression, depending on stimulus contrast. We call the region beyond the RF_low “far” surround. Arrowheads: surround radius measured at high (black) and low (gray) stimulus contrast. Top inset: schematics of the different components of the RF and surround of a V1 cell, with the white area indicating the RF, and the gray areas the surround, the latter consisting of a near (light gray) and a far (dark gray) region. Right inset indicates the stimulus paradigm, i.e. a grating centered on the cell's RF, which is systematically grown (arrows) in radius. (B) Surround suppression probed using a center-grating patch confined to the cell's RF and an annular grating in the surround whose inner radius is systematically grown (arrows) towards the RF (right inset). The black curve indicates the cell's response to center and surround gratings of high contrast: as the inner radius of the surround annulus is decreased (read the x axis from right to left), the cell's response is increasingly suppressed.

3. The multiple components of surround modulation: multiple neural circuits with multiple functions

Multiple lines of evidence suggest that the surround of V1 neurons consists of “near” and “far” components (Fig. 1) generated by different anatomical circuits. These circuits involve feedforward projections from the lateral geniculate nucleus (LGN), long-range intra-V1 horizontal connections, and extra-striate feedback connections to V1. We have previously proposed that feedforward and horizontal projections contribute to the near surround (Angelucci et al., 2002; Angelucci & Sainsbury, 2006) (green and red arrows in Fig. 2); this is the surround region lying nearest to the RF, that is spatially coextensive with the peak of a V1 cell size tuning curve measured at low stimulus contrast (RF_low in Fig. 1A). The far surround lies beyond the near surround and, we have proposed that it is generated by extra-striate feedback connections to V1 (blue arrows in Fig. 2) (Angelucci & Bressloff, 2006; Angelucci et al., 2002).

Figure 2. The multiple components of surround modulation, and their hypothetical underlying circuits and functions.

Colored arrows indicate the different anatomical circuits (according to legend) that are hypothesized to generate the RF (white area), and the different surround components (gray rings) of V1 neurons. The hypothesized function for each connection type and surround component is indicated on the left in color code.

3.1. Feedforward contribution to near-surround modulation: divisive contrast normalization

V1 receives driving feedforward inputs from the LGN, which are thought to contribute primarily to the size and tuning properties of V1 cell's RFs (Angelucci & Sainsbury, 2006; Hubel & Wiesel, 1962; Reid & Usrey, 2004). The most compelling evidence that surround modulation in V1 involves a feedforward component is that both LGN neurons (Alitto & Usrey, 2008; Bonin, Mante, & Carandini, 2005; Levick, Cleland, & Dubin, 1972; Sceniak, Chatterjee, & Callaway, 2006; Solomon, White, & Martin, 2002) and retinal ganglion cells (Solomon, Lee, & Sun, 2006) exhibit surround modulation. Thus, large stimuli induce surround suppression in the LGN, resulting in withdrawal of feedforward excitation to V1. Consistent with a feedforward component to V1 surround modulation, blockade of intra-V1 inhibition does not completely abolish near-surround suppression in V1 cells (Ozeki et al., 2004). Moreover, the spatio-temporal tuning of V1 surround modulation exceeds the range which drives most cortical neurons (Webb et al., 2005), resembling the tuning of LGN cells, suggesting that at least part of the modulation in V1 originates subcortically. This feedforward component to surround modulation in V1 is fast (often coincident with the latency of RF activation) and untuned for orientation (Henry et al., 2013), just as surround suppression in the LGN, which is untuned in primates (Solomon, White, & Martin, 2002; Webb et al., 2002), or less orientation-tuned than V1 suppression in cat (Bonin, Mante, & Carandini, 2005; Ozeki et al., 2009). Furthermore, this feedforward component is spatially confined to the near surround of V1 cells (Fig. 2), due to the limited anatomical spread of geniculocortical connections (Angelucci & Sainsbury, 2006), and the small size of LGN suppressive surround fields (Alitto & Usrey, 2008; Sceniak, Chatterjee, & Callaway, 2006). This feedforward component is insufficient to account for all surround modulation in V1, as the latter is orientation tuned and spatially more extensive than surround suppression in the LGN (see below). What could be the role of the broadly tuned, feedforward component of surround modulation?

In his pioneering study Heeger (1992) suggested that the response of each neuron in cat V1 is divisively normalized by a broadband local contrast signal (see also Bonds, 1989), reflected in the activity of a pool of neighboring neurons. We propose that it is the feedforward component of surround modulation that normalizes V1 responses with respect to stimulus contrast. Contrast normalization is a computation that allows V1 neurons to handle the wide range of contrasts existing in natural scenes, despite the neurons' limited dynamic range (Carandini & Heeger, 2012). For example, divisive normalization can account for the finding that despite the saturation of V1 responses at high contrast, the response ratio of neurons with different orientation preferences is independent of contrast (Albrecht & Hamilton, 1982). This is a desirable property, because it allows for contrast-invariant recovery of stimulus orientation from the neuronal population response (Heeger, 1992).

In summary, our current opinion is that the untuned component of surround modulation that originates in subcortical brain structures serves to normalize V1 responses with respect to contrast. However, we do not wish to imply that all untuned suppression in V1 arises from subcortical structures. There is both experimental and theoretical evidence for a cortical contribution to untuned suppression. For example, V1 contains many broadly orientation and spatial frequency tuned cells (Ringach, Shapley, & Hawken, 2002; Xing et al., 2004) which could contribute to untuned surround suppression; this suppression may or may not be related to the untuned suppression (Xing et al., 2005) or normalization signal (Smith, Bair, & Movshon, 2006) arising from within the RF of V1 cells. Moreover, in some instances, the cortical untuned suppression may serve different functional roles from that of feedforward untuned suppression. For example, our previous modeling studies have shown that the interaction of long-range tuned suppression (generated by horizontal connections to inhibitory neurons) with local untuned suppression (generated by withdrawal of local cortical recurrent excitation) causes surround suppression to become tuned to the stimulus orientation presented to the cell's RF, rather than to the orientation preferred by the cell (Shushruth et al., 2012).

3.2. The contribution of horizontal connections to near-surround modulation: efficient coding of natural images leading to extraction of object boundaries

In V1 of many mammalian species, such as monkey, cat and tree shrew, horizontal connections are millimeters-long axonal projections (Gilbert & Wiesel, 1983; Rockland & Lund, 1982, 1983) prominent in layers 2/3, arising from excitatory neurons, targeting both excitatory and inhibitory neurons, and linking V1 neurons with similar orientation preference (Bosking et al., 1997; Malach et al., 1993; Schmidt et al., 1997; Sincich & Blasdel, 2001). This connectivity pattern is well suited to generate the orientation tuning of surround modulation, and indeed, it was originally thought that horizontal connections are the sole anatomical substrate for surround modulation in V1 (Gilbert et al., 1996). Current evidence, however, suggests that horizontal connections generate the orientation-tuned component of near-surround modulation (Fig. 2), but cannot account for the spatio-temporal properties of far-surround modulation (see Section 3.3). Specifically, in macaque and cat V1, surround modulation is sharply orientation tuned when measured with stimuli that are confined to the spatial extent of monosynaptic horizontal connections (i.e. to the near surround), while it is broadly tuned for orientation when measured with stimuli that are placed in the far surround (Hashemi-Nezhad & Lyon, 2011; Shushruth et al., 2013). Notably, there are also laminar differences in the orientation tuning of near-surround suppression, the latter being more sharply tuned in the supra- than in the infra-granular layers of V1 (Shushruth et al., 2013). Given that very little is known about the functional organization of horizontal connections in infragranular layers, it is unclear how the broader tuning of near-surround suppression is generated in these layers, but it may result from weaker functional specificity of horizontal connections in lower layers (Li et al., 2003; Lund, Angelucci, & Bressloff, 2003). In summary, current data suggest that the orientation-tuned component of surround modulation involves horizontal connections in layers 2/3 of V1. What could be the functional role of this orientation-tuned, horizontal connection-mediated near-surround modulation?

In recent years, there have been many studies investigating the statistics of natural images, with the hope that this knowledge may lead to a better understanding of the functional roles that the receptive field properties of neurons at different stages of visual processing play in natural vision (for a review see: Hyvärinen, Hurri, & Hoyer, 2009). To understand the relationship between natural image statistics and surround modulation, a good starting point may be to examine the joint statistics of spatially displaced image regions, because surround modulation, by definition, relates to the interaction between spatially displaced image locations in the visual system. Geisler et al. (2001) estimated the joint natural image statistics by asking human observers to judge whether two edges extracted from different locations in a natural scene belonged to the same physical contour or to different contours. In line with the Gestalt-law of good continuation (Wertheimer, 1958), the observers were more likely to judge that two nearby edges belonged to the same contour if their orientation was similar, whereas more distant edges of the same contour could have a wider range of orientations. We have recently shown (Shushruth et al., 2013) that the orientation tuning of surround modulation follows a pattern similar to that of the joint statistics of spatially displaced edges. Specifically, our study showed that surround modulation was sharply tuned for orientation when the distance between the center and surround stimuli was small, while for distant stimuli the tuning was broad. Similarly, nearby parts of natural contours tend to be of the same orientation, whereas the more distant parts come from a wider distribution of orientations (Geisler et al., 2001). Thus, nearby edges of similar orientation that occur with higher probability in natural images would seem to evoke stronger suppression in V1 cells. Conversely, natural contours occurring with lower probability, such as those in which nearby edges have orthogonal orientation, would evoke weaker suppression or even facilitation of V1 cell responses (Shushruth et al., 2012; Sillito et al., 1995). Thus, the visual system seems to devote less spikes to contours that occur with high probability, and more spikes to contours that occur with low probability. In line with previous findings (Vinje & Gallant, 2000), this relationship suggests that surround modulation increases the sparseness of the neural code in V1 (Olshausen & Field, 2004). A sparse code implies that neurons respond to a more restricted set of stimuli, and therefore are more selective, a mechanism that has been suggested to increase the efficiency of information transmission about a visual stimulus (Vinje & Gallant, 2002).

Previous studies have shown that elongated contours and other global structures in natural images produce strong statistical dependencies between the responses of V1 RF-like linear filters, even when they are spatially non-overlapping (Schwartz & Simoncelli, 2001). This redundancy is statistically inefficient (Attneave, 1954) and, therefore, thought to be detrimental for visual processing (Barlow, 1961). Theoretical work by Schwartz & Simoncelli (2001) has led to the hypothesis that surround modulation serves to reduce the statistical dependencies between V1 neurons, thus increasing coding efficiency in the visual system. If this hypothesis is correct, then the tuning and distance-dependence of surround modulation should parallel those of the statistical dependencies of RF-like filters in response to natural images. However, we are not aware of any studies reporting such dependency measurements as a function of the spatial frequency difference and of the distance between the RFs. Moreover, while we (Shushruth et al., 2013) have shown that the orientation tuning of surround modulation qualitatively resembles natural contour statistics (Geisler et al., 2001), i.e. it is narrowly orientation tuned for nearby edges and broadly tuned for distant edges, it is difficult to directly relate these two measurements. This is because Geisler et al. (2001) analyzed the joint statistics of spatially displaced image elements only up to a distance of 1.25 degrees, which corresponds to our definition of near surround for a typical V1 cell; in contrast, our measurements of the orientation tuning of surround suppression were made up to distances >3 degrees (corresponding to the far surround of V1 cells). Moreover, we are not aware of any published measurements of natural image statistics that can be related to the spatial frequency tuning of surround modulation.

To address these questions, here we have measured the statistical dependencies between the responses of V1 RF-like filters to natural images as a function of orientation and spatial frequency difference and distance between the filters, and compared these measurements to surround suppression in V1 neurons and human perception. The dependencies were estimated as follows. First, a large number (10⁵) of natural image patches (80 × 80 pixels) were randomly sampled from 20 images randomly selected from the Van Hateren's natural image database (van Hateren & van der Schaaf, 1998). The sampling was done separately for each distance. The mean luminance of each patch was subtracted from each pixel in the patch. Second, the sampled image patches were projected to linear visual filters resembling V1 RFs (see below). Third, the dependencies between filter responses were quantified by computing the mutual information between their response distributions.

We used raised cosine filters i.e. sinusoidal gratings multiplied by a raised cosine window (Fig. 3A). These filters were used instead of standard Gabor filters, because their design allows for complete separation of the filters in the spatial domain, ensuring that the measured dependencies reflect image structure instead of filter overlap. Like Gabor filters (Olshausen & Field, 1996), the raised cosine filters produce sparse distributions in response to natural images (Fig. 3B). Intuitively, sparseness means that most images produce near-zero responses and few images produce strong responses, and therefore the neural response is highly informative about the stimulus. In addition to sparseness, the joint distributions of similar filters show a dependency between the mean response of one filter and the response variance of the other filter, as previously shown with Gabor filters by Schwartz & Simoncelli (2001) (Fig. 3C).

Figure 3. Properties of RF-like visual filters.

(A) A raised cosine filter. (B) The red curve shows the response distribution of a raised cosine filter to a large set of natural image patches. For comparison, the black curve shows a Gaussian distribution with the same mean and variance as the distribution shown in red. The red curve displays the characteristics of a sparse distribution, i.e. heavy tails and strong peak at the mean. (C) Joint response histogram of two nearby co-oriented RF-like visual filters to a large sample of natural image patches. The brightness is proportional to the number of occurrences of each response pair, with each column being separately normalized. For example, the pixel in the middle of the figure marks the number of times that both filters simultaneously produced zero response.

In the current analysis, the wavelength of the center sinusoid was always 9 pixels, the diameter of the window plateau was 9 pixels and the window weights were zero beyond 12 pixels diameter. The mean over the filter was subtracted, and the filter was divided by its norm. The two filters were otherwise identical, except that one filter (the “surround filter”) was spatially displaced from the other filter (the “center filter”) along the axis of filter orientation by 1.8, 3.6, 5.3 and 7.1 wavelengths. Assuming a preferred spatial frequency of 1 cycle/degree, the 1.8 wavelength corresponds roughly to the near-surround of the RF, and the 7.1 wavelength to the far-surround. The mutual information between the center and the surround filter responses was computed for different distances and orientation differences between the filters. The mutual information I(center; surround) between the filters was computed as,

$$I(\mathit{\text{center}};\mathit{\text{surround}})=H(\mathit{\text{center}})+H(\mathit{\text{surround}})-H(\mathit{\text{center}},\mathit{\text{surround}})$$

where H(center) and H(surround) are the marginal entropies of the center and surround filter response distributions, respectively, and H(center, surround) is their joint entropy. Mutual information approaches zero as the variables become more independent.

In a first dependency measurement, the orientation difference between center and surround filters was varied in equally spaced steps between 0 and 90°, and mutual information was measured as a function of center-surround orientation difference for different filter distances (Fig. 4A). Positive and negative orientation differences did not show systematic differences and were averaged in the analysis. The mutual information between the center and near-surround filter responses clearly depended on their orientation difference. The highest mutual information was observed when the filters were of the same orientation, and as the orientation difference was increased the mutual information decreased (Fig. 4A, black solid curve). This orientation dependence was similar to the orientation tuning of near-surround suppression that we have previously measured in both human vision and V1 cells (Shushruth et al., 2013) (left panel in Fig. 4B).

Figure 4. The orientation-tuning of the statistical dependencies between V1-like filters and of surround suppression both depend on spatial separation.

(A) Mutual information between the center and surround filters as a function of their orientation difference, measured for different filter separations (indicated by the different curves according to the legend). The width of the shaded regions associated with each curve is the ± s.e.m. of the respective curve. (B) Mean suppression index (this is a measure of surround suppression strength, with zero indicating no suppression and 1 complete suppression) as a function of the orientation difference of center and surround gratings for a population of macaque V1 cells (n=106) and for 5 human subjects. LEFT: the annular surround grating was confined to the near surround; RIGHT: the annular surround grating was placed in the far surround. Surround suppression in human subjects was determined psychophysically by measuring the perceived contrast of a center grating (in a contrast matching task) as a function of the difference in orientation between a center and a surround grating. The suppression index for human subjects indicates suppression of the perceived contrast of the center grating in the presence of a surround grating. Data in (B) are from Shushruth et al. (2013).

The mutual information between the center and the surround filter responses progressively decreased as the spatial separation between filters increased (Fig. 4A), while also becoming progressively less orientation tuned, so that the mutual information between the center and the far-surround filters was only weakly dependent on their orientation difference (Fig. 4A, gray solid curve). Similarly, in both V1 cells and human vision, far-surround suppression was clearly more broadly tuned for orientation (right panel in Fig. 4B) than near-surround suppression (left panel in Fig. 4B) (Shushruth et al., 2013).

We next measured the mutual information between the center and surround filters as their spatial frequency was varied, for different filter distances (same distances as for the orientation measurements), and compared these measurements to published measurements of the spatial frequency tuning of near-surround suppression in V1 cells and human vision (Fig. 5). We are not aware of previous studies that explored the relationship between natural image statistics and spatial frequency tuning of surround suppression. The spatial frequency of the center filter was kept constant, while the spatial frequency of the surround filter was varied from 0.5 to 4 times the spatial frequency of the center filter. For the smallest filter separation (1.8 wavelengths), the mutual information between the filter responses decreased as the spatial frequency of the surround was increased (Fig. 5A, solid black curve). Mutual information decreased with spatial separation of the filters and became slightly less tuned for spatial frequency at the largest separation examined (Fig. 5A solid gray curve), but remained well tuned. Near-surround modulation in macaque V1 cells (left panel in Fig. 5B) (Webb et al., 2005) and human vision (right panel in Fig. 5B) (Chubb, Sperling, & Solomon, 1989; Petrov, Carandini, & McKee, 2005) shows a similar dependency on spatial frequency as the mutual information, decreasing in strength as the spatial frequency of the surround increases. However, near-surround modulation in V1 cells shows low-pass spatial frequency tuning, like the mutual information between filter responses, but it shows band-pass tuning in human vision. Since no previous study has examined how the spatial frequency tuning of surround modulation varies with distance between the center and the surround gratings, we cannot compare the tuning of mutual information between the responses of filters at the largest spatial separation with that of far-surround modulation in V1 and human vision. A prediction of our hypothesis, therefore, is that the spatial frequency tuning of surround modulation should change slightly as the distance between the center and surround stimulus is increased.

Figure 5. The spatial frequency-tuning of the statistical dependencies between V1-like filters and of surround suppression both depend on spatial separation.

(A) Mutual information between the center and surround filters as a function of their spatial frequency (SF) ratio (SF of the surround filter normalized to the SF of the center filter), measured for different filter separations (indicated by the different curves according to legend). (B) LEFT: Mean suppression index as a function of the SF ratio of center and surround gratings for a population of macaque V1 cells. Center and surround gratings were at the orientation preferred by the cell, and the surround grating mainly stimulated the near surround (6-8° outer diameter). Data are from Webb et al. (2005). RIGHT: Mean suppression factor (ratio of the contrast detection thresholds of the center+surround to the center alone gratings) as a function of the SF ratio of center and surround gratings for 4 human subjects. Surround suppression was measured using a contrast detection task. Data are from Petrov, Carandini, & McKee (2005).

This analysis shows that the statistical dependencies in natural images are structured with respect to orientation and spatial frequency in a manner resembling the orientation and spatial frequency tuning of surround modulation in V1 neurons and human vision. While we recognize that this similarity, as presented here, is qualitative, it supports our hypothesis that surround modulation reflects the statistical dependencies in the natural visual environment. A number of previous studies have also related surround modulation and neuronal dependencies in V1 (Coen-Cagli, Dayan, & Schwartz, 2012; Schwartz & Simoncelli, 2001; Shushruth et al., 2013; Vanni & Rosenström, 2011; Vinje & Gallant, 2000). For example, Schwartz and Simoncelli (2001) showed that properties such as the orientation tuning of surround modulation and the contrast dependence of size tuning arise from a divisive normalization model in which the model parameters are optimized to maximize the statistical independencies between the responses of the model neurons. Vinje & Gallant (2000) recorded the responses of V1 cells to natural images, and found that the responses of separately recorded neuron pairs were less correlated when both the RF and surround were simultaneously stimulated, compared to when only the RF was stimulated. Similarly, a previous fMRI study (Vanni & Rosenström, 2011) showed that responses to complete human faces could be predicted by assuming that the responses to face parts were decorrelated in the brain. These studies suggest that, by decreasing dependencies between neuronal responses, surround modulation increases the amount of information that a neural population conveys about natural images. The intuition is that when information is not redundant across neurons, the population can convey more information about the stimulus.

In summary, our hypothesis is that the narrowly tuned component of surround modulation in V1 neurons (i.e. horizontal connection-dependent near-surround modulation) benefits vision in two major ways. First, it increases the specificity of V1 neuron responses to visual features by increasing the sparseness of the neural responses to natural images (Vinje & Gallant, 2000). Increased sparseness is a direct consequence of the tuning of surround modulation that causes less spikes to be devoted to the most frequently occurring natural contours, and a larger number of spikes to be elicited by rarely occurring contours. Second, by decreasing the statistical dependencies between the responses of V1 neurons, surround modulation increases the amount of information the neural population conveys about the image. In addition to actively reducing neuronal dependencies, our analysis shows that surround modulation weighs the neuronal population response so that the most independent neurons (i.e. the ones activated by stimuli of dissimilar orientation or spatial frequency) produce the largest and most reliable responses. Thus, the narrowly tuned component of near-surround modulation seems to aid the visual system in forming an efficient representation of natural scenes.

What important perceptual functions might emerge from the relationship of this narrowly tuned component of near-surround modulation with natural image statistics? Our analysis in this paper and the study of Coen-Cagli et al. (2012) suggest that surround suppression is strongest when the statistical dependencies between the stimulus in the RF and surround are strongest (for iso-oriented nearby contours; see Figs. 4-5). Interestingly, regions of natural images that belong to the same visual object show stronger statistical dependencies than regions arising from different objects (Coen-Cagli, Dayan, & Schwartz, 2012). Therefore clearly, the weakest suppression should be observed when the RF and surround are stimulated by different objects in an image. This suggests that the narrowly tuned component of near-surround modulation plays an important role in segmenting object boundaries, the initial stage of figure-ground segregation (Lamme, 1995), a process that is then carried forward by V2 neurons (Schmid, Purpura, & Victor, 2014; Zhou, Friedman, & von der Heydt, 2000).

3.3. The contribution of topdown feedback connections to far-surround modulation: visual salience

While horizontal connections are thought to generate near-surround modulation, current evidence suggests that they cannot account for the modulation arising from the far surround (reviewed in: Angelucci & Bressloff, 2006; Angelucci & Bullier, 2003). First, surround modulation has been observed up to 12.5° away from the RF center in parafoveal V1 of macaque monkeys (Ichida et al., 2007; Shushruth et al., 2009), and up to 7° in the human fovea (Nurminen, Peromaa, & Laurinen, 2010). These distances exceed the visuotopic reach of monosynaptic horizontal connections in V1 (Angelucci et al., 2002). While polysynaptic chains of horizontal connections could in principle underlie the far-surround effects, the conduction velocity of horizontal axons is too slow (Girard, Hupé, & Bullier, 2001) to account for the very fast onset of orientation-tuned surround suppression in V1 cells (Bair, Cavanaugh, & Movshon, 2003) and human vision (Kilpeläinen, Donner, & Laurinen, 2007). Moreover, if horizontal connections mediated far-surround modulation, the latency of suppression should be strongly distance dependent, but experiments show that it is nearly independent of the distance of the surround stimulus from the RF (Bair, Cavanaugh, & Movshon, 2003). Second, horizontal connections produce only subthreshold responses in their target neurons (Hirsch & Gilbert, 1991), and therefore cannot relay surround influences across extensive visual field regions in the absence of feedforward stimulation, something that has instead been observed experimentally (Ichida et al., 2007; Nurminen, Peromaa, & Laurinen, 2010; Shushruth et al., 2009).

In contrast, feedback projections from extra-striate visual cortex to V1 have many properties that are well suited to generate far-surround modulation (Angelucci & Bressloff, 2006; Angelucci & Bullier, 2003; Angelucci & Shushruth, 2014). Extrastriate feedback projections to V1 arise from excitatory neurons in layers 2/3 and 5/6 and target both excitatory and inhibitory neurons (Anderson & Martin, 2009) in layers 1, 2/3, 4B and 6. They are anatomically highly divergent and convergent, and their spatial spread is sufficiently extensive to convey far-surround modulation to V1 neurons (Angelucci et al., 2002) (Fig. 2). Feedback projections from areas V2, V3 and MT convey information to V1 from a visual field region corresponding on average to 5, 10 and 25 times, respectively, the RF size of V1 neurons. Moreover, feedback axons have high conduction velocities (Girard, Hupé, & Bullier, 2001), and therefore are well suited to mediate the very fast far-surround modulation (Bair, Cavanaugh, & Movshon, 2003; Kilpeläinen, Donner, & Laurinen, 2007).

The functional role of the feedback component of surround modulation is more difficult to envision than the roles of the feedforward and horizontal components, because the functional connectivity and specificity (or lack thereof) of feedback connections is unknown. For example, the organization of feedback connections from V2 to V1 with respect to the maps of stimulus orientation in these two areas is controversial. Stettler et al. (2002) showed an orientation non-specific arrangement of feedback in macaques, while Shmuel et al. (2005) reported orientation specific feedback connections in owl monkeys. However, our recent electrophysiological study (Shushruth et al., 2013) showed, that except in layer 4B [that provides direction-tuned inputs from V1 to MT (Movshon & Newsome, 1996) and receives feedback inputs from MT], surround modulation is only broadly orientation-tuned when the surround stimulus is located beyond the monosynaptic reach of horizontal connections, i.e. in the far surround (right panel in Fig. 4B). If indeed feedback connections generate far-surround modulation, this finding suggests that in layers other than 4B, where feedback may be orientation-specific, feedback is non-specific or only weakly biased for orientation. It is also possible that the feedback is not a single system but rather consists of multiple, laminar-dependent subsystems with different functions and feature specificities (Angelucci & Bressloff, 2006). Consistent with this hypothesis, Angelucci et al. (2002) found a “patchy” feedback termination pattern in V1 layers 2/3 and 4B, but not in 1 and 6 (Angelucci & Bressloff, 2006), of macaque V1, suggestive of feedback orientation specificity in some layers but not others.

Feedback has traditionally been associated with higher cognitive functions, such as attention, “read-out” or decoding of task-specific information from the responses of V1 cells. For example, spatial attention can enhance or suppress neuronal responses at a specific visual field location by decreasing or increasing surround suppression in V1 neurons at that location, thus strengthening the representation of attended locations. Indeed, Roberts et al. (2007) showed that spatial attention changes the spatial summation properties of V1 neurons in awake behaving macaques, by increasing the strength of surround suppression and shrinking foveal RFs, while causing the opposite effects on peripheral RFs. These changes in surround suppression strength could be mediated by a feedback mechanism that is unspecific for visual stimulus features, e.g. the anatomically diffuse feedback to layer 1 (Angelucci & Bressloff, 2006; Stettler et al., 2002), because spatial attention does not affect neuronal selectivity (McAdams & Maunsell, 1999). On the other hand, a feedback mechanism that is specific for visual stimulus features, e.g. the anatomically patchy and orientation specific feedback to V1 layers 2/3 and the patchy feedback to 4B (Angelucci et al., 2002; Shmuel et al., 2005), could underlie “feature-based” attention. The latter, enhances responses to a certain stimulus dimension (e.g. orientation) or value of a feature (e.g. vertical orientation) and can sharpen neuronal tuning at the level of single neurons (Spitzer, Desimone, & Moran, 1988) or the neuronal population (Martinez-Trujillo & Treue, 2004), and therefore most likely requires feature-specific feedback terminations.

While it is likely that one function of feedback is to modulate surround modulation, we think that feedback also generates far-surround modulation for the reasons explained above (Angelucci & Bressloff, 2006). Moreover, far-surround modulation occurs under anesthesia, suggesting it plays some role in pre-attentive encoding of visual stimuli. In Section 3.2 above, we have argued that the tuning of surround modulation reflects the statistics of natural images, thus aiding efficient encoding of visual information. Accordingly, the statistical dependencies in natural images decrease and become only weakly dependent on orientation for distant locations in the image (Fig. 4A), and surround modulation shows a similar distance-dependence of its strength and orientation tuning (Fig. 4B). Moreover, far-surround suppression follows the joint statistics of spatially displaced edges: it is broadly tuned for orientation (Shushruth et al., 2013), just like the more distant parts of natural contours can have a wide distribution of orientations (Geisler et al., 2001). Thus, far-surround modulation suppresses V1 responses to most oriented stimuli at distant locations in the images, except when the distant stimuli are of markedly different orientation. This property of far-surround modulation can be useful to enhance the salience of highly dissimilar stimuli at distant visual field locations to guide saccades and attention, and could be generated by a feedback mechanism that is only broadly tuned for orientation (e.g. feedback to layers 2/3 or 6).

Petrov & McKee (2006) have previously proposed that the only purpose of surround suppression is to detect salient features to guide saccadic eye movements. Their argument was based on two of their psychophysical observations: 1. that surround suppression of contrast detection does not occur at the fovea, a finding initially reported by Snowden & Hammett (1998) and later by Petrov, Carandini, & McKee (2005), and 2. that the spatial extent of the suppression field does not scale with the spatial frequency of the center and surround stimuli. Petrov & McKee argued that because natural images are roughly scale invariant (Ruderman, 1994), then surround suppression should scale with spatial frequency if its purpose is to extract contours and boundaries; however, a crude suppression mechanism designed to detect salient peripheral targets does not need to scale with spatial frequency, and would come at a cost of impaired spatial resolution. It is our opinion that these authors' argument is only partly valid, since surround suppression does occur at the fovea when the contrast of the target is above threshold, i.e. when suppression is measured either using supra-threshold contrast matching (Cannon & Fullenkamp, 1991, 1993; Chubb, Sperling, & Solomon, 1989; Ejima & Takahashi, 1985; Kilpeläinen, Donner, & Laurinen, 2007; Nurminen, Peromaa, & Laurinen, 2010; Olzak & Laurinen, 1999; Snowden & Hammett, 1998; Solomon, Sperling, & Chubb, 1993; Xing & Heeger, 2000, 2001) or detection on a pedestal (Chen & Tyler, 2002; Snowden & Hammett, 1998; Yu & Levi, 1997) tasks. Moreover, the argument that the spatial extent of surround modulation does not scale with spatial frequency is controversial, because the spatial extent of near-surround modulation does scale at supra-threshold contrast, at least up to 4.8 cycles surround width (Cannon & Fullenkamp, 1991). Therefore, efficient encoding of natural images leading to extraction of object boundaries remains a possible function for surround suppression, and we argue this is the primary function of near-surround suppression, while far-surround suppression serves visual salience.

4. Conclusions

In this article we have presented our view that surround modulation consists of multiple components serving distinct functional roles in natural vision, and generated by different neural circuits (Fig. 2). While some of the hypotheses we have laid out in this special issue article are rather speculative, they nevertheless provide plausible and testable predictions for future experimental testing. Feedforward afferents from the LGN contribute an untuned component to surround suppression in V1, which is spatially restricted to the near surround and results from surround suppression in LGN neurons; we suggest that the function of this feedforward component is to normalize V1 cell responses to local contrast. Horizontal connections within V1 likely contribute a narrowly orientation-tuned component to surround modulation, which is also spatially confined to the near surround; we propose that its function is to increase the coding efficiency of natural images by increasing the sparseness of the neuronal response and reducing statistical dependencies, leading to extraction of object boundaries in natural scenes. Feedback connections from extrastriate cortex to V1 likely contribute a broadly orientation-tuned component to surround modulation in V1, which is spatially coextensive with the far surround; we suggest that its function is to increase the salience of distant visual targets. In summary, near- and far-surround modulation in V1 may function as two parallel mechanisms/pathways for visual processing: far-surround modulation quickly detects and guides saccadic eye movements to salient parts of the visual scene that may be behaviorally important, while near-surround modulation detects object boundaries.

Highlights.

• Surround modulation (SM) in V1 has multiple components generated by distinct circuits

• Local untuned SM, generated by LGN afferents, serves contrast normalization

• Tuned near-SM, generated by horizontal axons, serves efficient natural image coding

• Efficient natural image coding by tuned near-SM leads to boundary segmentation

• Broadly tuned, feedback-mediated, far-SM serves to enhance distant target salience