Responses to second-order texture modulations undergo surround suppression

Abstract

First-order (contrast) surround suppression has been well characterized both psychophysically and physiologically, but relatively little is known as to whether the perception of second-order visual stimuli exhibits analogous center-surround interactions. Second-order surround suppression was characterized by requiring subjects to detect second-order modulation in stimuli presented alone or embedded in a surround. Both contrast-(CM) and orientation-modulated (OM) stimuli were used. For most subjects and both OM and CM stimuli, second-order surrounds caused thresholds to be higher, indicative of second-order suppression. For CM stimuli, suppression was orientation-specific, i.e., higher thresholds for parallel than for orthogonal surrounds. However, the evidence for orientation specificity of suppression for OM stimuli was weaker. These results suggest that normalization, leading to surround suppression, operates at multiple stages in cortical processing.

1. Introduction

An abundance of evidence suggests that the early visual system analyzes visual information using relatively independent “channels” selective for orientation and spatial frequency (Blakemore & Campbell, 1969; Campbell & Robson, 1968; Campbell, Carpenter, & Levinson, 1969; De Valois & De Valois, 1988; Graham, 1989; Graham & Nachmias, 1971). Each channel is composed of a set of spatially localized linear filters that together tile the visual field. In particular, psychophysical sensitivity to luminance modulations (a type of “first-order” cue in the visual image) is adequately captured by a computational model involving linear filtering followed by rectification; these linear filters are in turn represented neurophysiologically by the classical receptive fields of neurons in the primary visual cortex. While linearity and independence provide a good first approximation to the filter responses, complex, nonlinear spatial interactions among filters have also been well documented.

One such nonlinear spatial interaction is surround suppression. Psychophysically, when a target stimulus is embedded in a high-contrast mask or placed in the vicinity of high-contrast flankers, it becomes harder to detect or discriminate (Petrov, Carandini, & McKee, 2005; Polat & Sagi, 1993; Snowden & Hammett, 1998; Wilkinson, Wilson, & Ellemberg, 1997; Zenger-Landolt & Heeger, 2003) and its perceived contrast is lower (Cannon & Fullenkamp, 1991; Chubb, Sperling, & Solomon, 1989; Ejima & Takahashi, 1985; Olzak & Laurinen, 1999; Snowden & Hammett, 1998; Solomon, Sperling, & Chubb, 1993; Xing & Heeger, 2000; 2001). This is known as surround suppression. Suppression is maximal when the target and surround stimuli have matching spatial frequency and orientation (Cannon & Fullenkamp, 1991; Chubb et al., 1989; Olzak & Laurinen, 1999; Solomon et al., 1993; Xing & Heeger, 2001) and increases with increasing contrast of the surround (Olzak & Laurinen, 1999; Snowden & Hammett, 1998; Solomon et al., 1993).

The human visual system is also able to detect image attributes other than luminance modulations. Spatial variations of texture properties (e.g., local orientation, spatial frequency, or contrast) in the visual image are called “second-order.” These kinds of patterns are distinct from first-order, luminance-defined patterns in that they cannot be detected by a simple linear mechanism since there is no variation in mean luminance across the image. The boardwalk in Fig. 1A is an example of a texture-defined pattern that contains modulations of local orientation. The computational models typically used to explain human sensitivity to second-order image structure are called “filter-rectify-filter” (FRF) or “back-pocket” models (Fig. 1B; Chubb & Landy, 1991; Chubb, Econopouly, & Landy, 1994; see Landy & Graham, 2004, for a review). An initial stage of linear filtering is selective for a constituent texture. The output from the first stage is subjected to a static nonlinearity (e.g., full-wave rectification). A second-stage linear filter at a coarser spatial scale is then applied to the rectified, first-stage responses. This results in selectivity for the orientation and spatial frequency of second-order texture modulation. The detection of second-order image structure is thought to operate independently of that of first-order structure.

Figure 1.

Models of second-order processing. A. A natural scene containing second-order patterns. The boardwalk contains modulations of texture (defined by local orientation), which cannot be detected by a simple linear mechanism. B. A typical model of visual processing depicting the parallel pathways for first- and second-order stimuli. Top, first-order, luminance-defined stimuli are processed by a linear filter. Bottom, second-order, texture-defined stimuli are processed via a filter-rectify-filter (FRF) cascade.

Relatively little is known as to whether the perception of second-order stimuli exhibits analogous center-surround interactions observed for first-order stimuli. One psychophysical study provided evidence for second-order surround suppression based on the appearance of texture stimuli, in particular the perceived modulation depth of contrast-modulated stimuli (Ellemberg, Allen, & Hess, 2004). If a surround suppresses the response to a central, second-order stimulus, then its perceived modulation depth would be reduced. The authors found that, analogous to first-order suppression, second-order suppression was selective for orientation and spatial frequency, but the tuning was more broadband (i.e., the suppression effect was evident for greater differences in relative orientations or spatial frequencies between the target and the surround, as compared to first-order suppression).

We wondered whether the same suppressive effects generalized across different types of second-order stimuli. Here, we used a psychophysical protocol involving the detection of both contrast and orientation modulation to test for and characterize second-order surround suppression. This mirrors analogous experiments on first-order suppression that measured perceived contrast or detection/discrimination sensitivity (Cannon & Fullenkamp, 1991; Chubb et al., 1989; Ejima & Takahashi, 1985; Olzak & Laurinen, 1999; Petrov et al., 2005; Polat & Sagi, 1993; Snowden & Hammett, 1998; Solomon et al., 1993; Wilkinson et al., 1997; Xing & Heeger, 2000; 2001; Zenger-Landolt & Heeger, 2003), though there is no simple way to relate appearance and discrimination measures (see, e.g., Snowden & Hammett, 1998). Furthermore, the use of orientation-modulated stimuli also helped to put aside concerns about potential artifacts present in contrast-modulated stimuli, such as distortion products caused by nonlinearities in the display or early luminance nonlinearities in the visual system (Schofield & Georgeson, 1999; Smith & Ledgeway, 1997). We measured thresholds for second-order target stimuli in the presence of surround stimuli with varying depth and orientation of modulation, and found that target thresholds were greater when the surround comprised a second-order modulation. Furthermore, to our surprise, suppression was only consistently orientation-specific for contrast-modulated stimuli, while support for orientation-specific suppression in orientation-modulated stimuli was weaker. These results are consistent with the idea that there is a plethora of distinct second-order mechanisms, with different second-stage suppression mechanisms, and that the goal of second-order vision is not only to detect boundaries, but also to extract and characterize image statistics, as required by models of texture appearance (e.g., Portilla & Simoncelli, 2000).

2. Methods

2.1. Subjects

Six subjects (two females, aged 25–52) with normal or corrected-to-normal vision participated in the study. Subjects included two of the co-authors. All subjects were experienced psychophysical observers. Subjects provided written informed consent, and the experimental protocol was approved by the University Committee on Activities involving Human Subjects at New York University.

2.2. Visual stimuli

Stimuli were generated using MATLAB (MathWorks, MA) and displayed on a 22’’ flat-screen CRT monitor (Hewlett-Packard p1230; resolution: 1152 × 870; refresh rate: 75 Hz) at a distance of 57 cm. The monitor provided approximately 39.1 × 30.0 deg viewing angle. The display was calibrated and gamma-corrected using a linearized lookup table.

The second-order stimuli were contrast-modulated (CM) or orientation-modulated (OM) horizontal and vertical grating patterns (Fig. 2A,B). A CM grating L_CM (Fig. 2A) was generated by sinusoidally modulating the luminance contrast of a noise carrier image N(x,y),

$$L_{\text{CM}}(x,y)=L_0[1+A_{\mathrm{M}}M(x,y)N(x,y)],$$ (1)

where L₀ is the background luminance, A_M is the modulation amplitude, and M(x,y) is the modulator image of a two-dimensional vertical or horizontal sine wave grating with spatial frequency (SF) f and phase ϕ. M(x,y) = sin(2π f x + ϕ) (vertical) or M(x,y) = sin(2π f y + ϕ) (horizontal). The carrier image N(x,y) was white noise filtered with an isotropic bandpass filter. The filter was a cosine-ramped annulus in the Fourier domain, with a center SF of 8 cyc/deg and a bandwidth of 1 octave (i.e., the annulus extended from 5.7 – 11.3 cyc/deg). N(x,y) was normalized so that 99.5% of the pixels had values within the range of [−1,1]; the small number of pixels with values outside of that range were clipped to −1 or 1.

Figure 2.

Stimuli and task. A. Contrast-modulated grating. Left, the luminance contrast of a noise carrier varies sinusoidally in space (Eq. 1, A_M = 1). Right, noise carrier only, with no contrast modulation (Eq. 1, A_M = 0). B. Orientation-modulated grating. Left, the relative contribution of two oriented noise carriers (45° and 135°) varies sinusoidally (Eq. 2, A_M = 1). Right, an orientation-defined pattern in which the two carriers have equal contrast (Eq. 2, A_M = 0). Black curves indicate modulation over space for the patterns above them. C–F, Surround conditions; second-order stimuli are shown schematically to make the figure legible. Actual stimuli contained noise rather than plaid carriers as in A–B. C. Target-only (“none”) condition. The target stimulus was defined in an annular region (2 – 3.5 deg in eccentricity) and divided into four quadrant segments by black lines. In each trial, two diagonally opposed segments (either 1 & 3, or 2 & 4; shown here as 1 and 3) contained a modulated second-order grating, while the other two segments contained zero second-order modulation (noise carrier only for CM; an equal mixture of the two noise carriers for OM). The orientation of modulation was either horizontal or vertical (shown here as vertical). Numbers did not appear on the actual stimulus. D. The surround stimulus contained zero second-order modulation (“uniform” condition). The target was embedded in a surround inside and outside of the target annulus. The entire stimulus covered a circular region (diam: 15.1 deg). E. The surround stimulus was fully modulated (A_M = 1 in Eqs. 1 and 2), with the orientation of modulation orthogonal to the target modulation (“orthogonal” condition). F. The surround stimulus was fully modulated, with the orientation of modulation parallel to the target modulation (“parallel” condition).

An OM grating (Fig. 2B) was generated by sinusoidally modulating between two orthogonally oriented noise carrier patterns N₁ and N₂ (Landy & Oruç, 2002; Larsson, Landy, & Heeger, 2006),

$$L_{\text{OM}}(x,y)=L_0(1+{[0.5(1-A_{\mathrm{M}}M(x,y))]}^{0.5}{N}_1(x,y)+{[0.5(1+A_{\mathrm{M}}M(x,y))]}^{0.5}{N}_2(x,y)),$$ (2)

where L₀, A_M and M were as defined earlier. The noise carriers N₁ and N₂ were generated similarly to N for CM gratings above, but were instead filtered with bandpass filters oriented at 45° and 135° respectively. The filters were sharp-edged annular wedges in the Fourier domain (orientation bandwidth: 10°; center SF: 6 cyc/deg; SF bandwidth: 1 octave). The square root ensured that expected contrast power was constant across the stimuli (Landy & Oruç, 2002). We used a modulator SF f = 1.5 cyc/deg for both CM and OM stimuli, and the modulator phase was randomized from trial to trial.

2.3. Experimental procedure

Experiments were performed separately for CM and OM stimuli. The target stimulus was an annular region that extended between 2 and 3.5 deg eccentricity (Fig. 2C–F). The annulus was divided into four quadrant segments separated by black lines. During each trial, two diagonally positioned segments (either 1 and 3, or 2 and 4, Fig 2C) contained a (horizontally or vertically) modulated second-order grating, while the other two segments contained a pattern with no modulation (A_M = 0). In the case of CM stimuli, this was simply the noise carrier (Fig. 2A, right); in the case of OM stimuli, this was an equal mixture of the two oriented noise carriers (Fig. 2B, right). The target was either presented alone (“none”, Fig. 2C) or embedded in a surround stimulus inside and outside of the annulus (Fig. 2D–F). All annular envelopes that contained the target or the surround had raised cosine edges (raised-cosine widths for inner and outermost edges of the surround: 0.5 deg; for the edges at which the target and the surround bordered each other: ~0.1 deg). The annular surround outside of the target region extended the entire stimulus to 15.1 deg in diameter. Three types of surround stimuli were used: no second-order modulation (“uniform”, A_M = 0; Fig. 2D), full modulation (A_M = 1) with the modulator orientation orthogonal to the target modulation (“orthogonal”, Fig. 2E), and full modulation with the modulator orientation parallel to the target modulation (“parallel”, Fig. 2F). In all, this yielded four possible trial types corresponding to the four surround conditions.

Subjects performed a 2-alternative, forced-choice (2AFC) task in which they indicated with a key press which segments of the target region (1&3 or 2&4) contained patterns with second-order modulation. The subject maintained fixation on a central 0.4-deg crosshair throughout the experiment, and received feedback after each trial through a change in the color of the fixation cross. The stimulus was displayed for 250 ms, and the subject had unlimited time to respond. The next trial began 750 ms after the response. All trial types were counterbalanced and presented in pseudorandom order using two interleaved adaptive staircases per surround condition (1-up-3 down and 1-up-2-down; Levitt, 1971), resulting in a total of 8 staircases per experimental session. Each session typically consisted of 4 blocks of 200 – 256 trials. Different sessions were held on separate days. Within each session the staircases carried over from block to block. For the CM experiments, each of three subjects completed 1600 trials in total (from two sessions); one subject completed 2520 trials in total (from three sessions). For the OM experiments, each of the four subjects completed 2048 trials (from two sessions).

2.4. Data analysis

For each subject, we computed the percentage correct as a function of the modulation amplitude of the target for each of the four surround conditions, pooling across target locations (1&3 or 2&4), target modulator orientations (horizontal and vertical), and staircases.

For each subject, we fit a modified Weibull function (Quick, 1974) to the data for each surround condition,

$$P_s(m_j)=0.5+(0.5-\lambda )(1-\text{exp}[-{(m_j/{\alpha }_s)}^{\beta }]),$$ (3)

where P_s was the probability of a correct choice for condition s, and m_j was the jth target modulation. The “lapse rate” parameter λ was introduced to avoid biased parameter estimates (Wichmann & Hill, 2001). We fit the four psychometric functions for all surround conditions simultaneously, using a single slope parameter β and lapse-rate parameter λ across conditions, while allowing for a condition-specific threshold parameter α_s. Values for the six free parameters (β, λ, and α₁–α₄) were estimated using a maximum-likelihood procedure. We estimated a discrimination threshold for each condition as the target modulation needed to obtain 75% performance accuracy. Overall goodness-of-fit for each subject did not differ significantly between psychometric functions fit separately to each surround condition and those fit simultaneously to all conditions with shared parameters β and λ (p > 0.05 for all subjects; likelihood ratio test; Hoel, 1971); we therefore opted to use the latter model, which contained fewer parameters.

We used bootstrapping to obtain confidence intervals for the threshold estimates and p values for differences in thresholds across conditions. For each subject and each condition, we non-parametrically bootstrapped the trials at each tested target modulation amplitude (equivalent to drawing from a binomial distribution with parameters indicated by the measured number of trials and probability of success), obtained a new psychometric function, and recalculated the fit on the bootstrapped data to obtain a threshold. We repeated this procedure 1000 times, and the 2.5^th and 97.5^th percentiles of the resulting distribution of threshold values provided a 95% confidence interval. To determine whether the threshold for one condition was significantly higher than the threshold for another condition in a single subject (i.e., whether their ratio was significantly greater than 1), we computed the ratio between two randomly selected bootstrapped values corresponding to the two thresholds. We repeated this procedure 5000 times and determined the p value as the fraction of times that the bootstrapped ratios were less than 1. To determine whether thresholds differed significantly between two conditions across all subjects, we performed a similar analysis by randomly sampling bootstrapped thresholds from those computed above for individual subjects. The sampled thresholds for each condition were averaged across subjects and then the ratio of average thresholds for the two conditions was computed. This procedure was repeated for 5000 times and the p value corresponded to the fraction of these bootstrapped ratios that was less than 1.

3. Results

We tested for second-order surround suppression by measuring the modulation sensitivity for second-order target stimuli as a function of the type of surround modulation. A higher discrimination threshold when surround modulation is present would suggest that second-order modulation of the surround exerts a suppressive effect on the sensitivity for target stimuli. Comparing thresholds for parallel versus orthogonal surround modulation would indicate whether second-order surround suppression is orientation-selective. Any potential confounding, first-order suppression was accounted for by comparing the thresholds for the target when it was presented in isolation versus embedded in a uniform surround with no second-order modulation.

3.1. Contrast-modulated stimuli

Contrast modulation sensitivity was lower for the target stimulus when the target was viewed in the presence of a modulated surround stimulus, indicative of second-order surround suppression (Fig. 3). Compared to the uniform-contrast surround condition, psychometric functions for the full-modulation surround conditions were systematically shifted to the right, reflecting higher discrimination thresholds (Fig. 3A). In addition, threshold was greater for the parallel-surround condition than for the orthogonal-surround condition, indicating that the suppressive effect was orientation-selective. Second-order suppression was robust across all four tested subjects (Fig. 3B; Fig 3C, left bar group) as well as when thresholds were averaged across subjects (see Section 2.4; p < 0.0002). Furthermore, all subjects showed an orientation-specific effect (Fig. 3B; Fig. 3C, middle bar group; p = 0.003 across all subjects). None of the subjects showed significant change in threshold between the no-surround and uniform-surround conditions (Fig. 3C, right bar group; p = 0.62 across all subjects), confirming that the observed suppression of target modulation sensitivity was not due to first-order features (overall contrast) of the surround stimulus.

Figure 3.

Discrimination performance for contrast-modulated stimuli. A. Psychometric functions for an example subject. A psychometric function was computed for each of the four surround conditions (see Fig. 2C–F). Data points show percentage correct as a function of the modulation amplitude of the target; larger sizes indicate more trials. The size of each data point corresponds to the number of trials presented for that target modulation amplitude. The curves show best-fitting Weibull functions. We estimated the modulation threshold as the target modulation corresponding to 75% correct. Error bars beneath the curves indicate 95% confidence intervals for the threshold estimates, obtained via bootstrapping. B. Modulation threshold for each surround condition and each subject. Error bars: 95% bootstrapped confidence intervals. C. Ratios between thresholds for pairs of conditions. Error bars: 95% bootstrapped confidence intervals. Dashed line indicates a ratio of 1 (identical thresholds). Asterisks indicate statistical significance for threshold ratios > 1 (* p < 0.01).

3.2. Orientation-modulated stimuli

Three out of four subjects showed higher target modulation thresholds for the parallel surround compared to the uniform surround (Fig. 4B; Fig. 4C, left bar group; p < 0.0002 across all subjects). Most subjects did not show a statistically significant difference in thresholds for parallel versus orthogonal surrounds, suggesting that second-order suppression for OM stimuli is weakly or not orientation-selective (Fig. 4C, middle bar group). However, when thresholds were averaged across subjects, the threshold difference between parallel and orthogonal surrounds was statistically significant (p = 0.012). Three out of four subjects also showed significant first-order suppression, that is, their thresholds were lower in the no-surround than the uniform-surround condition (Fig. 4C, right bar group; p = 0.014 across all subjects). The strength of second-order suppression was weaker for OM stimuli than for CM stimuli, and fewer subjects exhibited statistically significant second-order surround suppression.

Figure 4.

Discrimination performance for orientation-modulated stimuli. Same conventions as Fig. 3. A. Psychometric functions for an example subject. B. Modulation threshold for each surround condition and each subject. C. Ratios between thresholds for pairs of conditions. Asterisks indicate statistical significance for threshold ratios > 1 (* p < 0.01, † p < 0.05).

We also measured OM detection in five subjects with a carrier that was broader-band and higher spatial frequency (orientation bandwidth: 30°; center SF: 8 cyc/deg). In those data, psychometric functions were shallower and, in some conditions, only reached 70–80% by A_M = 1. The small range of usable modulation amplitude resulted in noisier measurements and poorer fits; we therefore opted to use the narrow-band carrier stimuli (Landy & Oruç, 2002). Nonetheless, we obtained results (data not shown) that were qualitatively similar. Four out of five subjects showed significant second-order suppression, exhibiting higher modulation thresholds for parallel than uniform surrounds (p < 0.05).

4. Discussion

4.1. Normalization and the cascade model of visual processing

First-order (contrast) surround suppression has been closely linked to neurophysiological suppression in V1 cells. The responses of a V1 neuron are smaller in the presence of a surrounding stimulus placed outside of its classical receptive field, which is ineffective in driving the cell when presented alone (Bair, Cavanaugh, & Movshon, 2003; Cavanaugh, Bair, & Movshon, 2002a; 2002b; DeAngelis, Freeman, & Ohzawa, 1994; Heeger, 1992a; Hubel & Wiesel, 1968; J. B. Levitt & Lund, 1997; Maffei & Fiorentini, 1976). Analogous to psychophysical suppression, the suppression is largest when the surround stimulus is at the neuron’s preferred orientation and spatial frequency (Cavanaugh, Bair, & Movshon, 2002b; DeAngelis et al., 1994; Knierim & Van Essen, 1992).

V1 surround suppression is well described by divisive normalization, a functional model introduced to explain a variety of suppressive phenomena evident in the responses of V1 neurons (Albrecht & Geisler, 1991; Carandini & Heeger, 1994; 2012; Carandini, Heeger, & Movshon, 1997; Heeger, 1991; 1992b; 1992a; 1993; 1994; Nestares & Heeger, 1997; Tolhurst & Heeger, 1997a; 1997b). The normalization model posits that the rectified responses of a neuron to a preferred stimulus are suppressed divisively (normalized) by the summed responses across a population of neurons. During psychophysical surround suppression, perceptual sensitivity for a target stimulus is degraded by a high-contrast surround stimulus. Normalization implies that the excitatory drive from the target stimulus is suppressed divisively by neural responses selective to the surround stimulus, and the suppression is stronger when surround contrast is higher. The target must then have a higher contrast to evoke a criterion response for detection or discrimination.

While nonlinearities in neural responses such as surround suppression have been studied most extensively in V1, analogous suppressive effects have been reported in some extrastriate areas as well: divisive normalization has been used to account for suppressive effects in dorsal-stream visual cortical areas MT and MST (Britten & Heuer, 1999; Heuer & Britten, 2002; Recanzone, Wurtz, & Schwarz, 1997; Snowden, Treue, Erickson, & Andersen, 1991) and in ventral stream areas V4 and IT (Miller, Gochin, & Gross, 1993; Missal, Vogels, & Orban, 1997; Reynolds & Desimone, 2003; Reynolds, Chelazzi, & Desimone, 1999; Richmond, Wurtz, & Sato, 1983; Rolls & Tovee, 1995; Sato, 1989; Zoccolan, Cox, & DiCarlo, 2005). Therefore, it has been proposed that, akin to linear filtering and rectification, normalization may be a “canonical” operation carried out by neural populations at multiple stages of cortical computation (Carandini & Heeger, 2012; Grossberg, 1973; Heeger, Simoncelli, & Movshon, 1996; Kouh & Poggio, 2008; Luo, Axel, & Abbott, 2010; Olsen, Bhandawat, & Wilson, 2010; Simoncelli & Heeger, 1998).

FRF models provide a relatively simple mechanism to account for second-order pattern perception that could be implemented in neuronal circuitry. Neurons with properties consistent with the second-stage filter have been found in extrastriate visual areas of cat and monkey (see Section 4.2 below). Variants of the FRF model also include first-order normalization, in which the rectified outputs of first-stage filters are inhibited by the pooled responses of other first-stage filters, before providing input to the second-stage filters (Graham, 1991; Graham & Sutter, 1998; Graham, Beck, & Sutter, 1992; Olzak & Thomas, 1999). This is consistent with the cross-channel nonlinear interactions (i.e., first-order normalization) observed both psychophysically and neurophysiologically.

An extension of the FRF models is to combine them with the normalization model, thus treating normalization as a stage in a canonical series of computations. In this “cascade model”, the three operations—linear filtering, rectification, and normalization—are cascaded across sequential stages: F₁R₁N₁F₂R₂N₂ (Fig. 5). The cascade model thus bridges theories inferred from both psychophysics and single-cell physiology. A key prediction of this model is that the rectified outputs of second-stage filters undergo normalization in a manner similar to those of first-stage filters. This model therefore predicts that the strength of second-order suppression should increase with the overall outputs of second-order filters, which depend on the overall second-order modulation of the image.

Figure 5.

The cascade model of visual processing. The cascade model combines the normalization model and the FRF model, and posits that the three operations—filtering, rectification, and normalization—are repeated and cascaded across sequential stages, leading to a series of stages F₁R₁N₁, F₂R₂N₂, etc.

We used the detection of contrast and orientation modulation to measure observers’ perceptual sensitivity for second-order features and infer possible underlying neural suppression associated with second-order visual processing. We found significant threshold elevation when the surround stimulus contained high modulation depth compared to when it contained no modulation, which indicated second-order surround suppression. For CM stimuli, suppression was orientation-specific, i.e., subjects exhibited higher thresholds when the orientation of second-order surround modulation was parallel to that of the target. Suppression was not consistently orientation-specific for OM stimuli. For both OM and CM stimuli, space-averaged responses of first-stage filters (F₁) were independent of the modulation depth of the surround, and therefore first-stage suppression (N₁) was constant across all surround-present conditions. In sum, our results provided evidence for the existence of second-order suppression computed independently of N₁. While these results do not demonstrate whether the nature of suppression is divisive as predicted by normalization, they are consistent with the cascade model of cortical processing, and suggest that a general gain-control mechanism may contribute to both first-and second-order visual processing.

4.2. Second-order visual processing and its neural substrates

The cascade model provides a functional rather than a mechanistic description of visual computation, and does not make specific predictions about where each of the component operations might occur in visual cortex. Although drawn as a feedforward process in Fig. 5, the operations of linear summation and division can be computed with either feedforward (Carandini, Heeger, & Senn, 2002; Olsen & Wilson, 2008; Reichardt, Poggio, & Hausen, 1983) or feedback mechanisms (Angelucci, Levitt, & Lund, 2002; Carandini et al., 1997; Carandini & Heeger, 1994; Heeger, 1992a; 1993).

While psychophysical models adequately capture human sensitivity to a variety of second-order modulations, relatively little is known about how second-order vision is represented in visual cortex. Theoretical and psychophysical results show that the detection of first and second-order patterns require separate mechanisms, as their interactions are weak or absent (Ellemberg et al., 2004; Lin & Wilson, 1996; Morgan, Mason, & Baldassi, 2000; Schofield & Georgeson, 1999; Scott-Samuel & Georgeson, 1999). Like first-order channels, second-order channels are tuned for orientation and spatial frequency (Arsenault, Wilkinson, & Kingdom, 1999; Dakin, Williams, & Hess, 1999; Landy & Oruç, 2002; Schofield & Georgeson, 1999; Scott-Samuel & Georgeson, 1999; Sutter, Sperling, & Chubb, 1995), but have wider bandwidth (Landy & Oruç, 2002). Some neurons in area 18 of the anesthetized cat (Mareschal & Baker, 1998; Song & Baker, 2007; Zhou & Baker, 1993), and in macaque areas V2 (Leventhal et al., 1998; Rossi et al., 2001; but see El-Shamayleh & Movshon, 2011) and MT (O'Keefe & Movshon, 1998; Olavarria et al., 1992), have been reported to show selectivity for second-order cues. fMRI-adaptation studies demonstrated selectivity for second-order modulation of orientation (Larsson et al., 2006) and spatial frequency (Hallum, Landy, & Heeger, 2011) in human visual cortex. Progressively stronger adaption for second-order modulation was reported in higher-order visual areas, providing support for the idea that second-order processing takes places outside of V1. However, the presence of orientation and spatial-frequency selective adaption in V1 suggests that subpopulations of V1 neurons may also perform second-order processing (Hallum et al., 2011).

4.3. Previous evidence for higher-order suppression

Several physiology studies have found divisive suppression in extrastriate areas (e.g., Britten & Heuer, 1999; Heuer & Britten, 2002; Recanzone et al., 1997; Snowden et al., 1991), but evidence for multiple stages of divisive suppression has been largely inconclusive because it is difficult to determine if the suppression is computed de novo or inherited from the inputs (Rust, Mante, Simoncelli, & Movshon, 2006). For instance, one study (Britten & Heuer, 1999) examined the spatial interaction of multiple small Gabor stimuli within the receptive fields of single MT neurons, and found that the responses of MT neurons were less than those predicted by linear summation, and were instead well described by divisive normalization. It is known that neurons in area MT receives direct inputs from V1. While this presents some evidence for N₂ in the dorsal stream, it is unclear to what extent the measured suppression included normalization computed in and inherited from V1 versus that computed in MT.

4.4. Strong evidence of orientation-dependent suppression for CM but not for OM stimuli

Analogous to N₁, we found that N₂ for CM stimuli was orientation-selective, but N₂ was not consistently orientation-selective for OM stimuli. According to some implementations of the FRF model, F₂ does not discriminate between inputs from different types of carriers (“carrier-invariance”). However, some classes of FRF models detect different texture-modulation types with separate FRF mechanisms (e.g., Kingdom, Prins, & Hayes, 2003). This might seem computationally inefficient because a large number of neurons would be required for representing different types of second-order stimuli. However, it makes sense if the goal of second-order vision is not only to detect boundaries, but also to extract and characterize image statistics, as required by models of texture appearance (e.g., Portilla & Simoncelli, 2000). Following that logic, signals from different carrier types would need to be kept separate rather than recombined for characterizing the textures, and second-order suppression for different carrier types would be mediated by different normalization pools.

4.5. Lack of first-order suppression for CM stimuli

We did not observe significant first-order suppression for CM stimuli, as thresholds for no-surround and uniform-surround conditions did not differ. This might be seen as surprising because the responses of the first-stage filters should be suppressed (due to normalization) by the contrast of a surround stimulus (e.g., the first-order contrast might be perceived to decrease from 100% to 80%). As the outputs of first-stage filters provide input to second-stage filters, weaker inputs would predict lower modulation sensitivity. However, second-order sensitivity for the target stimulus may remain the same despite modest attenuation of its first-order inputs, consistent with a number of studies that found that detectability of second-order contrast modulation is only weakly dependent on carrier contrast (for a variety of carriers) once the contrast was sufficiently above detection threshold (Cropper, 1998; Dakin & Mareschal, 2000; Jamar & Koenderink, 1985; Schofield & Georgeson, 1999). Functionally, this could be due to an invariance mechanism that ensures that the second-stage of the model maintains sensitivity to second-order features despite variations in first-order contrast. In addition, this lack of dependence on carrier contrast is more pronounced for high-spatial-frequency carriers (Dakin & Mareschal, 2000). This is consistent with our measurements, which were conducted at a relatively high carrier frequency (center SF = 8 cyc/deg) and 100% carrier contrast. Lastly, similar to our results, Ellemberg et al. (2004) also found no reduction in the apparent modulation depth of a contrast-modulated target when it was flanked by unmodulated carrier flanks compared to when it was presented alone.

We did observe significant first-order suppression in 3 out of 4 subjects for OM stimuli. This result is consistent with the idea that the detection of OM and CM may be mediated by separate FRF mechanisms. One previous study, however, suggested that discriminability of OM did not depend on carrier contrast for a carrier contrast range of 60–80% (Barbot, Landy, & Carrasco, 2011). This would predict that the first-order surround in our experiment would have little effect on the second-order modulation sensitivity of the target, even when the surround lowers the effective first-order target contrast (due to first-order suppression). However, this previous study was conducted using different parameters (e.g., lower carrier and modulator spatial frequencies and a larger eccentricity than those tested here), and it is unknown whether any of those variables could have contributed to the differences.

4.6. Center-surround configuration

Our results, using a discrimination task, agree with those of Ellemberg et al. (2004), which were based on the appearance of second-order stimuli. However, one difference between our study and that of Ellemberg et al (2004) is that we used a large surround stimulus, while they used localized Gabor stimuli. Previous first-order suppression experiments have used designs in which the target was flanked by Gabors or Gabor-like elements (Wilkinson et al., 1997; Williams & Hess, 1998; Zenger-Landolt & Koch, 2001) or was embedded in a large surround (Chubb et al., 1989; Petrov et al., 2005; Snowden & Hammett, 1998; Solomon et al., 1993; Xing & Heeger, 2000), generally producing similar patterns of results. Therefore, we do not expect the size of target and surround stimuli to contribute to any substantial differences in results. In addition, in some first-order experiments, suppression also depends on the relative phase between target and surround stimuli (Ejima & Takahashi, 1985; Olzak & Laurinen, 1999; Polat & Sagi, 1993; Williams & Hess, 1998; Zenger & Sagi, 1996), though these experiments were mostly performed for foveal targets. It has been found that first-order surround suppression in the periphery does not depend on the relative phase between the target and the surround (Petrov & McKee, 2006). Here we randomized the relative phase between the target and the surround for peripheral target stimuli, and did not systematically examine how second-order surround suppression depends on relative target-surround configuration.

4.7. Surround facilitation versus suppression

In reports of first-order center-surround interactions, the effect of a surround is typically suppressive, but in some manipulations, it has been found to be facilitative, i.e., enhanced detection or greater apparent contrast of the target stimulus in the presence of a surround (Cannon & Fullenkamp, 1993; Ejima & Takahashi, 1985; Polat & Sagi, 1993; Xing & Heeger, 2001). How do these findings relate to our results? First-order facilitation is mainly evident in low surround contrasts (Cannon & Fullenkamp, 1993; Ejima & Takahashi, 1985; Yu, Klein, & Levi, 2002; Zenger & Sagi, 1996; Zenger-Landolt & Koch, 2001), which has been hypothesized to be due to signal-to-noise enhancement from spatial summation. For our manipulation we only used surround modulation depths of 0% or 100%, and did not use any weak surround modulations. In addition, first-order facilitation has mainly been reported for foveal target stimuli (Cannon & Fullenkamp, 1993; Ejima & Takahashi, 1985; Polat & Sagi, 1993; Zenger & Sagi, 1996); facilitation switches to suppression when the target stimuli are placed in the periphery (Chubb et al., 1989; Petrov et al., 2005; Snowden & Hammett, 1998; Solomon et al., 1993; Wilkinson et al., 1997; Williams & Hess, 1998; Xing & Heeger, 2000; Zenger-Landolt & Koch, 2001). Our target stimuli were presented 2 – 3.5 degrees in the periphery (Fig. 2C–F). The mainly suppressive effects that we found are therefore consistent with those observed for first-order target stimuli presented in the periphery. But it remains an open question how second-order suppression depends on variables such as target eccentricity. The strength of N₂ might increase as a function of eccentricity, analogous to first-order data. In addition, facilitation might also occur with a low modulation depth of the surround or with a foveal target.

Highlights.

• We tested for second-order surround suppression by measuring modulation thresholds.

• Thresholds were higher with than without second-order surrounds.

• Orientation-specific suppression for contrast- but not orientation-modulated stimuli.

• Normalization may underlie surround suppression at multiple visual processing stages.