Human primary visual cortex (V1) is selective for second-order spatial frequency

Abstract

A variety of cues can differentiate objects from their surrounds. These include “first-order” cues such as luminance modulations and “second-order” cues involving modulations of orientation and contrast. Human sensitivity to first-order modulations is well described by a computational model involving spatially localized filters that are selective for orientation and spatial frequency (SF). It is widely held that first-order modulations are represented by the firing rates of simple and complex cells (“first-order” neurons) in primary visual cortex (V1) that, likewise, have spatially localized receptive fields that are selective for orientation- and SF. Human sensitivity to second-order modulations is well described by a filter-rectify-filter (FRF) model, with first- and second-order filters selective for orientation and SF. However, little is known about how neuronal activity in visual cortex represents second-order modulations. We tested the FRF model by using an functional (f)MRI-adaptation protocol to characterize the selectivity of activity in visual cortex to second-order, orientation-defined gratings of two different SFs. fMRI responses throughout early visual cortex exhibited selective adaptation to these stimuli. The low-SF grating was a more effective adapter than the high-SF grating, incompatible with the FRF model. To explain the results, we extended the FRF model by incorporating normalization, yielding a filter-rectify-normalize-filter model, in which normalization enhances selectivity for second-order SF but only for low spatial frequencies. We conclude that neurons in human visual cortex are selective for second-order SF, that normalization (surround suppression) contributes to this selectivity, and that the selectivity in higher visual areas is simply fed forward from V1.

models of texture perception typically consist of a linear spatial filter, rectification of filter outputs, and a second linear spatial filter [called filter-rectify-filter (FRF) models]. The term “1st-order” refers to modulations of luminance, such as the edge illustrated in Fig. 1A. The texture-defined boundary in Fig. 1B is an example “2nd-order” image modulation. The average luminance is identical on either side of the boundary, as is the average luminance contrast; the boundary is defined by a change in a local texture property (here, orientation). A linear filter will respond to differences in luminance across a boundary (Fig. 1A). However, a linear filter cannot respond to the texture-defined edge of Fig. 1B because there is no change in average luminance across the edge. A FRF model is commonly used to explain human sensitivity to such texture boundaries (Fig. 1B). Each 1st-stage filter is selective for one of the constituent textures (e.g., 1 of the 2 orientations in Fig. 1B). The 2nd-stage filter computes a difference between rectified, 1st-stage responses at a coarser spatial scale, resulting in selectivity for the orientation of the texture boundary (Landy and Graham 2004).

Fig. 1.

Visual processing of 1st- and 2nd-order features. A: 1st-order processing. Luminance-defined edge, a 1st-order modulation, can be signaled by a linear filter, as shown by the cartoon receptive field. B: 2nd-order processing. Mean luminance and mean contrast are equal on both sides of the orientation-defined edge, a second-order modulation. Therefore, this edge cannot be signaled by a linear mechanism as in A. A filter-rectify-filter mechanism as shown is commonly used to explain human sensitivity to 2nd-order features. The 1st-stage filter is selective for 1 of the constituent textures (e.g., the 45° orientation). This filter will have strong responses to its preferred texture, both positive and negative (depending on the relative phase of the stimulus and receptive field), with weaker responses to the other texture. A pointwise rectification is applied to the output of the 1st-stage filter, resulting in stronger positive responses to the preferred compared with the nonpreferred texture. The 2nd-stage filter computes a difference between the rectified responses of the 1st-stage filters, at a coarser spatial scale, and is selective for the orientation of the texture boundary.

There is a wealth of information on the tuning of 1st-order, linear, spatial filters, but much less is known about 2nd-order filters. Behavioral studies have estimated 1st-order tuning bandwidths of approximately an octave in spatial frequency (SF) and 30° in orientation using various techniques (summation, adaptation, masking, etc.; see Graham 1989). These bandwidth estimates are reasonably consistent with neurophysiological measurements in macaque V1 (see De Valois and De Valois 1988). Second-order contrast sensitivity is relatively broadband for a variety of texture modulations (Kingdom et al. 1995; Sutter et al. 1995; Schofield and Georgeson 1999; Landy and Oruç 2002). It is hypothesized that this sensitivity arises from multiple, narrowly tuned mechanisms (Arsenault et al. 1999; Schofield and Georgeson 1999; Landy and Oruç 2002). However, very few physiological studies have demonstrated selectivity for 2nd-order features. In area 18 of the anesthetized cat, there are reports of neurons tuned for 2nd-order orientation (Mareschal and Baker 1998; Baker and Song 2008; Tanaka and Ohzawa 2009), but there are no comparable results from the macaque (El-Shamayleh 2009). Second-order orientation selectivity has been demonstrated in human cortex (Larsson et al. 2006), but there are no studies demonstrating SF selectivity to 2nd-order texture modulation.

The normalization model of visual cortical responses was introduced to explain a variety of suppressive phenomena evident in the responses of V1 neurons (Robson 1988; Albrecht and Geisler 1991; Heeger 1991, 1992a,b, 1993; Carandini and Heeger 1994; Heeger 1994; Carandini et al. 1997; Nestares and Heeger 1997; Tolhurst and Heeger 1997a,b). The responses of a V1 neuron to a preferred stimulus are diminished by the simultaneous presentation of a second, nonpreferred stimulus that is ineffective in driving the cell when presented alone (Morrone et al. 1982; Bonds 1989; Bauman and Bonds 1991; Heeger 1992b; Deangelis et al. 1994; Carandini et al. 1997; Levitt and Lund 1997; Cavanaugh et al. 2002a,b; Bair et al. 2003). Analogous suppressive phenomena have been observed in dorsal-stream visual cortical areas MT and MST (Snowden et al. 1991; Recanzone et al. 1997; Britten and Heuer 1999; Treue et al. 2000) and in ventral stream areas V4 and IT (Richmond et al. 1983; Sato 1989; Miller et al. 1993; Rolls and Tovee 1995; Missal et al. 1997; Reynolds et al. 1999; Reynolds and Desimone 2003; Zoccolan et al. 2005). The normalization model explains these results by positing that the stimulus drive is suppressed, effectively normalizing (dividing) the response of each neuron by the sum total stimulus drive across a population of neurons. Normalization has been proposed to operate at multiple stages of processing in multiple neural systems; therefore, it has been proposed as a “canonical” cortical computation (Grossberg 1973; Heeger et al. 1996; Simoncelli and Heeger 1998; Kouh and Poggio 2008; Luo et al. 2010; Olsen et al. 2010). Both the FRF model and the normalization model were initially proposed around 1990. Even so, the theoretical and empirical literature on each of these models has remained largely separate and distinct; specifically, the literature on 2nd-order selectivity has largely ignored normalization. This is surprising and represents a serious gap in vision science.

Surround suppression, which is one of the nonlinear aspects of neural responses explained by the normalization model, might contribute to 2nd-order selectivity. Surround suppression can be demonstrated by placing mask stimuli outside the classical receptive field of a V1 neuron and noticing that these stimuli reduce responses to stimuli placed inside the classical receptive field (Blakemore and Tobin 1972; Dreher 1972; Allman et al. 1985; Nelson and Frost 1985; Deangelis et al. 1994; Levitt and Lund 1997; Fitzpatrick 2000; Cavanaugh et al. 2002a,b; Bair et al. 2003; Tanaka and Ohzawa 2009). It can also be demonstrated by enlarging the size of a single high-contrast stimulus and noticing that responses peak and then decrease (Kapadia et al. 1999; Sceniak et al. 1999; Tailby et al. 2007). Normalization accounts for surround suppression because the normalization factor has a larger spatial footprint than the classical receptive field (Carandini 2004). If 2nd-order stimulus modulation is matched in its spatial frequency to the spatial scale of the suppressive surround, then normalization will be strong and neural responses will be suppressed. If, on the other hand, the 2nd-order SF is not matched to the spatial scale of surround suppression, then there will be less suppression.

Here, we used a functional (f)MRI-adaptation protocol to measure activity in visual cortex to 2nd-order SF modulation. We found that visual cortical fMRI responses were not well described by the standard FRF model. Rather, we developed a modified FRNF (where N stands for normalization) model involving both 1st- and 2nd-order SF- and orientation-selective filters and surround suppression. We conclude that neurons in human visual cortex are selective for 2nd-order SF and that normalization (surround suppression) contributes to this selectivity.

METHODS

Following Larsson et al. (2006), we used an fMRI-adaptation protocol to characterize selectivity for 2nd-order stimulus modulations. Larsson et al. (2006) characterized orientation selectivity by comparing responses to 2nd-order modulations of different orientations. On each trial, a top-up adapter was shown for several seconds followed either by a uniform display (“blank”), a 2nd-order test stimulus with the same orientation as the adapter (“parallel”), or a 2nd-order test stimulus oriented perpendicular to the adapter (“orthogonal”). By subtracting the fMRI response to the blank from responses to the other two trial types, the response to the test stimulus was isolated. A smaller response in the parallel compared with the orthogonal condition was taken as an indication that a subset of neurons in that region of cortex was selective for 2nd-order orientation with a preferred orientation close to that of the adapter. Neurons selective to the orthogonal orientation and neurons unselective for 2nd-order orientation were not expected to be affected by the adaptation. This logic depends on an assumption that fMRI response in the absence of adaptation will be identical for the parallel and orthogonal orientations, so that any difference in response may be attributed to orientation-specific adaptation.

In the current study, we used a similar design except that two 2nd-order spatial frequencies were used rather than two orientations. However, there is little reason to expect fMRI responses to be identical for different 2nd-order spatial frequencies. Although 2nd-order contrast sensitivity is nearly constant across the range of 2nd-order spatial frequencies we use, that does not necessarily imply that the numbers of neurons supporting sensitivity for those spatial frequencies or their response gains are equal in any given cortical area. Thus we cannot assume that a smaller response to the adapted 2nd-order spatial frequency, relative to the unadapted frequency, is due to spatial-frequency-specific adaptation; it could merely be a difference in sensitivity to the two spatial frequencies. Therefore, we also measured fMRI responses in an unadapted state and used those measurements to compensate for any differences in local sensitivity.

Subjects.

Three subjects (S1-S3; 23- to 52-yr-old), including two of the coauthors, participated in the psychophysical and fMRI experiments; one subject (S4; 26-yr-old) participated only in the psychophysical experiments. All subjects were experienced psychophysical observers. Subjects S1 and S2 had extensive prior exposure to 2nd-order gratings, S3 had some prior exposure, and S4 had none. Subjects gave informed consent to participate in accordance with the Helsinki convention and National Institutes of Health guidelines for experiments involving human subjects. The experimental protocol was approved by the New York University Committee on Activities Involving Human Subjects.

Stimuli.

Stimuli were 2nd-order, orientation-defined gratings (Landy and Oruç 2002). Each stimulus was generated as follows. First, two “carrier” noise images, N₁ and N₂, were generated by filtering white noise using an odd-symmetric Gabor filter kernel (orientation: 45° for N₁, 135° for N₂; SF: 8 cycle/°; Gaussian SD: 3.6'). These carrier images were multiplied by a scalar so that 98% of pixels occupied the range [−1,1]. The outlying 2% of pixels were clipped to −1 or 1. The stimulus was defined as

$$L\left(x,y\right)=L_0\left(1+{\left\{0.5\left[1+cM\left(x,y\right)\right]\right\}}^{1/2}{N}_1\left(x,y\right)+{\left\{0.5\left[1-cM\left(x,y\right)\right]\right\}}^{1/2}{N}_2\left(x,y\right)\right),$$ (1)

where L₀ is the mean luminance, c is the 2nd-order contrast, and M(x,y) = sin(2πfx + φ) is a vertical, 2nd-order sinusoidal modulator with frequency f and random phase φ. The square root ensured there was no modulation of root-mean-square contrast across the stimulus (Watson and Eckert 1994) so that detection of the modulation required a mechanism that detects modulation in orientation rather than contrast. Stimuli were presented within an annulus (inner and outer radii: 1.5 and 5°); the rest of the display was uniform with the same mean luminance L₀. In both psychophysics and imaging, new random adapter stimuli were displayed at 4 Hz. Each new stimulus involved new carriers and also randomization of modulator phase. However, each trial's 1 sec probe was a single, static stimulus. Example stimuli are shown in Fig. 2.

Fig. 2.

Example 2nd-order, orientation-defined gratings. A: low-spatial frequency (SF) grating (0.18 cycle/°). Black curve indicates the 2nd-order, orientation-defined modulation, that is, the modulation between the 45° and the 135° carriers. B: high-SF grating (1.25 cycle/°). Images shown here correspond to a 1.5 × 3° detail taken from an example stimulus. In the experiments, gratings filled an annulus (radii: 1.5 and 5°; see Fig. 3) around fixation on a mean-luminance background.

Second-order psychophysics.

We measured the effect of adaptation on thresholds for the detection of 2nd-order spatial structure. Each subject participated in six blocks of 240 two-interval, forced-choice trials. Two blocks used a “1st-order-only” adapter, containing no 2nd-order modulation (that is, c = 0, resulting in a plaid mixture of the two carrier noises, see Stimuli), two used a 2nd-order, low-SF, vertical adapter (0.18 cycle/°; c = 1; Fig. 2A), and two used a 2nd-order, high-SF, vertical adapter (1.25 cycle/°; c = 1; Fig. 2B). Subjects initially viewed the adapter for 100 s. The structure of each trial is shown in Fig. 3A. Subjects fixated a central 1° crosshair throughout. On each trial subjects viewed two stimuli, a target and a blank (c = 0), and indicated which interval contained the target. Targets had either vertical or horizontal modulators, with one of two spatial frequencies (0.18 or 1.25 cycle/°), and one of 6 possible 2nd-order contrasts (c). These 24 targets were presented in pseudorandom order across trials. For each of the four subjects, three adapters, and four target types, 75% correct detection thresholds were estimated by fitting a three-parameter cumulative Gaussian. The upper asymptote of each curve was allowed to range between 95 and 100% to account for stimulus-independent errors (Wichmann and Hill 2001a,b).

Fig. 3.

Experimental protocols. A: psychophysics. Each trial began with a 4-sec top-up adapter; a vertical, low-SF, top-up adapter is shown schematically. Four target types, with various forms of 2nd-order modulation, were presented with equal probability in pseudorandomized order across trials. These target stimuli were presented in either interval 1 as shown or interval 2, while the other interval contained a 1st-order-only probe. During the response interval, subjects indicated, by pressing a button, which of the two stimulus intervals contained the 2nd-order target. B: fMRI. Each trial began with a 4-s top-up adapter; a vertical, low-SF top-up adapter is shown schematically. An 800-ms mean-luminance field preceded the 1-s probe that, with equal probability, was either a 2nd-order, low-SF probe (0.18 cycle/deg), a 2nd-order, high-SF probe (1.25 cycle/°), or a mean-luminance field. Inter-trial interval was jittered between 5 and 9.8 s and was set so that each trial was an integral multiple of the 1.2-s TR. Digits at fixation represent the “two-back” task performed throughout scanning. In both experiments, stimuli were presented within an annulus (radii: 1.5 and 5°) around fixation.

Second-order imaging (main experiment).

Each subject participated in five 2nd-order imaging sessions. In one of these sessions a 1st-order-only adapter was used that contained no 2nd-order modulation of orientation (that is, c = 0; see Stimuli); in two sessions a vertical, low-SF adapter was used (c = 1, f = 0.18 cycle/°); and in two sessions a vertical, high-SF adapter was used (c = 1, f = 1.25 cycle/°; Fig. 2). Each session began with a 100-s period during which the subject observed the adapter, followed by 8 (S2) or 10 (S1 and S3) runs. Each run comprised 24 trials. Each trial began with the top-up adapter, followed by one of three probe stimuli (high SF, low SF, or blank), followed by an intertrial interval of variable (pseudorandomized) duration (Fig. 3B). The randomized intertrial intervals and the randomly interleaved blank probes enabled us to distinguish the response time course to the top-up adapter from the response to the probe, in spite of the sluggishness of the hemodynamics. We refer to the high-SF probe as “adapted” for the sessions in which the adapter was also high SF (1.25 cycle/°), otherwise “unadapted,” and likewise for the low-SF probe.

Throughout each run, attention was controlled and diverted from the stimulus by requiring subjects to perform a demanding task at fixation, signaling, via a button press, “two-back” repetitions in a sequence of single digits. Numerous psychophysical and fMRI experiments show that adapters remain effective even when attention is diverted from the stimulus (He and MacLeod 2001; Festman and Ahissar 2004; Larsson et al. 2006; Montaser-Kouhsari et al. 2007). Similarly, adapters have proven effective in many single-unit electrophysiological experiments performed under anesthesia, thereby removing the effects of attention (e.g., Movshon and Lennie 1979). The presentation rate of digits that we used varied (1.25–2.5 Hz) across subjects and runs to ensure that the task was neither too difficult nor too easy. Overall, hit rate ranged from 78 to 95%. This range for this task left subjects completely unaware of the stimulus appearing within the annulus, consistent with previous reports (Lee et al. 2007). Furthermore, we examined how two-back task performance varied across trial types and found that performance did not depend on stimulus conditions. In each adapter condition, the proportion of two-back targets occurring within 5 s of the low-SF probe onset that were detected (pooled over subjects and runs) was not significantly different from the corresponding proportion for the high-SF probes (low-SF adapter: z = 1.46, P = 0.15; high-SF adapter: z = −0.72, P = 0.46).

Preprocessing.

fMRI time series were motion corrected within and between runs, and coregistered across sessions (Nestares and Heeger 2000). For each run, the first 24 s of the time series were discarded and the remaining time series was divided by its mean value. We then subtracted 1 and multiplied by 100, yielding a time series in units of “percent signal change.” Then, the time series from all runs were concatenated and bandpass filtered (zero-phase, fifth-order Butterworth filter with 0.02 and 0.40 Hz cut-offs).

Response time courses.

The mean fMRI response time courses were estimated using deconvolution (Dale 1999), i.e., linear regression. Specifically, we computed the mean responses for 24 s following the stimulus presentations, averaged across trials and across voxels in each visual area region of interest (ROI) (see Defining ROIs), but separately for each subject and separately for the top-up adapter and for each of the two probe types (high and low SF). The regression matrix had 60 columns. Column 1 took the value 1 at the onset of each top-up adapter and 0 elsewhere. Column 2 took the value 1 at one time point later (i.e., shifted down and right from column 1). Columns 2 to 20 were successive, down-rightward shifts of column 1. Column 21 took the value 1 at the onset of each adapted probe and 0 elsewhere. Columns 22 to 40 were successive, down-rightward shifts of column 21. Column 41 took the value 1 at the onset of each unadapted probe, and 0 elsewhere. Columns 42 to 60 were successive, down-rightward shifts of column 41. The mean response time courses were then computed by solving y = Ax, where A is the regression matrix, x is a vector containing the desired mean response time courses, and y is a vector containing the measured and preprocessed fMRI time series. Specifically, the pseudoinverse of the regression matrix was multiplied by the preprocessed fMRI time series to yield the mean response time courses.

Response amplitudes.

In a complementary analysis, response amplitudes were estimated, again using linear regression, but this time by adopting a parametric model, specifically, a “difference of gammas” model, of the hemodynamic response function (HRF). The HRFs were determined (see HRFs) separately for each visual area in each subject and separately for the top-up adapter and the probe stimuli. The HRF for the top-up adapter was assumed to be different from the probe HRF because the duration of the top-up adapter was longer than that of the probe (note that these are hemodynamic response functions, not hemodynamic impulse response functions, because of the relatively long durations of stimulus presentation). However, we determined only one HRF for the two probe types under the assumption that the responses evoked by the two probe types differed only in amplitude, not time course. To estimate response amplitudes evoked by the probes, we first removed the component of the measured (and preprocessed) fMRI response time series evoked by the top-up adapters. This was done by convolving the HRF corresponding to the top-up adapter with the binary time series representing the onset of each presentation of the top-up adapter and subtracting the resulting time series from the measured fMRI response time series. The result was a residual time series that reflected only the probe responses. This residual time series was multiplied by the pseudoinverse of a regression matrix. The regression matrix had two columns, constructed by convolving the probe HRF with the binary time series representing the onsets of the high and low SF probes. This yielded two probe response amplitudes. Across sessions for each subject, we estimated six mean response amplitudes corresponding to combinations of two probe types (L: low-SF, and H: high-SF) and three adapter types (L, H, or 1, for 1st-order only). We denote these mean response amplitudes by indicating both the probe and adapter types, e.g., L_AH indicates the mean response to the low-SF probe when adapted to the high-SF stimulus.

HRFs.

The measurement of response amplitudes (see Response amplitudes) required estimates of the HRFs for each visual area, separately for the top-up adapter and the probe stimuli (treating the two probe types, high- and low-SF, as having identical response time courses). We used deconvolution (Dale 1999), i.e., linear regression, to compute the mean response time courses for the 24 s following the stimulus presentations, averaged across trials. The regression matrix was like that described Response time courses but contained only 40 columns, combining the two probe types in columns 22 to 40. This procedure assumed temporal summation of the fMRI responses, but it did not assume a particular time course for the HRFs (Boynton et al. 1996). For each voxel, we then computed the goodness of fit, r², of this regression model.

The HRFs were computed separately for each voxel and then averaged across a subset of voxels in each region of interest (ROI) (see Defining ROI), but including only those voxels that exhibited robust responses to the adapter and probe stimuli. For each visual area ROI, we computed the median r² across voxels and kept those voxels for which r² exceeded the median r² value. The HRFs for the remaining voxels (with r² exceeding the median) were averaged, forming two data-derived, canonical HRFs for each ROI: one HRF for the top-up adapter, and one HRF for the probes. Finally, we fit a “difference of gammas” function to each of those HRFs, and used the best-fit parameters for the response amplitude analysis (see Response amplitudes).

Adaptation ratios.

Adaptation ratios were computed to quantify the effects of adaptation on the fMRI responses to the probes. As mentioned above, it is reasonable to expect a difference in cortical sensitivity to the two spatial frequencies even in the absence of adaptation (e.g., if more cortical neurons are selective for high than low 2nd-order SF). Hence, the adaptation ratios were computed as a ratio of the responses to the two types of probe (for a given adapter) relative to the responses to these probes when adapted to the 1st-order-only stimulus. For adaptation to the high-SF adapter, the adaptation ratio (AR) was defined as

$$A{R}_H=\frac{L_{AH}/L_{A1}}{H_{AH}/H_{A1}}.$$ (2)

Similarly, the adaptation ratio for adaptation to the low-SF adapter was

$$A{R}_L=\frac{H_{AL}/H_{A1}}{L_{AL}/L_{A1}}.$$ (3)

Absence of adaptation, or identical adaptation effects for both probes, would have resulted in an adaptation ratio of 1. To the extent that adaptation is tuned for spatial frequency, leading to a smaller response for the probe identical in frequency to the adapter, the adaptation ratio would have been >1. We also estimated a sensitivity ratio as the ratio of the response amplitudes evoked by the low- and high-SF probes following the 1st-order-only adapter:

$$SR=\frac{L_{A1}}{H_{A1}}.$$ (4)

Statistics.

We computed confidence intervals of psychophysical detection thresholds using a bootstrapping procedure (Wichmann and Hill 2001a,b). A parametric bootstrap (see Wichmann and Hill 2001a) was used to generate a simulated data set for each psychometric function, and detection thresholds were reestimated from the simulated data sets. This procedure was repeated 2,000 times providing a distribution of thresholds from which 95% confidence intervals were computed.

We estimated the SEs (in fact, 68% confidence intervals) at each time point of the deconvolved, mean fMRI response time courses using a bootstrapping procedure (Efron and Tibshirani 1998). As outlined above, the mean response time courses were computed by linear regression, y = Ax. We sampled, with replacement, contiguous rows of that system corresponding to individual runs, thus forming the system y* = A*x, and recomputed x. This procedure was repeated 2,000 times providing a distribution of mean response time courses from which the standard error at each time point was estimated. An analogous procedure was applied to put error bars on the response amplitudes for the adapted and unadapted probes.

ANOVA was used to assess the statistical significance of the sensitivity and adaptation ratios. A mixed-effects model (Snedecor and Cochrane 1967) tested for the main effects of subject (a random factor), ROI (a fixed factor), and adapter (a fixed factor), plus the interaction ROI × adapter. We used a two-tailed t-test to assess whether the sensitivity ratio was significantly different than 1. We used one-tailed t-tests to determine whether the adaptation ratios (AR_L or AR_H) were significantly larger than 1. Before ANOVA and t-testing, data were inspected for stable variance and passed a Kolmogorov-Smirnov test for normality (P > 0.05).

Defining ROIs.

Standard traveling-wave methods for retinotopic mapping (Engel et al. 1994) were used to identify the visual meridian representations in visual cortex. Those locations were then used to delineate boundaries between retinotopically organized visual areas V1, V2, V3, and V4 (e.g., Larsson and Heeger 2006; Wandell et al. 2007). The definition of human visual area V4 is controversial (Tootell and Hadjikhani 2001; Brewer et al. 2005; Hansen et al. 2007); we followed the conventions used by Wandell and colleagues (2007). Human visual area V4 has alternatively been called hV4 by some laboratories (Wandell et al. 2007), but we use the term V4 throughout this paper. These retinotopic-mapping data were acquired in separate sessions, on different days, from the main experiment (see Second-order imaging).

The visual area ROIs were restricted, based on the responses to “localizer” runs, to correspond retinotopically with the stimulus annulus in the main experiment. Each session of the main experiment included one 240-s localizer run. During the localizer, the subject fixated a central crosshairs and viewed sinusoidal gratings (1st-order, 2 cycle/°). The phases and orientations of these gratings were randomized at 4 Hz. Gratings were displayed for 12 s within the annulus while a mean-luminance field was displayed in the complement of the annulus. Then, gratings were displayed for 12 s within the complement of the annulus while a mean-luminance field was displayed in the annulus. This 24-s cycle (0.0417 Hz) was repeated 10 times. The visual area ROIs were further restricted to include only those voxels activated by stimuli presented within the annulus. For each voxel within an ROI, we computed the phase of the response at the period of stimulus alternation during the localizer run. We kept for further analysis only those voxels for which the phase of that component fell in a π/2-radian interval (centered on −2.2 radians, chosen by inspection) corresponding to the presentation of stimuli within the annulus.

MRI acquisition.

Images were acquired using a 3T Siemens Allegra scanner, a transmitter head coil (NM-011) and a four-channel phased-array receiver surface coil (NMSC-021; both manufactured by Nova Medical) positioned at the back of the head. We measured blood oxygenation level-dependent (BOLD) changes in MRI signal intensity via a standard echo planar imaging sequence with the following parameters: repetition time (TR), 1.2 s; echo time (TE), 30 ms; flip angle, 75°; 64 × 64 matrix; voxel size, 3 × 3 × 3 mm; 22 slices oriented approximately perpendicularly to the calcarine sulcus, covering the occipital lobe and part of the temporal and parietal lobes. In retinotopic-mapping sessions, we used the same imaging parameters with the following exceptions: TR, 1.5 s; flip angle, 75°; 27 slices.

At the beginning of each scanning session, we acquired an anatomical T1-weighted magnetization-prepared rapid gradient echo (MPRAGE) volume with the same slice prescription (position and orientation) as the functional volumes but with twice the in-plane resolution: TR, 1.4 s; inversion time (TI), 900 ms; TE, 3.79 ms; flip angle, 8°; voxel size, 1.5 × 1.5 × 2.0 mm. This anatomical volume was used to coregister the fMRI data with a high-resolution structural volume of each observer's brain using a robust image-registration algorithm (Nestares and Heeger 2000) to identify voxels in the fMRI data from each scanning session within each ROI.

The high-resolution anatomical volume was acquired for each subject in a separate scanning session using an MPRAGE pulse sequence: TR, 1.5 s; TI, 900 ms, TE, 3 ms, flip angle, 10°; voxel size, 1 × 1 × 1 mm. This volume was for registration across scanning sessions and also for cortical-surface extraction and flattening to visualize the retinotopic maps and define visual cortical areas.

Subjects were stabilized throughout each scanning session with foam padding and/or a bite bar.

RESULTS

Psychophysics.

Adaptation affected detection thresholds in a manner that was consistent with the FRF model. The detection thresholds of both 2nd-order low- and high-SF gratings in the presence of the 1st-order-only adapter were relatively low and comparable to each other. Thresholds were higher with 2nd-order adapters but only when the probe SF and orientation matched that of the adapter (Fig. 4). Specifically, the vertical, low-SF adapter resulted in higher detection thresholds for the vertical, low-SF probe, but not for the horizontal probe, nor for the high-SF probes. Likewise for the high-SF adapter.

Fig. 4.

Psychophysics. Psychometric functions for a typical subject (S2). A: vertical, 2nd-order, low-SF probe. Thick, black curve shows that the vertical, 2nd-order, low-SF adapter elevated the detection threshold for the vertical, 2nd-order, low-SF probe. Gray and thin, black curves show detectability of the same probe for the vertical, 2nd-order, high-SF and the 1st-order adapters, respectively. Data points are shown for 2 curves to illustrate the fitting procedure. B: vertical, 2nd-order, high-SF probe. Gray curve shows that the vertical, 2nd-order, high-SF adapter elevated the vertical, 2nd-order, high-SF detection threshold. Black and dashed curves show detectability of the same probe for the 2nd-order, low-SF and the 1st-order adapters, respectively. Data points are shown for 2 curves to illustrate the fitting procedure. C: horizontal, 2nd-order, low-SF probe. Detection thresholds for this probe were not affected by the (vertical) adapters. D: horizontal, 2nd-order, high-SF probe. Adaptation elevated detection thresholds in a 2nd-order, SF- and orientation-specific manner. Horizontal lines: 95% confidence intervals for the 75% correct detection thresholds.

fMRI.

fMRI responses were measured to the two probe SFs with a 1st-order-only adapter to determine if there was a difference in cortical sensitivity to the two 2nd-order SFs even in the absence of 2nd-order adaptation (e.g., as would be expected if more cortical neurons were selective for high than low 2nd-order SF). Figure 5, A and B, shows response time courses from an example subject (S3). To quantify any difference in the responses to the two probe SFs, a sensitivity ratio was computed as the ratio of the fMRI response amplitudes evoked by the two probes (Fig. 6A). The fMRI response amplitudes in primary visual cortex (V1) to both low- and high-SF probes were approximately equal. The sensitivity ratio for V1 did not differ significantly from 1 and likewise for V2 (V1: P = 0.37; V2: P = 0.17, two-tailed t-test). Sensitivity ratios in V3 and V4 were >1, indicating a stronger response to low 2nd-order SF (V3: P = 0.025; V4: P = 0.0046, two-tailed t-test). A mixed-effects ANOVA showed that the sensitivity ratio differed significantly across visual areas, i.e., a significant main effect of ROI (P = 0.0030).

Fig. 5.

fMRI response time courses. Deconvolved responses for a typical example subject (S3). A and B: 1st-order-only adapter. C and D: 2nd-order, low-SF adapter. A and C: V1 responses. B and D: V4 responses. Black curves show the responses to the 2nd-order, low-SF probe. Gray curves show the responses to the 2nd-order, high-SF probe. Shaded areas, SE across runs, estimated with bootstrapping (see methods).

Fig. 6.

Sensitivity ratios and adaptation ratios. A: sensitivity ratio, defined as ratio of the fMRI response amplitudes evoked by the low and high 2nd-order SF probes following the 1st-order-only adapter (see methods). Height of each bar indicates the median across hemispheres and subjects. Error bars, SE across hemispheres and subjects. *Indicates statistically significant difference from unity (P < 0.05, two-tailed t-test). B: adaptation ratios. Black bars, low-SF adaptation ratio (see methods). Gray bars, high-SF adaptation ratio. Height of each bar indicates the median across hemispheres and subjects. Error bars, SE across hemispheres and subjects. *Indicates adaptation ratios significantly >1 (P < 0.05, one-tailed t-test).

Activity in visual cortex exhibited selective adaptation to 2nd-order spatial frequency (Figs. 5, C and D, and 6B). For example, with the low-SF 2nd-order adapter, V1 responses to the low-SF probe were weaker than responses to the high-SF probe (Fig. 5C). This differential response could not be attributed to a difference in cortical sensitivity to the two 2nd-order SFs because there was no evidence for any differential response with the 1st-order-only adapter. Conversely, V1 responses to the high-SF probe were smaller than responses to the low-SF probe, with the 2nd-order high-SF adapter. Similar results were observed in each of the four visual areas: V1, V2, V3, and V4 (Fig. 6B). The results in V3 and V4 were, however, more difficult to interpret because each of those visual areas exhibited a differential response to the high- and low-SF probes even without a 2nd-order adapter (Fig. 5D). Hence, the adaptation ratios were defined relative to responses to the probes with the 1st-order-only adapter (Fig. 6B; Adaptation ratios). The 2nd-order low-SF adaptation ratio (AR_L) was significantly >1 in all four visual areas (V1: P ≈ 0; V2: P ≈ 0; V3: P = 0.0001; V4: P = 0.014, one-tailed t-test). The 2nd-order high-SF adaptation ratio (AR_H) was significantly >1 in three out of four visual areas (V1: P = 0.024; V2: P = 0.0073; V3: P = 0.0084; V4: P = 0.0504, one-tailed t-test). Adaptation to the low-SF adapter was stronger than to the high-SF adapter in all four visual areas (Fig. 6B). A mixed-effects ANOVA showed a main effect of adapter (that is, the 2nd-order, low- vs. high-SF adapter: P = 0.0002), but no evidence of any difference across visual areas (P = 0.77), nor was there evidence of an interaction between visual area and adapter SF (P = 0.23).

Second-order modeling.

Surround suppression might have contributed to our finding of greater adaptation to low- than to high-SF adapters. The responses of a V1 neuron to a preferred stimulus are diminished by the simultaneous presentation of a second, surrounding stimulus that is ineffective in driving the cell when presented alone (see Introduction). This effect is orientation-specific, i.e., when the stimulus in the surround has a different orientation from the probe, the suppressive effect is reduced (Blakemore and Tobin 1972; Knierim and Van Essen 1992; Deangelis et al. 1994; Cavanaugh et al. 2002a,b). Because of the contrast and orientation dependence of surround suppression, we would expect the responses of these neurons to depend on the 2nd-order SF of contrast- or orientation-modulated stimuli. As the 2nd-order SF of an orientation-defined texture is varied, the surround context of neurons with receptive fields centered in one stripe of the pattern varies (Fig. 7, A and B).

Fig. 7.

Model. A: first-order mechanism, with surround suppression, comprising an excitatory filter tuned to the 45° carrier (solid circle) and similarly tuned, suppressive surrounds (dashed circles). Mechanism is superimposed on a 2nd-order, low-SF grating (1 × 1° detail from Fig. 1A; reduced 1st-order contrast for clarity). Bars and circles indicate orientation and SF preference, respectively. Mechanism's response is suppressed because the center and the surround see their preferred stimulus. B: same as in A but the mechanism is superimposed on a 2nd-order, high-SF grating. There is less suppression in B than in A because the surround sees the nonpreferred (perpendicular) orientation (Cavanaugh et al. 2002a,b). C: measured and modeled sensitivity and adaptation ratios. V1 data are reproduced from Fig. 6. Model parameters: full-scale adaptation, α_F = 0.18; surround suppression, G_S = 0.66 ; 2nd-order population sizes, G_L = 0.93 and G_H = 0.22 (see text for details).

Suppose that we incorporate a stage of normalization into the FRF model, resulting in FRNF. The responses will be 2nd-order spatial-frequency dependent because of both the 1st-stage normalization (divisive suppression) and the inhibitory regions in the 2nd-stage linear filter (subtractive inhibition). It is nontrivial to distinguish these two effects, and surround suppression in V1 may contribute to the perception of 2nd-order structure.

We therefore developed a model, described in detail in the appendix, to predict the measured sensitivity and adaptation ratios. The model assumes that the fMRI responses in V1 are the sum of the activity of two types of neurons: 1st- and 2nd-order. The parameters G_L and G_H denote the relative fMRI responses to low and high 2nd-order SF stimuli, combining effects of relative neural population size and responsiveness. The model also includes surround suppression, as illustrated in Fig. 7. In the model, surround suppression attenuates 1st-order responses by a factor G_S, but only for stimuli with low 2nd-order spatial frequency (Fig. 7A); for the 2nd-order, low-SF grating, the surround is stimulated by the orientation that the surround prefers. In the model, both 1st- and 2nd-order neurons are subject to adaptation, which attenuates responses by an additional gain factor. This gain factor is a function of the effective contrast (C_eff) of the adapter:

$$A\left(C_{eff}\right)=e^{-\alpha C_{eff}}$$ (5)

For full-contrast adapter stimuli, attenuation is A(1) = α_F = exp(−α), but is reduced to A(G_S) when the adapting stimulus undergoes surround suppression. The resulting model predictions (see appendix) are:

$$SR=\frac{G_S\left(1+G_L\right)}{1+G_H},$$ (6)

$$A{R}_1=\frac{1+G_L}{1+G_{L}A\left(G_S\right)},$$ (7)

and

$$A{R}_H=\frac{1+G_H}{1+G_{H\alpha F}}.$$ (8)

By hand, we found a set of parameters for the model that yielded a good prediction of the measured results (Fig. 7C). In V1, we measured a 1st-order sensitivity ratio (SR) ≈ 1. Given that by definition G_S < 1, it follows that in V1 there exists a relatively large population of neurons selective for low 2nd-order SF, that is, G_L > G_H. Also, by definition α_F < A(G_S) < 1, so that both AR_L > 1 and AR_H > 1.

In our model, both surround suppression and an abundance of 2nd-order, low-SF-tuned neurons are required to explain our fMRI responses (Fig. 7). If we remove either feature from the model, the resulting model fails to account for our data. First, consider the behavior of the model if we make the two 2nd-order population sizes equal, that is, we set G_L = G_H. In our data, AR_L ≠ AR_H. Comparing Eqs. 7 and 8, these two statements imply that α_F ≠ A(G_S) and hence by Eq. 5 that G_S < 1. Finally, this implies that SR < 1, which is inconsistent with our data. Second, suppose instead we remove the effect of surround suppression, that is, we set G_S = 1. This, coupled with the measured sensitivity ratio of 1, implies that G_L = G_H (Eq. 5). However, if G_L = G_H and G_S = 1, then A(G_S) = α_F (Eq. 5) and hence AR_L = AR_H (comparing Eqs. 7 and 8). This is inconsistent with the data. Thus neither surround suppression nor differing population sizes alone can account for the data. On the other hand, with a combination of surround suppression and an abundance of 2nd-order, low-SF-tuned neurons, SR can remain at unity, while AR_L and AR_H can range independently above unity.

How might the model, which accounts for our V1 data, be extended to account for the data from extrastriate areas? If 2nd-order responses in V2, V3 and V4 are inherited from V1 inputs (directly, or indirectly), with no further effects of surround suppression or adaptation, then adaptation ratios in these downstream areas would be identical to those in V1, consistent with our data (Fig. 6B). However, the measured SR differed across visual areas. The ratio was indistinguishable from unity in V1 and V2 but significantly greater than unity in V3 and V4 (Fig. 6A). Since the adaptation ratios use the SR to account for any differing sensitivity to 2nd-order SF modulations in the absence of adaptation, we conclude that adaptation measured in areas V2, V3, and V4 is a consequence of V1 adaptation, and we conclude that adaptation in those higher tier areas does not reflect further 2nd-order processing. However, unlike V1, areas V3 and V4 have stronger responses to low 2nd-order SF stimuli, due to either higher gain and/or more neurons tuned to lower 2nd-order SFs.

DISCUSSION

We used an fMRI-adaptation protocol to measure activity in visual cortex to 2nd-order SF modulation. We found that visual cortical fMRI responses were not well described by the standard FRF model but rather by an FRNF model that included normalization. The fMRI measurements yielded three main results. First, we found evidence for the representation of 2nd-order stimulus modulations in early visual cortex, in fact, as early as primary visual cortex (V1). Second, the effect of each adapter was about the same in areas V1, V2, V3, and V4. Third, the 2nd-order, low-SF adapter elicited more adaptation than the 2nd-order, high-SF adapter. These results were interpreted through model simulations. We found that an FRF model involving both 1st- and 2nd-order SF- and orientation-selective filters did not by itself explain the fMRI responses. Therefore, we developed an FRNF model that included surround suppression among 1st-order filters. In doing so, we brought together two computational theories of the visual cortex: the FRF computation, which was primarily inferred from psychophysical measurements, and normalization, which was primarily developed to model responses of V1 neurons. Our modeling led to the following conclusions. First, both surround suppression (normalization) and 2nd-stage filtering contribute to the selectivity for 2nd-order SF, thereby replacing the FRF model with a FRNF model. Second, the population of neurons selective for low 2nd-order SF is larger than that selective for high 2nd-order SF.

Our psychophysical results are consistent with the FRNF model (with the added normalization stage). The characteristics of 1st-order channels are well understood, but there is less consensus on the characteristics of 2nd-order channels. The 2nd-stage filter (that is, the “2nd F” of FRNF) is hypothesized to be both orientation and SF tuned. Our results suggest that the SF tuning is sufficiently narrow such that our 2nd-order, low- and high-SF stimuli (0.18 and 1.25 cycle/°, respectively) were processed by separate mechanisms.

Our imaging data provide evidence for the representation of 2nd-order SF modulation in V1. This result is surprising because it is not predicted by current, “standard” models of V1 processing (Rust and Movshon 2005). V1 simple cells are typically characterized as performing a localized, spatiotemporal linear filtering computation followed some form of rectification (e.g., halfwave rectification and squaring). V1 complex cells are typically characterized as spatiotemporal energy mechanisms that compute a sum of the squared responses of a quadrature pair of spatiotemporal linear filters. Both the simple- and complex-cell models include normalization that accounts for cross-orientation inhibition and surround suppression. However, none of these computations, as they are presently understood, can account for the selectivity we observed in V1 to 2nd-order, orientation-defined texture modulation. Hence, we infer that subpopulations of V1 neurons also perform a 2nd-stage filtering operation (the “2nd F” in FRNF).

Our study is not the first to find evidence of V1 processing that goes beyond the “standard” models of simple and complex cells outlined above. A small population of cells in macaque visual cortex (V1: 4% of neurons isolated; V2: 1.6%) have been proposed to subserve texture perception (von der Heydt et al. 1992; Petkov and Kruizinga 1997; Grigorescu et al. 2002). These cells respond to spatial patterns but not to the Fourier components thereof, contrary to predictions of the “standard” models. Some complex cells (∼50% of those isolated) in V1 of the anesthetized macaque exhibit orientation-selective responses to illusory contours (Grosof et al. 1993), and both the local field potential and single-unit activity in macaque V1 signal high-order spatial correlations (Purpura et al. 1994), again contrary to the predictions of the “standard” models. Some neurons in area 18 of the anesthetized cat have been reported to respond selectively to 2nd-order orientation (Mareschal and Baker 1998; Baker and Song 2008). Although primate-feline comparison is not straightforward, it is worth noting that the cat's area 18 receives input from the lateral geniculate nucleus and, in this regard, is a first stage of cortical processing (along with area 17). The above-mentioned studies, taken together with ours, suggest that V1 is equipped with computations for representing object boundaries defined by 2nd-order stimulus modulations, not just modulations of luminance (Shapley 1998).

Our results are similar to two previous fMRI studies that used adaptation to characterize selectivity for higher-order stimulus modulations (Larsson et al. 2006; Montaser-Kouhsari et al. 2007). These studies reported, consistent with our current results, orientation-selective adaptation in early visual cortex for luminance (1st-order) modulations, contrast (2nd-order) modulations, orientation (2nd-order) modulations, and illusory contours. However, both of these previous studies reported that selectivity for higher-order stimulus modulations was stronger in higher visual areas whereas our adaptation ratios, AR_L and AR_H, did not vary across visual areas V1-V4. Specifically, Montaser-Kouhsari et al. reported that adaptation was significantly stronger in V3 and V4 than in V1 and V2, whereas Larsson et al. reported only a trend (not statistically significant) for progressively stronger adaptation from V1 to V4. Moreover, neither of these previous studies provided evidence in favor of the FRNF model instead of the FRF model.

Previous psychophysical observations provide indirect support for the conclusion that surround suppression affects the processing of 2nd-order stimulus modulations. First, for 2nd-order contrast modulation wherein the carrier is oriented, bandpass-filtered noise, sensitivity is affected by the relative orientations of the carrier and the modulator. When the carrier and modulator orientations are equal, sensitivity is reduced (Dakin and Mareschal 2000). For that stimulus configuration, the receptive fields of 1st-order, V1 neurons and their surrounding ends (lying along the axis of preferred orientation) receive similar input. This stimulus configuration has been shown to produce greater suppression in 1st-order, V1 neurons of the macaque (Cavanaugh et al. 2002b) and, since 2nd-order neurons are driven by 1st-order input, should thus reduce both the output of 2nd-order neurons and observer sensitivity. When the carrier and modulator orientations differ by 90°, sensitivity is increased (Dakin and Mareschal 2000). That stimulus configuration produces less suppression in 1st-order, V1 neurons of the macaque (Cavanaugh et al. 2002b) and should leave 2nd-order neurons relatively unaffected, thus restoring observer sensitivity. Furthermore, this sensitivity difference itself is reduced for carriers at higher SFs (Dakin and Mareschal 2000). For high SFs, both stimuli drive the area surrounding the receptive field in almost identical ways. A second psychophysical observation providing support for the conclusion that surround suppression contributes to 2nd-order processing is that sensitivity to 2nd-order, contrast modulation has been found to increase only gradually as a function of carrier contrast (Schofield and Georgeson 1999). This gradual increase may be a consequence of surround suppression amongst 1st-order neurons. At higher contrast, the relative effect of surround suppression itself is greater (Cavanaugh et al. 2002a). It follows that, as the carrier contrast in a 2nd-order grating increases, the output of 1st-order, V1 neurons increases, but the rate of that increase is reduced by the increasing effect of surround suppression. Since 2nd-order neurons are driven by 1st-order input, 2nd-order output and observer sensitivity are a shallow function of carrier contrast.

Our model suggests that V1 comprises a relatively large population of 2nd-order, low-SF neurons and a smaller, high-SF population. This difference might reflect the statistics of natural scenes. The statistics of 2nd-order stimulus modulations in natural scenes have not been fully characterized (Johnson and Baker 2004); we hypothesize based on our results that they tend to occur at low SFs. A relatively large population of 2nd-order, low-SF neurons predicts heightened psychophysical sensitivity to 2nd-order, low-SF modulations. However, that prediction is ostensibly at odds with the psychophysical literature; human 2nd-order sensitivity is approximately constant across the range of SFs (0.18 to 1.25 cycle/°) that we examined (Landy and Oruç 2002). A potential explanation for this discrepancy is that surround suppression and a large population of 2nd-order, low-SF neurons effectively compensate for one another. This suppression may serve an important computational function that is intermediate to the signaling of 2nd-order features. For example, normalization is thought to enable invariant 1st-order orientation- and SF-tuning bandwidths despite variations in 1st-order contrast (Heeger 1992b). Invariance such as this could ensure that the 2nd-stage of the model maintains sensitivity to 2nd-order features despite variations in 1st-order contrast. Consider the response of a population of normalized, 1st-order neurons to our 2nd-order, low-SF stimulus (see Fig. 7A). The responses of that population are suppressed (due to normalization), so when they drive a population of hypothesized, 2nd-order neurons, the output of that 2nd-order population is in turn suppressed. This predicts lower psychophysical sensitivity. However, sensitivity could be restored by increasing the size of that 2nd-order population. Indeed, this compensation is predicted by our model.

GRANTS

This work was supported by National Eye Institute Grant EY-016165.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

APPENDIX

In this appendix, we outline a simple model of fMRI responses to 2nd-order probe stimuli following adaptation. Table 1 lists the parameters of the model, and Table 2 gives the model predictions for each component of fMRI responses to each type of stimulus used in this study.

Table 1.

Model parameters

Parameter	Definition
R1	Full 1st-order response
GL	Gain factor for 2nd-order response to low 2nd-order spatial frequency stimulus
GH	Gain factor for 2nd-order response to high 2nd-order spatial frequency stimulus
αF	Maximum attenuation due to adaptation
A(Ceff)	Adaptation in response to effective adaptation contrast Ceff
GS	Gain factor due to surround suppression

Table 2.

Model responses

Stimulus Condition	1st-Order Response	2nd-Order Response
HA1	R1αF	GH(R1αF)
LA1	R1αFGS	GL(R1αFGS)
HAH	R1αF	GHαF(R1αF)
LAH	R1αFGS	GL(R1αFGS)
HAL	R1A(1 × GS)	GH[R1A(1 × GS)]
LAL	R1A(1 × GS)GS	GLA(1 × GS)[R1A(1 × GS)GS]

The response from V1 to a full-contrast 1st-order pattern is R₁ in the absence of any adaptation or surround suppression. Adaptation can reduce both 1st- and 2nd-order responses by a factor defined as

$$A\left(C_{eff}\right)=e^{-\alpha C_{eff}},$$ (A1)

where C_eff is the effective adapter contrast. For 1st-order adaptation, C_eff is the product of the stimulus root-mean-square 1st-order contrast of the adapter (a constant in our experiments) and any attenuation due to surround suppression. For 2nd-order adaptation, C_eff includes an additional factor of the 2nd-order contrast. Maximal adaptation, when C_eff = 1, attenuates responses by a factor of α_F = exp(−α).

First-order responses can also be attenuated by surround suppression. Surround suppression is most effective when stimulation outside the classical receptive field is similar (in spatial frequency and orientation) to the preferred stimulus of the neuron (Blakemore and Tobin 1972; Knierim and Van Essen 1992; Deangelis et al. 1994; Cavanaugh et al. 2002a,b). In our experiment, surround suppression is most effective for adapter and probe stimuli having low 2nd-order spatial frequency, and hence we include an attenuation factor (G_S) for responses to those stimuli.

Second-order neurons are driven by 1st-order input, and thus their responses include a factor identical to the estimated 1st-order response. In the absence of any 2nd-order adaptation, this input drive is amplified by a factor G_L or G_H (for low or high 2nd-order spatial frequency stimuli, respectively) that takes into account the relative number of 2nd-order neurons and their responsivity. Thus each entry in Table 2 for 2nd-order responses includes a factor (in parentheses) equal to the 1st-order drive, multiplied by the above amplification factor as well as a factor due to any 2nd-order adaptation effect. The fMRI response to a stimulus is then taken to be the sum of 1st- and 2nd-order responses.

In Table 2, most 1st-order responses include an adaptation factor of α_F, i.e., A(1), due to the high 1st-order contrast of the adapter. We treat this effective contrast as identical for the 1st-order and high-SF adapters, because phase was jittered throughout, and average local root-mean-square contrast was approximately the same for these two stimuli. However, the low-SF adapters underwent surround suppression, reducing 1st-order adaptation effects for these stimuli to A(cG_S), where c = 1 (100% 1st-order contrast) in our experiments. The 2nd-order responses show an additional adaptation factor when the adapter and probe shared the same 2nd-order spatial frequency. This factor was α_F for high SF but was reduced to A(cG_S) for low SF due to the 1st-order surround suppression during adaptation.

Deriving predictions from the model is straightforward. The sensitivity ratio is

$$SR=\frac{L_{Al}}{H_{A1}}=\frac{R_1{\alpha }_F{G}_S+G_L{R}_1{\alpha }_F{G}_S}{R_1{\alpha }_F+G_H{R}_1{\alpha }_F}=\frac{G_S\left(1+G_L\right)}{1+G_H}.$$ (A2)

After algebraic simplification, the adaptation ratios turn out to be

$$A{R}_L=\frac{H_{AL}/H_{A1}}{L_{AL}/L_{A1}}=\frac{1+G_L}{1+G_{L}A\left(G_S\right)}$$ (A3)

and

$$A{R}_H=\frac{L_{AH}/L_{A1}}{H_{AH}/L_{A1}}=\frac{1+G_H}{1+G_H{\alpha }_F}.$$ (A4)