Orientation Selectivity of Motion-Boundary Responses in Human Visual Cortex

Abstract

Motion boundaries (local changes in visual motion direction) arise naturally when objects move relative to an observer. In human visual cortex, neuroimaging studies have identified a region (the kinetic occipital area [KO]) that responds more strongly to motion-boundary stimuli than to transparent-motion stimuli. However, some functional magnetic resonance imaging (fMRI) studies suggest that KO may encompass multiple visual areas and single-unit studies in macaque visual cortex have identified neurons selective for motion-boundary orientation in areas V2, V3, and V4, implying that motion-boundary selectivity may not be restricted to a single area. It is not known whether fMRI responses to motion boundaries are selective for motion-boundary orientation, as would be expected if these responses reflected the population activity of motion-boundary–selective neurons. We used an event-related fMRI adaptation protocol to measure orientation-selective responses to motion boundaries in human visual cortex. On each trial, we measured the response to a probe stimulus presented after an adapter stimulus (a vertical or horizontal motion-boundary grating). The probe stimulus was either a motion-boundary grating oriented parallel or orthogonal to the adapter stimulus or a transparent-motion stimulus. Orientation-selective adaptation for motion boundaries—smaller responses for trials in which test and adapter stimuli were parallel to each other—was observed in multiple extrastriate visual areas. The strongest adaptation, relative to the unadapted responses, was found in V3A, V3B, LO1, LO2, and V7. Most of the visual areas that exhibited orientation-selective adaptation in our data also showed response preference for motion boundaries over transparent motion, indicating that most of the human visual areas previously shown to respond to motion boundaries are also selective for motion-boundary orientation. These results suggest that neurons selective for motion-boundary orientation are distributed across multiple human visual cortical areas and argue against the existence of a single region or area specialized for motion-boundary processing.

INTRODUCTION

Motion, or kinetic, boundaries (local changes in visual motion direction) are ubiquitous features of natural scenes that arise along the occlusion boundaries of objects moving relative to an observer. Humans and other primates are highly sensitive to motion boundaries and many studies have sought to identify the neuronal mechanisms underlying this ability. Single-unit recordings in macaque visual cortex (Marcar et al. 1995, 2000; Mysore et al. 2006; Sáry et al. 1995; Zeki et al. 2003) have identified neurons that respond selectively to orientations of motion boundaries both in early visual areas (e.g., V2 and V3) and in higher extrastriate visual areas (V3A, V4, and IT), with the highest proportion of such neurons located in dorsolateral visual areas V3 and V3A. In human visual cortex, early neuroimaging studies identified a region of visual cortex dorsal and lateral to V3 that responds more strongly to random-dot stimuli containing motion boundaries than to transparent-motion control stimuli (Dupont et al. 1997; Van Oostende et al. 1997). This region was labeled the “kinetic occipital area” or KO. Later studies suggested that KO is not a single entity, but extends over multiple retinotopic visual areas including V3, V3A, and V3B (Larsson and Heeger 2006), and that many visual areas outside of this region (such as V2 and hV4) show a stronger response to motion-boundary stimuli over transparent-motion stimuli (Larsson and Heeger 2006; Tyler et al. 2006; Zeki et al. 2003). Superficially these findings agree well with those from monkey electrophysiology; in both species responses to motion boundaries are found in multiple visual areas, but are most pronounced in V3, V3A, and neighboring parts of dorsolateral visual cortex.

A direct comparison between the findings from neuroimaging and electrophysiology is complicated by differences in the key properties used to identify neuronal responses to motion boundaries by the two techniques. Single-unit studies have identified motion-boundary–selective neurons by measuring orientation selectivity for motion boundaries. In contrast, neuroimaging studies have identified areas showing a response preference (stronger response) for motion-boundary stimuli relative to control stimuli containing transparent motion. It is not obvious that orientation selectivity for motion boundaries need be associated with a stronger response to motion boundaries than to transparent motion, for three reasons. First, although the transparent-motion stimuli used as controls contain the same average motion components as those of motion-boundary stimuli, the spatial distributions of these components differ between the stimuli, implying that the distribution and total local motion energy (both of which are known to influence the responses of neurons selective for visual motion) are not well matched across the two stimuli. Second, transparent-motion stimuli are associated with strong perceptual effects not found in motion-boundary stimuli (e.g., observers often perceive pronounced bistability in the former, with the two transparent surfaces alternating in perceived depth); such perceptual effects are likely to be associated with differences in overall response that are unrelated to the processing of motion boundaries per se. Third, even assuming that transparent-motion stimuli are sufficiently well matched controls, neurons that respond to motion boundaries but with weak or no orientation tuning—which would not be classified as motion-boundary–selective by the criteria used in single-unit studies—would also respond more strongly to the motion-boundary stimuli than to the transparent-motion controls. Thus it is possible that some or all of the motion-boundary–preferring areas identified with neuroimaging are in fact not selective for motion-boundary orientation.

In this study we used functional magnetic resonance imaging (fMRI) adaptation techniques to identify neuronal populations selective for motion-boundary orientation in human visual cortex. Specifically we sought to determine 1) which human visual areas are selective for motion-boundary orientation and to what degree and 2) whether the degree of orientation selectivity in each area is related to the degree of motion-boundary preference (defined as stronger responses to motion-boundary stimuli than to transparent-motion control stimuli). The results show a relatively high degree of correspondence between motion-boundary preference and orientation selectivity for motion boundaries, suggesting that most areas that respond to motion boundaries are also selective for motion-boundary orientation.

METHODS

Subjects

Five subjects, between 32 and 49 years of age, took part in the experiment. Subjects gave informed consent to participate in accordance with the Helsinki convention. The experiments were approved by the University Committee on Activities Involving Human Subjects of New York University. The experiments were undertaken in compliance with the safety guidelines for magnetic resonance imaging (Kanal et al. 2002).

Stimuli and experimental conditions

Stimuli were back-projected on a rear projection screen using an EIKI LC-XG100 liquid crystal display projector (Eiki Industrial, Osaka, Japan) at a refresh rate of 60 Hz. Subjects viewed the screen at a viewing distance of 57 cm through a front-silvered mirror. Motion-boundary stimuli were designed to be similar to those used in previous studies (e.g., Van Oostende et al. 1997) and consisted of horizontal or vertical motion-boundary grating patterns within an annular aperture (inner diameter 1.5°; outer diameter 10°) around the center of fixation, displayed against a uniform gray background (Fig. 1A). The motion-boundary gratings were composed of 2,000 random white 0.1° dots moving diagonally at a speed of 3°/s in 1.3°-wide parallel horizontal or vertical strips, the direction of motion reversing 180° from one strip to another, resulting in vivid motion (kinetic) boundaries between the strips. Individual dots had a lifetime of 0.42 s (corresponding to the time taken for a single dot to traverse 70% of the width of a single motion strip). The spatial phase of the gratings changed randomly every 0.5 s. Concurrently with the phase changes, the axis of motion alternated between oblique right (45°/225°) and oblique left (135°/315°). The transparent-motion stimuli contained the same local motion components as those of the motion-boundary stimuli, but dots moving in the same direction were distributed uniformly across the stimulus rather than being segregated into strips, resulting in the percept of two superimposed transparent surfaces moving in opposite directions. As with the motion-boundary stimuli, the direction of motion alternated every 0.5 s.

Fig. 1.

Experimental design. A: schematic representation of stimulus conditions. Stimuli were presented in an annulus (inner diameter 1.5°; outer diameter 10°) around fixation. White arrows indicate direction of local dot motion; black dotted lines indicate location of motion-defined boundaries (arrows and lines were not present in the actual stimuli and the relative size of dots has been exaggerated for visualization purposes). In parallel trials (PARA), adapter and probe stimuli were motion-defined gratings with the same orientation. In orthogonal trials (ORTHO), adapter and probe stimuli were orthogonal motion-defined gratings. In transparent-motion control trials (TRANS), adapter stimuli were the same as those in ORTHO and PARA, but probe stimuli were transparent-motion patterns with the same local motion components as those of the motion-defined gratings. B: trial structure. On each trial, the adapter stimulus was shown for 4 s, followed after a 1-s interstimulus interval (stimulus annulus replaced by gray background) by the probe stimulus for 1 s. Throughout the experiment and concurrent with the adapter and probe stimuli, subjects performed an attention-demanding counting task at fixation (counting number of target letters “X” in a rapid stream of distractor letters).

For the main experiment (referred to as the “adaptation scans” throughout), an event-related experimental design was used. On each trial, an adapting stimulus (a vertical or horizontal motion-boundary grating pattern) was shown for 4 s. After an interstimulus interval of 1 s, a probe stimulus was shown for 1 s (Fig. 1B). Three types of probe stimuli were used: 1) motion-boundary gratings oriented parallel to the adapting stimulus (PARA trials), 2) motion-boundary gratings oriented orthogonal to the adapting stimulus (ORTHO trials), or 3) transparent-motion stimuli (TRANS trials). After the probe stimulus was shown, the screen remained blank (except for the fixation stimulus described in the following paragraph) for an interval that varied randomly between 1.2 and 4.8 s in increments of 1.2 s, such that all trial onsets were aligned with the scanner image acquisition (repetition time [TR] = 1.2 s; see MRI acquisition in the following text). Each experimental run (scan) consisted of 36 trials (12 of each type) and lasted for 324 s. Trial order was randomly permuted within each block of six trials, such that each trial type was equally likely to be preceded by any other trial type. Two adapter orientations (horizontal or vertical) were used in separate sessions. Each session (i.e., each adapter orientation) consisted of eight scans, yielding a total of 96 trials per trial type and subject.

Concurrent with the adaptation trials, subjects performed an attention-demanding rapid serial visual presentation (RSVP) task at the center of fixation that required counting the number of target letters “X” that appeared at random intervals in a rapid stream of distractor letters (Z, N, L, T) (Larsson et al. 2006). Each letter was presented in yellow at the center of the screen for 160 ms. Shortly (1.2 s) before the beginning of the next trial (i.e., the onset of the adapter), a yellow fixation cross was shown, prompting subjects to respond with respect to how many targets (0–4) they had seen. Visual feedback was provided by changing the color of the fixation cross to green for correct responses and red for incorrect responses. Each RSVP trial began with the onset of the adapting stimulus and ended just before the onset of the next adapting stimulus, although otherwise the timing of the task and stimulus presentation was independent. Because the RSVP task was identical across trial types and required subjects to divert attention away from the stimuli, it served to control and equate spatial attentional load across trial types.

fMRI preprocessing

Functional image volumes acquired at different time points were spatially aligned using motion-correction software (FSL). Through the use of custom software, data for each scanning session were aligned across sessions by coregistering them with high-resolution anatomical MR images of each subject's brain (Nestares and Heeger 2000). Cortical surface models of each individual subject's brain (used for visualization and visual-area identification) were reconstructed from the high-resolution anatomical MR images using the public domain software SurfRelax (Larsson 2001).

fMRI data analysis

Data from the adaptation scans were analyzed separately for individual visual areas (see Identification of visual cortical areas in the following text) for each scanning session in three steps. First, to visualize the time course of stimulus-evoked responses, the average response time courses to the adapter and probe stimuli for each of the three trial types (PARA, ORTHO, and TRANS) were estimated by linear deconvolution. Second, the response amplitudes to individual probe stimuli were estimated by a general linear model by using estimates of the hemodynamic impulse response function (HIRF) to model the fMRI response to each probe stimulus separately. These response amplitudes were used to compute the statistical significance of response adaptation (reduction in response amplitude after adaptation) and response preference (greater response to motion boundaries than that to transparent motion). Third, to compare the degree of response adaptation and response preference across visual areas that differed greatly in absolute response magnitudes, indexes quantifying the relative amount of response adaptation and response preference, respectively, were computed for each visual area, using the mean response amplitudes for each probe stimulus type.

Response time courses

Average event-related response time courses for each trial type were estimated by modeling the fMRI responses as a linear system (Burock and Dale 2000)

$$Y=X_1{b}_1$$ (1)

where Y is the M long vector of detrended and high-pass filtered fMRI response time courses averaged across voxels for each visual area region of interest (ROI) and concatenated across runs (where M = 2,160 is the total number of time points or volumes acquired across all adaptation scans). X₁ is an [M × (1 + 3N)] design matrix (N = 16, the number of time points in the event-related average) and b₁ is a (1 + 3N) long column vector corresponding to the beta weights for each time point and trial type. The first column (coded as 1's) in the design matrix X₁ and the first row of the beta weight vector b₁ modeled the mean response, to account for any round-off errors in the detrending operation. Columns 2 … N + 1 modeled the responses to the PARA trials as follows. Column 2 had a value of 1 at the onset of each PARA trial and zero elsewhere; this column modeled the response immediately following the onset of the trial. Column 3 was a copy of column 2 but shifted one row down; this column modeled the response one time point later. Each of the subsequent columns (4 … N + 1) consisted of a copy of the immediately preceding column shifted down one row (with the exception that events did not extend across runs or sessions). Together these 16 columns modeled the responses to the 16 time points (0–19.2 s) following the onset of each PARA trial. For each of these columns, the corresponding row in the beta weight vector represented the mean response amplitude at that time point following the onset of a particular trial type. Responses to the ORTHO and TRANS trials were modeled in the same fashion using the remaining 2N columns of the design matrix and the last 2N rows of the beta weights vector. We computed a least-squares estimate of the beta weights vector ${\widehat{b}}_1$ using linear regression

$${\widehat{b}}_1={\left(X_1^T{X}_1\right)}^{-1}{X}_1^{T}Y$$ (2)

Estimation of hemodynamic impulse response function

HIRFs were estimated separately for each scanning session and used to measure response amplitudes to individual adapter and probe stimuli (see Response amplitudes in the following text). We estimated the HIRF for each visual area ROI separately using linear deconvolution analogous to that used to measure average response time courses to the probe stimuli (Burock and Dale 2000). This analysis was also based on the data from the main (adaptation) experiment, but treated all stimulus presentations identically regardless of the type of probe stimulus or trial type. The HIRF was estimated as the fMRI response to a single stimulus impulse by treating trials as consisting of a series of identical stimulus impulses (or brief events) and modeling the total time-varying response to the stimuli as the sum of scaled and time-shifted responses to all stimulus impulses. The response vector Y was again modeled as a linear system Y = X₂b₂. Here the columns of the [M × (1 + N)] design matrix X₂ represented the responses to each of the N = 16 time points (0–18 s) following each stimulus presentation (adapter and probe, irrespective of trial type). The first column of this design matrix (all 1's) estimated the mean or baseline response. The second column had a value of 1 at each time point when an adapter or probe stimulus was present on the screen and a value of 0 elsewhere. Each of the subsequent N − 1 columns was a copy of the immediately preceding column, shifted one row (time point) down (except responses were not modeled as extending across runs or sessions). These columns modeled the responses to the stimuli at each of the N − 1 time points following stimulus onset. The HIRF was estimated as the N regression coefficients (beta weights) corresponding to the N rightmost columns of the design matrix, computed by least-squares regression as in Eq. 2.

Although HIRFs differed across subjects and scanning session, within each session the HIRFs for different visual areas were very similar in shape and temporal characteristics (time to peak, time to undershoot, ratio of undershoot to peak response). However, HIRF estimates were noisier for higher extrastriate areas with weaker responses. For this reason, we selected the seven visual areas that showed the most reliable responses across sessions and subjects (V1–V3, V4, V3A, LO1, and V7). For each individual scanning session, the HIRFs of these seven areas were averaged and weighted by response amplitude; the resulting average HIRF was then used in subsequent analysis of data from that session. Although this procedure might have caused the analysis to be most sensitive in early visual areas with stronger responses (and thus more reliable HIRF estimates), the lack of significant adaptation in V1 (see results) suggests such bias had little influence on the results. Importantly, because the estimated HIRFs would have been dominated by the adapter response (which was identical across trials) and the different probe types were presented equally often, the procedure would not have biased the analysis to be more sensitive for any trial type. The average HIRFs were very similar in shape to previously published estimates of hemodynamic impulse responses (Boynton et al. 1996; Glover 1999). To avoid overfitting, we therefore fit the average HIRFs with a synthetic HIRF (a difference of two gammas) and used the latter for estimating response amplitudes in the main experiment.

Response amplitudes

To estimate the amplitudes of individual probe responses, we again modeled the fMRI responses as a linear system, this time modeling the fMRI response to each individual probe stimulus separately and using the estimate of the HIRF described earlier

$$Y=X_3{b}_3$$ (3)

The first column of the [M × (2 + 3P)] design matrix X₃ represented the baseline response (mean) as in Eq. 1. The second column represented the response to all of the adapter stimuli. This column consisted of a vector of 1's for every time point when the adapter stimulus was shown (thus modeling both the onset and duration of the adapter) and a zero elsewhere, convolved with the HIRF. Each of the subsequent 3P columns corresponded to the response to a single probe stimulus presentation, modeled by convolving the estimated HIRF with a delta function vector having a value of 1 at the onset time of the probe stimulus and 0 elsewhere (P = 96, the number of single probe stimulus presentations per trial type and scanning session). A least-squares estimate of the beta weights vector b̂₃, corresponding to the estimated response amplitudes for each individual probe stimulus, was computed by linear regression as in Eq. 2.

The response amplitudes for each trial type were averaged separately for each subject, session, and visual area ROI. Paired t-tests (across all 10 scanning sessions) were then applied to identify areas with response amplitudes significantly greater than zero and to test for significant differences between trial types. Significance estimates were corrected for multiple comparisons across ROIs using a false discovery rate, with α = 0.05 (Benjamini and Hochberg 1995). In addition, the statistical significance of within-subject differences in response amplitudes was assessed with a randomization test based on iterative resampling as follows. For each subject, response amplitudes for individual trials were pooled across adapter orientations (vertical and horizontal) separately for each probe stimulus type. From these pooled response amplitudes we computed the response adaptation magnitude (the difference between the mean responses to orthogonal and parallel probes) and response preference magnitude (the difference between the mean responses to orthogonal and transparent-motion probes) for each ROI. The significance of these differences was assessed by testing the hypothesis that the response amplitudes for the two trial types (orthogonal and parallel or orthogonal and transparent) were different, against the null hypothesis that they were the same. On each resampling iteration, individual response amplitudes for the two trial types being compared were assigned randomly to either trial type (with the constraint that the same number of amplitudes were assigned to each trial type as in the original data) to generate two resampled null distributions. The difference between the mean response amplitudes for each resampled distribution was computed and the process was repeated 1,000 times. The P value of the measured difference in response amplitudes was then computed as the proportion of the resampled amplitude differences that were equal to or greater than the measured difference.

Orientation-selectivity index

For each visual area ROI we computed an orientation-selectivity index (OI) that quantified the magnitude of orientation-selective response adaptation (the difference in response to probes parallel to the adapter stimulus compared with the response to probes orthogonal to the adapter) as a proportion of the absolute summed response to both probes

$$OI=\frac{R_O-R_P}{\left|R_O\right|+\left|R_P\right|}$$ (4)

where R_P is the mean response amplitude to parallel probe stimuli and R_O is the mean response to the orthogonal stimuli. The orientation-selectivity index could vary between −1 (complete negative adaptation, meaning no response to the orthogonal probes) and 1 (complete adaptation, meaning no response to the parallel probes). A value of 0 meant there was no adaptation, i.e., no difference between responses to parallel and orthogonal probes. The use of absolute values in the denominator ensured numerical stability for ROIs for which the average response to the orthogonal and parallel probes had an opposite sign (which occurred only when both responses were close to 0). An orientation-selectivity index significantly >0 would indicate selective adaptation to the adapter orientation, which would be evidence for neurons selective for the orientation of motion-boundary stimuli. The magnitude of the index reflected the proportion of orientation-selective neurons and the sharpness of orientation tuning and/or the susceptibility of these neurons to adaptation.

In addition, a motion-boundary “preference index” (PI) was computed in a way similar to that of the orientation-selectivity index. The preference index quantified the preference for motion-boundary stimuli over transparent-motion stimuli as the difference between the fMRI response to the orthogonal motion-boundary stimuli and the response to transparent-motion stimuli, divided by the absolute summed response to both stimulus types

$$PI=\frac{R_O-R_T}{\left|R_O\right|+\left|R_T\right|}$$ (5)

where R_T is the mean response amplitude to the transparent-motion probe stimuli. The preference index does not include the response to the parallel (adapted) probe stimuli, so it is not sensitive to the degree of orientation-selective adaptation. Unlike the orientation-selectivity index, the preference index does not have an interpretation in neuronal terms, but is merely a measure of the relative preference for motion-boundary stimuli compared with transparent-motion stimuli and is thus analogous to subtraction-based measures used in previous studies to identify motion-boundary selectivity with fMRI (Dupont et al. 1997; Tyler et al. 2006; Van Oostende et al. 1997; Zeki et al. 2003).

Orientation-selectivity indexes and preference indexes were computed for each individual subject from the mean response amplitudes for each stimulus type (averaged across adapter-stimulus orientations). Bootstrapping was used to estimate population confidence limits on the mean orientation-selectivity and preference indexes (averaged across subjects) and used to identify ROIs with indexes significantly different from 0 (Efron and Tibshirani 1993). Individual bootstrap estimates of orientation-selectivity and preference indexes were computed as follows. On each bootstrap iteration, a random sample of eight runs (corresponding to the actual number of runs) was chosen with replacement from each of the two scanning sessions for each subject. For each such sample an adaptation or preference index was computed in the same way as for the original data and the indexes averaged across subjects. This procedure was repeated 1,000 times to generate a bootstrapped estimate of the underlying distribution of orientation-selectivity or preference indexes, from which 68% and 95% confidence intervals (CIs) could be directly computed. Indexes were considered significantly different from zero if their 95% CIs did not include zero.

Identification of visual cortical areas

In separate scanning sessions, conventional retinotopic-mapping procedures were used to identify visual cortical areas (Engel et al. 1994; Larsson and Heeger 2006; Sereno et al. 1995). High-contrast radial checkerboard patterns were presented within 45° rotating-wedge apertures or within expanding or contracting 1°-wide rings. Six runs of rotating wedges (three clockwise and three counterclockwise) and four runs of rings (two expanding and two contracting) were acquired. Data were preprocessed (motion-corrected, coregistered to a high-resolution anatomy, linearly detrended, and high-pass filtered) in the same way as for the adaptation data. Maps of phase (corresponding to polar angle and eccentricity) and coherence (corresponding to response modulation) were visualized on computationally flattened representations of the occipital cortical surfaces (“flat maps”) of each individual subject. Boundaries between retinotopic visual areas were drawn by hand on these flat maps following the conventions of Larsson and Heeger (2006) and Wandell et al. (2007). ROIs were combined across left and right hemispheres.

At the beginning of each scanning session, a localizer experiment was run to identify the subregion of each visual area that corresponded retinotopically to the stimulus annulus. A block design was used, alternating between 12-s long “stimulus-off” epochs (blank screen) and 12-s “stimulus-on” epochs (horizontal and vertical motion-boundary grating patterns identical to those used in the event-related experiments, switching phase, and orientation every 1.2 s). Attention was diverted from the visual stimuli by the same RSVP task used for the event-related scans (one trial per stimulus epoch). Localizer scans consisted of 10.5 stimulus-off/stimulus-on cycles; the first half cycle of each scan was discarded in the analysis. Localizer-scan data were analyzed separately for each session by fitting individual voxel time series (linearly detrended and high-pass filtered with a cutoff of 0.015 Hz) with a sinusoid at the stimulus frequency. This yielded for each voxel an estimate of the response amplitude, response phase, and coherence. Visual-area ROIs defined by retinotopic mapping were restricted to include only voxels with a coherence ≥ 0.2 in the localizer scans. In four of five subjects, a conventional KO localizer experiment was also run at the end of the scanning session. This localizer used the same RSVP task and timing parameters as those used earlier, but transparent motion control stimuli were shown instead of a blank screen during “stimulus-off” epochs.

MRI acquisition

Standard gradient-echo echoplanar imaging methods were used to measure blood oxygenation level dependent (BOLD) contrast (Ogawa et al. 1990) (a measure of population neural activity) in the visual cortex. Measurements were performed on a Siemens Allegra 3T scanner equipped with a four-channel phased-array surface coil (NM-011 transmit head coil and NMSC-021 receive coil; Nova Medical, Wakefield, MA) with the following parameters: voxel size = 3 × 3 × 3 mm³, TR = 1.2 s, echo time (TE) = 30 ms, flip angle = 75°, 22 oblique slices oriented perpendicular to the calcarine sulcus. At the beginning of each session, we also acquired an anatomical T1-weighted MPRAGE (magnetization-prepared rapid-acquisition gradient echo) that covered the same volume as the functional scans, but with twice the in-plane resolution (voxel size = 1.5 × 1.5 × 3 mm³). An automated robust image-registration method (Nestares and Heeger 2000) was used to compute the alignment between this “in-plane” anatomical volume, the functional volumes, and the high-resolution anatomical volume (acquired in a separate session; voxel size = 1 × 1 × 1 mm³, T1-weighted MPRAGE). The high-resolution MPRAGE was also used to extract cortical surfaces.

RESULTS

Localizer scans

The localizer stimuli evoked widespread activity corresponding to the stimulus location across all retinotopic areas investigated (V1–V3, V3A/B, LO1/2, hV4, VO1/2, V5/MT+, V7, IPS1/2) (Supplemental Fig. S1, A and C).¹ In parietal cortex (IPS1 and IPS2), the average time course was dominated by strong transient responses associated with the onset (and, for IPS2, offset) of visual stimulation, whereas early visual areas V1–V3 and ventral and lateral occipital areas exhibited a sustained response to the localizer stimuli (Supplemental Fig. S2). The conventional KO localizer stimulus evoked weaker and more spatially restricted fMRI response modulation, the strongest responses being clustered in and around V3A/B, V7, and V3d, with separate clusters of weaker modulation in V3v, hV4, VO1/2, and V5/MT+ (Supplemental Fig. S1, B and D). Weak response modulation was also observed in V1; in some subjects this was evident only at lower coherence thresholds (e.g., 0.15). These results are consistent with those of a previous study (Larsson and Heeger 2006), suggesting that functionally defined KO is not a single area, but extends across multiple dorsolateral visual areas such as V3A/B, V7, and possibly also V3d and LO1.

Adaptation scans

Eight areas (V2, V3, V3A, V3B, LO1, LO2, hV4, and V7) exhibited significant orientation-selective adaptation; the mean fMRI response to parallel probe stimuli was significantly weaker than the mean response to orthogonal probe stimuli (Figs. 2 and 3, Table 1). In all of these areas the orthogonal motion-boundary probe stimuli (relative to baseline) evoked robust responses (statistically significantly greater than zero). Thus the response difference between parallel and orthogonal stimuli reflected a reduction of the visually evoked response to the parallel stimulus orientation, rather than a decrease in activity relative to baseline (i.e., negative BOLD response). Adaptation was not evident (no significant response difference between parallel and orthogonal probe stimuli) in V1, in ventral occipital areas VO1 and VO2, in V5/MT, nor in intraparietal areas IPS1 and IPS2. With the exception of IPS2, which did not respond significantly to any of the probe stimuli, the lack of significant orientation-selective adaptation in these areas was not due to a lack of visual responsiveness because the responses to the motion-boundary probe stimuli were robust and statistically significant.

Fig. 2.

Event-related time courses (estimated by linear deconvolution and averaged across scanning sessions) for the 4 visual areas showing the greatest absolute difference in response amplitude between parallel and orthogonal probe stimuli. Dark and light bars in the bottom left corner of each panel indicate presentation of adapter and probe stimuli. Error bars (smaller than plot symbols) represent the SE for each time point (computed as the root mean square of the regression SEs across scanning sessions).

Fig. 3.

Probe stimulus response amplitudes for all visual area regions of interest (ROIs). Error bars: SE across scanning sessions. Bold: areas showing significantly greater response amplitudes to the orthogonal motion-boundary gratings than those to parallel motion-boundary gratings (asterisks *) and/or transparent motion (hashes #) (paired t-test across scanning sessions, df = 9, false discovery rate (FDR) corrected for multiple comparisons at α = 0.05).

Table 1.

Mean amplitude differences between probe stimulus responses

Visual Area ROI	Orientation Selectivity (ORTHO > PARA)	Motion-Boundary Preference (ORTHO > TRANS)
V1	0.024 (0.972, P > 0.3) [1/5]	0.012 (0.810, P > 0.4) [0/5]
V2	0.055 (4.03, P < 0.01) [2/5]	0.041 (3.81, P < 0.01) [0/5]
V3	0.072 (5.51, P < 0.001) [4/5]	0.055 (3.22, P < 0.05) [3/5]
V3A	0.110 (5.21, P < 0.001) [5/5]	0.081 (4.00, P < 0.01) [5/5]
V3B	0.083 (3.95, P < 0.01) [4/5]	0.063 (3.74, P < 0.01) [3/5]
LO1	0.100 (4.79, P < 0.001) [5/5]	0.055 (4.68, P < 0.01) [3/5]
LO2	0.064 (4.38, P < 0.01) [2/5]	0.032 (0.861, P > 0.4) [0/5]
hV4	0.072 (3.50, P < 0.01) [1/5]	0.063 (2.74, P < 0.05) [2/5]
VO1	0.046 (1.95, P > 0.08) [1/5]	0.077 (3.35, P < 0.01) [3/5]
VO2	0.033 (1.11, P > 0.2) [1/5]	0.065 (2.46, P < 0.05) [3/5]
V5/MT+	0.038 (1.65, P > 0.1) [2/5]	0.028 (1.58, P > 0.1) [1/5]
V7	0.059 (3.13, P < 0.05) [4/5]	0.043 (2.49, P < 0.05) [2/5]
IPS1	0.057 (1.82, P > 0.1) [2/5]	0.047 (2.12, P > 0.06) [3/5]
IPS2	0.029 (1.65, P > 0.1) [1/5]	0.018 (1.19, P > 0.2) [2/5]

Numbers are in units of % signal modulation (numbers in parentheses are t-statistics and P values, paired t-test across scanning sessions, df = 9, uncorrected for multiple comparisons). Areas with statistically significant differences between response amplitudes (FDR corrected for multiple comparisons at α = 0.05) are indicated in bold. Numbers in square brackets [] indicate the proportion of subjects for which the differences in response amplitudes were significant (resampling test, P < 0.05, uncorrected for multiple comparisons) in each subject individually.

We quantified the amount of orientation-selective adaptation in two ways: first, as the response-amplitude difference (the absolute difference between the responses to the orthogonal and parallel probe stimuli) and, second, as the orientation-selectivity index (the response-amplitude difference divided by the summed responses to both probes; see methods) (Fig. 4). Among the areas showing significant adaptation, response-amplitude differences varied relatively little. The largest differences were observed in lateral occipital areas V3A, V3B, and LO1 and were about 50% greater than those in V2, V3, hV4, LO2, or V7 (Table 1, Fig. 3). On the other hand, orientation-selectivity indexes varied greatly across areas, reflecting the large differences in absolute fMRI response across visual areas (Fig. 4). The largest orientation-selectivity indexes were found in V3A, V3B, LO1, LO2, and V7. For example, in LO1 the orientation-selectivity index (0.68) was more than 10-fold larger than that of hV4 (0.061), even though the difference in response amplitudes between orthogonal and parallel probes in LO1 was only about 40% greater than that in hV4, reflecting the much greater baseline response in hV4. We interpret this as evidence that neurons in LO1 are more strongly tuned for motion-boundary orientation than in hV4 and/or make up a greater proportion of the total neuronal population.

Fig. 4.

Orientation-selectivity indexes and preference indexes for visual area ROIs with significantly greater response amplitudes to orthogonal probe stimuli than to parallel probe stimuli and/or transparent-motion probes (ROIs in boldface in Table 1 and Fig. 3). Error bars represent 68% confidence intervals (CIs) for the mean index across scanning sessions, estimated using bootstrapping (see methods). Except for the preference index for LO2 and the orientation-selectivity index for VO1, all indexes were significantly greater than zero (bootstrapped 95% CIs >0; P < 0.05, uncorrected for multiple comparisons). n.s., not significant.

In most areas the responses to the transparent-motion probe stimuli were similar in amplitude to the parallel motion-boundary stimuli and weaker than the responses to the orthogonal motion-boundary stimulus (Fig. 3, Table 1). Analogous to the measures of adaptation, we quantified this both as an absolute response-amplitude difference (the difference between the responses to the orthogonal, or unadapted, motion-boundary probe and the transparent-motion probes), and as a preference index (response-amplitude difference as a fraction of the summed response to both probe stimuli) (Fig. 4). In most areas that showed significant adaptation (i.e., significantly greater responses to orthogonal than to parallel probe stimuli), the response-amplitude differences between the orthogonal motion-boundary probes and the transparent-motion probes were also significantly greater than zero. This comparison is analogous to a conventional subtraction analysis comparing the responses to motion boundaries and transparent motion. For these areas, which included most early and lower-tier extrastriate areas (V2, V3, V3A, V3B, LO1, hV4), both adaptation and preference indexes were significantly greater than zero, implying that the preference for motion-boundary stimuli over transparent motion was also associated with selectivity for motion-boundary orientation, as assessed with adaptation. In V7, the preference for motion boundaries over transparent motion was significant when considered in isolation (Table 1, Fig. 4), but borderline nonsignificant after correcting for multiple comparisons (Table 1, Fig. 3). Because the differences between the responses to motion-boundary probes and transparent-motion probes were very small in V7 (Fig. 3), this result is unlikely to reflect a real difference in response preference in this area. In two areas, however, the responses to motion-boundary probes and transparent-motion stimuli differed significantly. In VO1, which did not show significant orientation-selective adaptation to motion boundaries, the responses to motion-boundary stimuli were significantly greater than the responses to transparent motion, indicating a preference for motion-boundary stimuli but with no evidence for selectivity for the orientation of the motion boundaries. A similar result was observed in VO2, although the response-amplitude difference between motion boundaries and transparent-motion stimuli for this area was not significant after correction for multiple comparisons (Table 1). Conversely, LO2 exhibited significant orientation-selective motion-boundary adaptation, but no significant response difference between motion boundaries and transparent motion. Thus although LO2 showed selectivity for motion-boundary orientation, this property would not have become apparent based on the subtraction analysis alone. It should be emphasized, however, that although these response differences were statistically significant in both VO1 and LO2, the differences were small. Thus we cannot rule out the possibility that the observed dissociation in response preference in these areas was due to variability in the measured responses and the effects of statistical thresholding, rather than reflecting a real underlying difference in response properties between these areas.

DISCUSSION

We used fMRI adaptation techniques to measure the degree of orientation selectivity for motion-boundary stimuli across human visual areas. In the same experiments we concurrently measured the degree of response preference for motion boundaries relative to control stimuli containing transparent motion, allowing us to directly compare the distribution of selectivity for motion-boundary orientation with preference for motion-boundary stimuli across visual areas. The results can be summarized by three main findings. First, we demonstrate that responses to motion boundaries in many human visual areas (and particularly in dorsolateral areas V3A, V3B, LO1, LO2, and V7) are orientation-selective, consistent with the response properties of motion-boundary–selective neurons in dorsolateral macaque extrastriate visual areas (e.g., V3, V3A, V4d). Second, we found that responses to motion-boundary stimuli—whether measured by orientation-selective adaptation or by a greater response to motion-boundary stimuli than transparent-motion stimuli—are widespread in visual cortex and are found not only in the dorsolateral occipital region originally identified as KO (corresponding to LO1, LO2, and V3B in the present study), but also in early extrastriate visual areas V2 and V3 as well as in ventral occipital area VO1. Interestingly, we found no evidence either for selectivity for motion-boundary orientation or for motion-boundary preference in V5/MT+, despite the well-known preference for visual motion stimuli in this area, a result consistent with single-unit measurements of motion-boundary selectivity in MT (Marcar et al. 1995). Third, since our experimental design allowed us to directly compare orientation selectivity (the orientation-selectivity index) with preference for motion boundaries over transparent motion (the preference index), we were interested in whether these different measures would be in agreement. For most visual areas, the two measures gave similar results, but two exhibited differences. In one area, LO2, we found significant orientation-selective response adaptation, but there was no significant evidence for a stronger response to motion boundaries than that to transparent motion. Conversely, in area VO1, which responded significantly more strongly to motion boundaries than to transparent motion, there was no evidence of orientation-selective response adaptation. However, because these differences were small and might have been due to statistical thresholding effects, it is difficult to draw strong conclusions from this particular result.

Relation to previous neuroimaging studies of motion-boundary perception

Our results are largely consistent with earlier neuroimaging studies of motion-boundary perception (Dupont et al. 1997; Larsson and Heeger 2006; Tyler et al. 2006; Van Oostende et al. 1997; Zeki et al. 2003). In agreement with these studies we found the strongest preference for motion-boundary stimuli (relative to transparent motion) in dorsolateral occipital visual cortex in the region originally defined as KO. However, there are some notable differences between our results and those of previous studies. First, our finding of widespread responses to motion-boundary stimuli is at odds with initial claims of the existence of a single area “KO” that is uniquely specialized for processing such stimuli (Dupont et al. 1997; Van Oostende et al. 1997). It is likely that these early studies, which used less sensitive fMRI methods, failed to detect the relatively weaker motion-boundary responses in, e.g., V2 and V3. Later studies have also shown that KO is not a single entity, but likely extends over multiple visual areas including V3, V3A, V3B, LO1, and LO2 (Larsson and Heeger 2006; Tyler et al. 2006). Second, our results go beyond previous studies in demonstrating orientation selectivity for motion boundaries, as opposed to merely showing a preference for motion boundaries over transparent motion. Interpretation of previous studies is complicated by the difficulty of determining to what degree a stronger response to motion boundaries reflected selectivity for motion boundaries, as such, or low-level differences between the motion-boundary and transparent-motion stimuli. For example, responses of MT neurons are suppressed by the presence of motion in the nonpreferred direction within their receptive field (RF) (Heeger et al. 1999; Snowden et al. 1991). The difference in fMRI responses to motion-boundary gratings and transparent-motion stimuli could, in principle, be due to motion-opponent neurons with RFs smaller than the width of the stripes in the motion-boundary gratings. Neurons with RFs within each stripe would “see” only a single motion direction and thus would show no motion opponency. This would predict an overall greater population response to the motion-boundary gratings than the transparent-motion stimuli, independent of any preference for motion boundaries or motion-boundary orientation. Moreover, the transparent-motion stimuli are perceptually quite distinct from the motion-boundary stimuli (e.g., the former evoke the percept of two distinct depth planes). The two types of stimuli may very well engage other perceptual processes differently (e.g., depth perception, surface perception, perceptual organization, and image segmentation) in addition to motion processing per se, which might have accounted for any differences in overall responses.

Relation to models of motion-boundary perception

Models of motion-boundary processing generally comprise two stages, the first stage consisting of a set of linear or first-order filter mechanisms that compute local motion direction at several nearby spatial locations, the responses of which are subsequently rectified and then integrated by a second-stage or second-order filter that computes oriented motion direction contrast over a larger region of visual space (Durant and Zanker 2009; Sachtler and Zaidi 1995). Models of this type are commonly known as filter–rectify–filter (FRF) or linear–nonlinear–linear (LNL) models and have been widely used to explain sensitivity to modulations of nonlinear image properties such as texture and contrast (Ellemberg et al. 2006; Kingdom et al. 2003; Landy and Oruç, 2002). The properties of the first-stage filters are similar to those of direction-selective neurons in V1. These filters would respond only to the local motion components of the stimuli and would not be selective (or adapt) to the orientation of the motion boundaries. [Initially it was postulated that the extraction of motion would take place in V5/MT, consistent with the lack of significant motion-boundary adaptation we observed in that area, but evidence from lesion studies (Lauwers et al. 2000) and latency measurements (Marcar et al. 1995, 2000; Mysore et al. 2006) indicates that motion-boundary processing does not in fact involve V5/MT.] The second-stage mechanism would be predicted to be selective for motion-boundary orientation, consistent with the orientation-selective adaptation we observed in several extrastriate visual areas, most notably V3A, V3B, LO1, and LO2. Although FRF models of motion-boundary processing are primarily supported by behavioral data (Durant and Zanker 2009; Sachtler and Zaidi 1995), they provide qualitative predictions consistent with our results. First, as mentioned previously, the first-stage filters would be expected to respond only to the local motion components and be unaffected by the orientation of motion boundaries, thus responding equally strongly to the parallel and orthogonal motion-boundary probes, consistent with our V1 measurements. The second-stage filters, however, would not only be predicted to be selective for motion-boundary orientation, but would also respond more weakly or not at all to transparent-motion stimuli. This is because the local motion inputs to the inhibitory and excitatory lobes of the second-stage filter would on average be equal and opposite and thus cancel out. If the excitatory and inhibitory inputs were not exactly balanced, for example, because of small local variations in the relative distribution of opposite motions in the stimulus, the responses of the second-stage filters would be predicted to be attenuated but not completely absent. FRF models thus predict that orientation selectivity for motion boundaries should be associated with weak or no responses to transparent motion. Although all visually responsive areas exhibited weaker responses to transparent motion than to motion boundaries, only in VO2 was this response close to zero, as predicted by FRF models (Fig. 3). However, because all of these areas also respond to visual stimuli that do not contain motion boundaries, the residual responses are likely to reflect activity of populations of neurons selective for other stimulus properties (e.g., contrast and motion).

A potential confound in motion-boundary stimuli in which a component of the motion is perpendicular to the orientation of the motion boundaries (such as those used in this study), is that the boundaries necessarily coincide with regions of greater appearance or disappearance of dots. This has been referred to variously as “flicker boundaries” or “dynamic occlusion boundaries” (Sáry et al. 1994). Such boundaries could in principle be detected by a second-order mechanism that is sensitive to variations in flicker rather than motion direction. Although we cannot rule out that some of the response adaptation we observed might have reflected such a flicker detection mechanism, it is unlikely to be the sole (or even major) underlying mechanism because single-unit recordings in macaque V4 (Mysore et al. 2006) have shown that the majority of neurons selective for motion-boundary orientation exhibit the same orientation tuning for motion boundaries defined by perpendicular motion (which contain flicker boundaries) as that for motion boundaries defined by parallel motion (which do not contain flicker boundaries). However, from our experimental design we cannot determine how much of the orientation-selective adaptation in our measurements was due to flicker versus motion boundaries per se.

In the context of FRF models, our results thus provide evidence that the second-stage filters are distributed across multiple visual areas, but may be most common in the dorsolateral occipital visual areas with the highest orientation-selectivity indexes (V3A, V3B, LO1, LO2, and V7). An alternative interpretation also compatible with our results is that motion boundaries are extracted by second-stage filter mechanisms in one area only (e.g., V3A) and are passively propagated to other areas. The response properties of VO1, which preferred motion boundaries but lacked significant orientation selectivity (assuming that the lack of orientation selectivity in this area was not simply a result of statistical thresholding), is consistent with a mechanism that responds to motion boundaries of any orientation. For example, a population of shape-selective neurons that responded to shapes defined by motion boundaries would also be predicted to respond (if weakly) to the component motion-boundary contours of the shapes. Such a mechanism would be predicted to respond preferentially to shapes defined by motion boundaries over transparent motion, but would not be selective for motion-boundary orientation per se. Although this interpretation remains conjectural, it is consistent with other studies showing boundary cue-invariant responses to shapes in ventral occipital areas (Grill-Spector et al. 1998; Kourtzi and Kanwisher 2000; Vinberg and Grill-Spector 2008) as well as measurements of boundary cue invariance in macaque IT (Sáry et al. 1995) and cat area 18 (Zhan and Baker 2006).

Comparison with fMRI measurements of orientation-selective adaptation

The adaptation experimental design used in this study has previously been applied to study orientation selectivity for boundaries defined by other cues such as luminance contrast (Larsson et al. 2006), carrier orientation (Larsson et al. 2006), and illusory contours (Montaser-Kouhsari et al. 2007). Like these previous studies, we found that orientation selectivity for second-order contours—whether defined by motion, carrier orientation, or contrast—is most pronounced in higher extrastriate areas such as LO1 and hV4. These results could be interpreted as evidence of a single cue-invariant boundary mechanism (Vinberg and Grill-Spector 2008), although evidence from cross-adaptation studies is more consistent with the existence of multiple independent, but spatially overlapping, mechanisms for different boundary cues (Kingdom et al. 2003; Larsson et al. 2006).

Motion-boundary selectivity in macaque visual cortex

A number of studies have investigated motion-boundary processing in macaque visual cortex using single-unit recordings, measuring the degree of orientation selectivity for motion boundaries in V1 and extrastriate areas V2, V3/V3A, V4, V5/MT, and IT (Marcar et al. 1995, 2000; Mysore et al. 2006, 2008; Sáry et al. 1995; Zeki et al. 2003). Although the stimuli used in these studies were not identical, they reveal a general pattern of increasing selectivity for motion-boundary orientation in extrastriate areas, with little or no selectivity in V1 and V5/MT. Previous neuroimaging studies have not measured orientation selectivity for motion-boundary stimuli, making a direct cross-species comparison difficult. In the present study we quantified the relative degree of orientation selectivity (as assessed by orientation-selective adaptation) across visual areas. It is thus theoretically possible to compare our results with single-unit measurements of motion-boundary orientation tuning in macaque visual areas. Such a comparison needs to be interpreted with great caution, however. The orientation-selectivity index in our study is a measure of the amount of orientation-selective response adaptation relative to the overall response of an area. This index conflates a number of properties: the proportion of orientation-selective neurons; the relative decrease in response following adaptation; the orientation tuning bandwidth of orientation-selective neurons; and the relationship between population neural activity and fMRI BOLD response (which may itself be modulated by adaptation). For single-unit data, the measure of motion-boundary selectivity that is most comparable is the proportion of neurons examined that showed orientation tuning for motion boundaries, a measure that may be influenced by methods used to isolate single units, overall responsivity of neurons (e.g., effects of anesthesia and arousal), and criteria used to define orientation tuning. Because of these many potentially confounding factors, there is little reason to expect a close correspondence between measures of orientation selectivity for motion boundaries obtained by fMRI or single-unit recordings. Bearing in mind these inherent limitations, the two measures agree surprisingly well, especially the relative rank ordering of visual areas (even allowing for the many unresolved issues of homology between higher-order extrastriate areas in humans and macaques; Kolster et al. 2009; Orban et al. 2004) is similar for both species (Marcar et al. 1995, 2000; Mysore et al. 2006; Sáry et al. 1995; Zeki et al. 2003). In both species selectivity for motion boundary stimuli is greater in extrastriate visual areas than that in V1 and is highest in cortical regions within and around dorsal V3/V4 and V3A. A recent study using intrinsic optical imaging (Lu et al. 2010) showed orientation maps for motion boundaries in V2 that were largely absent in V1, a result that agrees well with our results showing significant motion-boundary adaptation in V2 but not in V1. Future studies are likely to provide more directly comparable measurements, such as fMRI measurements of orientation-selective motion-boundary adaptation in macaque visual areas. At present, the available evidence suggests that motion-boundary processing involves similar sets of areas in humans and macaque monkeys, consistent with an underlying mechanism that is common to both species.

A notable exception to this was the motion boundary selectivity of area V4. In the macaque, V4 is split into a dorsal (V4d) and a ventral (V4v) part, which are adjacent to the dorsal and ventral parts of V3 (Gattass et al. 1988). In humans, hV4 (the presumed human homologue of macaque V4) is a single area anterior to ventral V3, with a full hemifield representation (Brewer et al. 2005; Wade et al. 2002; Wandell et al. 2005). Anterior to dorsal V3 is LO1, which like hV4 but unlike macaque V4d represents a full hemifield (Larsson and Heeger 2006; Wade et al. 2008; Wandell et al. 2005). An alternative scheme was proposed by Hansen et al. (2007) who claimed that hV4 and LO1 are the human homologues of macaque V4v and V4d respectively and are thus part of the same visual area. If this latter scheme were correct, one would expect the response properties of human hV4 and LO1 to be similar, but as Fig. 4 shows, these areas differed greatly in their selectivity for motion boundaries. Furthermore, in the macaque the proportions of motion-boundary–selective neurons in V4 and V3/V3A are similar (Mysore et al. 2006; Zeki et al. 2003). Virtually all of the single-unit data from macaque V4 have been recorded in V4d, which in the scheme of Hansen and colleagues (2007) corresponds to human LO1. Thus if LO1 were functionally identical to macaque V4d, the motion-boundary selectivity of this area should be close to that of human V3/V3A, but again these two areas differed in this measure (Fig. 4, Table 1). Together these results lend further support to the LO1/hV4 scheme of Larsson and Heeger (2006), but are inconsistent with the human V4 model proposed by Hansen et al. (2007).

GRANTS

This work was supported by National Eye Institute Grant R01-EY-016165 to D. J. Heeger and M. S. Landy.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.