Direction-Selective Patterns of Activity in Human Visual Cortex Suggest Common Neural Substrates for Different Types of Motion

Abstract

A sense of motion can be elicited by the movement of both luminance- and texture-defined patterns, what is commonly referred to as first- and second-order, respectively. Although there are differences in the perception of these two classes of motion stimuli, including differences in temporal and spatial sensitivity, it is debated whether common or separate direction-selective mechanisms are responsible for processing these two types of motion. Here, we measured direction-selective responses to luminance- and texture-defined motion in the human visual cortex by using functional MRI (fMRI) in conjunction with multivariate pattern analysis (MVPA). We found evidence of direction selectivity for both types of motion in all early visual areas (V1, V2, V3, V3A, V4, and MT+), implying that none of these early visual areas is specialized for processing a specific type of motion. More importantly, linear classifiers trained with cortical activity patterns to one type of motion (e.g., first-order motion) could reliably classify the direction of motion defined by the other type (e.g., second-order motion). Our results suggest that the direction-selective mechanisms that respond to these two types of motion share similar spatial distributions in the early visual cortex, consistent with the possibility that common mechanisms are responsible for processing both types of motion.

1. INTRODUCTION

The human visual system can readily extract motion information from dynamic displays, such as the direction and speed of a moving object. Motion energy models have provided a solution to explain how the direction and speed of luminance-defined patterns are detected with spatiotemporal receptive fields of physiologically realistic units (Adelson & Bergen, 1985; van Santen & Sperling, 1985; Watson & Ahumada, 1985). Each of these units responds to a particular direction of motion that is defined by position shifts in luminance edges, so called first-order motion. However, perception of motion can also be induced by second-order stimuli that lack any reliable motion of luminance edges, as can be achieved by the movement of a contrast envelope over an oriented pattern (Cavanagh & Mather, 1989) (see Figure 1B). Although a pure second-order motion stimulus might occur rarely in real-life situations, the fact that observers can readily perceive the movement of such patterns in laboratory situations indicates that motion-energy models built with luminance-based, spatiotemporal filters may be insufficient to explain human motion perception.

Figure 1.

Stimulus displays and cortical regions of interest. (a) First-order motion stimulus with sinusoidal variations in luminance. (b) Second-order motion stimulus with variations in contrast in the absence of overall variations in luminance. For both displays, rotational motion could occur either clockwise or counterclockwise. Note that the rotation of the sinusoidal contrast modulation is perpendicular to the local orientation of the concentric ring pattern. (c) Visual areas of interest for a representative participant, with activations plotted for the localizer scan.

A long-debated question is whether these different types of motion are processed by distinct neural pathways (Chubb & Sperling, 1988; Wilson, Ferrera & Yo, 1992) or by a single unified system (Benton & Johnston, 2001; Benton, Johnston, McOwan & Victor, 2001; Johnston, McOwan & Buxton, 1992). Numerous psychophysical studies support the proposal of distinct processing systems for the two types of motion. It has been shown that the temporal frequency tuning of second-order motion differs from that of first-order motion (Derrington & Badcock, 1985; Hutchinson & Ledgeway, 2010; Ledgeway & Hutchinson, 2005). Unlike first-order motion, prolonged adaptation to second-order motion results in little to no motion aftereffect upon presentation of a static test pattern (Derrington & Badcock, 1985; McCarthy, 1993), although an aftereffect can be observed with the presentation of a dynamically counterphasing test pattern (Ledgeway, 1994; Nishida, Ledgeway & Edwards, 1997; Pavan, Campana, Guerreschi, Manassi & Casco, 2009). Even with such dynamic test patterns, however, the influence of adaptation to one type of motion generally does not transfer to the other type of motion (Ledgeway & Smith, 1994; Nishida et al., 1997; Pavan et al., 2009; but also see Ledgeway & Smith, 1997). This lack of crossover adaptation provides further support for the notion of distinct motion processing mechanisms. Seiffert and Cavanagh (1998, 1999) found that observers were able to detect near-threshold first-order motion by relying on a velocity-sensitive motion energy mechanism, but had to rely on position tracking of individual features (cf. Ullman, 1979; Del Viva & Morrone, 1998) to detect the motion of second-order displays. These and other observations support the notion that first- and second-order motion may be processed by distinct visual mechanisms.

Although psychophysical studies have provided some compelling evidence to suggest a segregation between first- and second-order motion processing, neuroscientific studies have provided somewhat mixed results. Single-cell recordings in the visual cortex of cats and monkeys have generally found that direction-selective neurons respond to both first- and second-order motion. Recordings in areas V1 and MT of the monkey, and in areas 17 and 18 in the cat, have found that a majority of direction-selective neurons respond preferentially to luminance-defined, first-order motion, but that some portion of those cells also respond to second-order motion (Zhou & Baker, 1993, 1994; Chaudhuri & Albright, 1997; O’Keefe & Movshon, 1998; Baker, 1999). Importantly, these studies have failed to find neurons that respond preferentially to second-order motion only, suggesting that the same population of cells is likely responsible for processing both types of motion (O’Keefe & Movshon, 1998; Ledgeway, Zhan, Johnson, Song & Baker, 2005). On the other hand, neuropsychology studies have described human patients with cortical lesions who show evidence of a double dissociation between these two types of motion processing; that is, perception of one type of motion is impaired while the other remains intact (Vaina & Coway, 1996; Vaina, Makris, Kennedy & Coway, 1996, 1998; Plant, Laxer, Barbaro, Schiffman & Nakayama, 1993; Plant & Nakayama, 1993).

Human neuroimaging studies have provided mixed evidence regarding whether common or separate neural substrates support these two types of motion perception. An early fMRI study found greater responses to second-order than first-order motion at relatively higher stages of visual processing hierarchy, such as visual area V3 (Smith, Greenlee, Singh, Kraemer & Henning, 1998). However, subsequent fMRI studies have generally found similar responses to first- and second-order motion across all retinotopic visual areas, as well as motion-sensitive area MT+. Seiffert et al. (2003) compared fMRI responses to moving versus static displays of first- and second-order patterns, and found evidence of enhanced responses to motion in all early visual areas including the primary visual cortex (area V1). Thus, the responses to second-order motion were similar though weaker than those observed for first-order motion. However, it has been suggested that higher order areas beyond the extrastriate cortex tend to show greater activity for second-order motion. Dumoulin and colleagues found that several lateral occipital regions and posterior parietal regions were more responsive to second-order motion (Dumoulin, Baker, Hess & Evans, 2003). Although these fMRI studies attempted to isolate motion-specific responses by comparing moving and static stimuli, a concern is that such fMRI subtraction methodology does not allow for the isolation of direction-selective responses, a hallmark of motion processing. Other factors, such as differences in attentional demands when processing first- and second-order motion (see Ho, 1998; Ashida, Seiffert & Osaka, 2001) or differential adaptation for static and moving stimuli, could therefore contribute to the differential responses found in the visual cortex and higher order brain areas.

A few fMRI studies have attempted to isolate direction-selective responses by measuring the effects of adaptation to motion direction per se. Nishida and colleagues (2003) found effects of direction-selective adaptation for both first- and second-order motion in all early visual areas, extending from the primary visual cortex (V1) to area MT+. These results suggested no anatomical segregation for the two types of motion processing. However, a recent study found that the fMRI adaptation effect disappeared when adapting and test stimuli consisted of different types of motion (Ashida, Lingnau, Wall & Smith, 2007). The lack of cross-adaptation was interpreted to suggest that separate neural populations may process first- and second-order motion, although within the same cortical areas. Although adaptation can reveal selective responses, its underlying basis and relationship to response selectivity is still not well understood (Sawamura, Orban & Vogels, 2006). For example, neurons in area V4 exhibit weak direction selectivity, yet can acquire direction tuning after prolonged adaptation to motion stimuli (Tolias, Keliris, Smirnakis & Logothetis, 2005). Therefore, the stimulus selectivity revealed by adaptation might not accurately reflect the response selectivity of a given neuron or brain area.

In the present study, we applied a different approach to measure direction-selective responses in the human visual cortex, by analyzing fMRI BOLD signals with multivariate pattern analysis (MVPA), also called fMRI decoding. Following the approach of Kamitani and Tong (2005), we assumed that individual fMRI voxels sampled from the visual cortex would show a weak but reliable preference for particular motion directions, presumably due to random variations in the spatial distribution of feature-selective neurons or cortical columns. High-resolution functional imaging of orientation responses in cats and humans has provided further support for the notion that local variability in columnar organization can lead to weak biases in feature preference at much coarser spatial scales (Swisher et al., 2010).

With this decoding approach, Kamitani and Tong (2006) showed that the direction of coherently moving random dots can be reliably decoded by pooling the information from multiple voxels in early visual cortex, for individual areas from V1 through V4 and also human MT+. This result indicates that direction-selective information in visual cortex can be effectively extracted from patterns of fMRI activity. This motion-decoding approach has proven highly effective for investigating the neural bases of feature-based attention (Kamitani & Tong, 2006; Serences & Boynton, 2007a), perceptual binding of color and motion signals (Seymour, Clifford, Logothetis & Bartels, 2009), and conscious perception of ambiguous motion displays (Serences & Boynton, 2007b; Brouwer & van Ee, 2007).

Here, we hypothesized that if a common direction-selective mechanism is responsible for processing first- and second-order motion, then direction-selective activity patterns for these two types of motion should be very similar. We tested this hypothesis by examining direction classification performance across the two motion types (generalization analysis). This involved training a linear classifier on fMRI activity patterns evoked by first-order motion and testing the classifier on activity patterns evoked by second-order motion, and vice versa. By testing for generalization of feature-selective responses across different types of motion, multivoxel pattern analysis provides a powerful method to address the nature of the underlying perceptual representations in the human visual cortex and the mechanisms that are used to process different types of stimulus motion.

2. METHODS

2.1. Participants

Six healthy adult volunteers with normal or corrected-to-normal vision participated in the experiment. All participants provided informed consent to participate in the fMRI experiment, which was approved by Vanderbilt University Institutional Review Board.

2.2. Apparatus and Stimuli

Visual stimuli were generated by a Macintosh G4 computer running Matlab, and projected onto a rear-projection screen using an MR-compatible LED projector (Avotec, Inc). The LED projector was carefully calibrated using a Minolta LS110 luminance meter to ensure linearity of luminance output in the visual display.

Stimulus displays consisted of a finely textured pattern of concentric rings (~ 0.15 deg in thickness) that was modulated by a sinusoidal radial grating (8 cycles/stimulus) to create first- and second-order stimuli that were well matched in spatial and temporal frequency (Figure 1a,b). To make the first-order stimulus, the radial grating varied in luminance with a contrast of 10%. To make the second-order display, the radial grating varied only in the contrast of the concentric rings (from 0% to 25%) while keeping the mean luminance constant. Somewhat greater contrast modulation was chosen for the second-order stimulus to ensure that rotational motion was readily visible to the observer, as contrast sensitivity to second-order motion is generally worse than that for first-order motion (Hutchinson & Ledgeway, 2006). Both displays appeared within an annulus with an inner radius of ~2° and an outer radius of ~9°. In different stimulus blocks, the radial grating rotated either clockwise (CW) or counterclockwise (CCW) at a temporal frequency of 1.5 cycle/second. Small changes in the speed of the rotating grating occurred at random intervals, which observers had to detect and to report whether the speed increased or decreased. We used radial motion, rather than translational motion, since observers could maintain fixation better with radial motion, and also we could avoid any potential differential activity at leading or trailing edge of motion (cf. Whitney et al., 2003).

2.3. Experimental Procedure and fMRI Acquisition

All scanning was performed using a 3.0-Tesla Philips Intera Achieva MRI scanner with a standard 8-channel head coil at the Vanderbilt University Institute for Imaging Science. A high-resolution 3D anatomical T1-weighted scan (FOV 256x256, 1x1x1 mm resolution) was obtained for individual observers at the beginning of each scan. Functional BOLD responses were measured with standard gradient-echo echoplanar T2*-weighted imaging sequence (TR 2000 ms, TE 35 ms, flip angle 80°; FOV 192 x 192 mm, slice thickness 3 mm (no gap), in-plane resolution 3x3 mm). The imaged volume consisted of 28 slices that were collected perpendicular to the calcarine sulcus, covering the entire occipital lobe and the posterior parietal and temporal cortex. Each observer used an individually fitted bite bar that was attached to a custom-made mounting system to minimize head motion.

Each scanning session consisted of 3 visual localizer runs, 7–8 experimental runs of first-order motion and equal number of second-order motion runs for each participants. The entire scanning session lasted about 2.5 to 3 hours. Only one type of motion, either first- or second-order, was presented in each experimental run and the motion type alternated every run. Each experimental run included five blocks for each of the two motion directions (CW and CCW rotation), presented in a randomly shuffled order, with blank/fixation blocks occurring at the beginning and the end of each run. Each experimental run consisted of a total of 12 blocks that each lasted 16 seconds. During each stimulus block, brief changes in motion speed (333 ms duration) were introduced at randomly chosen intervals, occurring 2 to 4 times per block. The change in motion speed was a modest increase or decrease from the base temporal frequency (1.5 Hz), with motion always continuing in the same direction. Observers were instructed to make a speeded keypress response to indicate whether the speed increased or decreased while attending to the motion display. The magnitude of the speed change was varied for each subject, estimated from a pre-scanning session to yield approximately 75% correct performance on the speed discrimination task for each grating type. The magnitude of speed change remained fixed throughout the scanning session. Behavioral performance for this task ranged between 61% to 87% for first-order motion and between 39% to 70% for second-order motion, where a complete failure to detect the changes in speed would have led to 0% accuracy. It should be noted that any differences in performance accuracy across the two motion types cannot account for the ability to decode the direction of motion within a motion condition.

Additional localizer runs were performed to identify regions in the visual cortex that corresponded with the retinotopic extent of our motion displays. In localizer runs, observers viewed a luminance-based motion stimulus that was identical to that shown in the experimental runs, except that the size of the annulus was reduced (3-deg inner radius and 8-deg outer radius) to reduce the likelihood of selecting voxels near the edges of the motion stimuli in the experimental runs. The luminance contrast was also increased to 25% to ensure strong activation of retinotopic visual areas and motion-sensitive area MT+. For decoding, we further restricted our analysis to focus on the 100 most visually active voxels based on these localizer runs, to ensure that all selected voxels corresponded well with retinotopic location of the localizer stimulus. Each localizer run consisted of alternating 16-s blocks of fixation rest and moving gratings, with additional blank periods added to the beginning and end of the run, resulting in a total duration of 192 seconds (12 blocks) for three participants and 224 seconds (14 blocks) for the other participants. The observer’s task was the same as that of the experimental runs.

2.4. fMRI Data Processing

All fMRI data were motion-corrected using automated image registration software. The motion-corrected data were preprocessed using Brain Voyager QX (version 1.9, Brain Innovation, Maastricht, The Netherlands), which included slice timing correction and linear trend removal. No spatial or temporal smoothing was applied. Rigid-body transformations were performed to align the fMRI data to the within-session T1-weighted 3D anatomical scan, which in turn was aligned to the anatomical 3D scan collected in the separate retinotopic mapping session. All automated alignment procedures were carefully inspected visually and subjected to manual fine-tuning to correct for any visible residual misalignment. After across-session alignment, all data underwent Talairach transformation and reinterpolation using 3 x 3 x 3 mm voxels.

2.5. Regions of Interest

Retinotopic mapping data were collected from each participant in a separate scanning session, and early visual areas were delineated using well-established methods (Sereno et al., 1995; DeYeo et al., 1996; Engel et al., 1997). After retinotopic areas V1, V2, V3, V3A and V4 in both hemispheres were identified, we identified visually activated voxels based on statistical activation maps obtained from the visual localizer scans. The human homologue of the MT/MST complex (MT+) was identified by selecting regions in the lateral occipital cortex that responded substantially more to moving (i.e., expanding/contracting) than stationary random-dot displays. We selected the most activated 100 voxels from each ROI based on the in-session localizer runs, although the number of voxels available in areas V3A and V4 did not always reach quite that number for some observers. Early visual areas for one representative observer are shown in Figure 1c, with the activation map resulting from the localizer runs.

2.6. Linear Classifiers for Decoding Motion Direction

We used multivariate pattern classification to examine whether the neural representation of motion direction was similar for first- and second-order motion stimuli, based on fMRI measures of direction-selective activity patterns in the human visual cortex. Specifically, we used linear support vector machines (SVM) to obtain a linear function to discriminate the activity patterns elicited by the two different directions of motion (CW and CCW):

$$g(x_j)=\sum _{i=1}^n{w}_i{x}_{ij}+w_o$$

where x_j is a vector specifying the BOLD amplitude of all n voxels on block j, x_j is the amplitude of voxel i, w_i is weight of voxel i, and w_o is the overall bias. With a training data set, linear SVM attempts to find out the optimal weights and bias for the discriminant function, so that the function g(x_j) satisfies:

• g(x_j)> 0, when fMRI activity is induced by CW motion

• g(x_j) < 0, when fMRI activity is induced by CCW motion

With this trained discriminant function, independent test data were classified as CW when the output of the function was larger than 0, and as CCW otherwise.

We used a leave-one-run-out procedure for cross validation, which was repeated until all runs had served as a testing pattern, then calculated the average classification performance across all iterations. First we performed classification analysis for each motion type separately. Then we assessed generalization performance by building a linear classifier with one type of motion and testing decoding performance with the other type of motion.

3. RESULTS

3.1. Classification of each type of motion

Multivariate pattern analysis of BOLD responses showed that activity patterns reliably predicted the motion direction of both first- and second-order motion stimuli. As shown in Figure 2 (white bars), classification performance for first-order motion was significantly greater than chance level (50%) in all visual areas tested (smallest t₅ = 3.05, p < .05). By contrast, the univariate mean BOLD amplitude in these visual areas could not reliably discriminate between the two directions of motion (performance ranged between 46% to 57%), indicating that overall fMRI BOLD signal changes induced by the rotating radial pattern did not reliably differ between CW and CCW motion directions. A similar pattern of results was found for contrast-defined, second-order motion stimuli (Figure 2, black bars). Although decoding performance was less successful for second-order motion stimuli than that for first-order stimuli (2-WAY ANOVA, main effect of motion type, F(1,60) = 5.89, p < .05), greater than chance-level performance was observed for second-order motion in all early visual areas (smallest t₅ = 4.52, p < .01) with the exception of area V4 (V4: t₅ = 1.90, p = 0.12). The poorer decoding found for second- than first-order motion is consistent with the view that second-order stimuli produce weaker direction-selective responses in the visual cortex.

Figure 2.

Classification accuracy for decoding the direction of first-order motion (white bars), second-order motion (black bars), and for generalization across motion types (textured bars), shown for each visual area. Decoding accuracy was significantly greater than chance level for all visual areas tested, with the exception of area V4 for the second-order motion condition.

3.2. Generalized Classification between two types of motion

Reliable classification of direction of motion for both luminance-defined and contrast-defined motion stimuli suggests that early visual areas are sensitive to both types of motion signals. Next, we investigated whether the patterns of direction-selective activity elicited by these two types of motion resembled one another, by examining generalization performance across the two different types of motion. A linear classifier was trained with fMRI responses to the first-order motion stimuli and then tested on fMRI responses to the second-order motion stimuli, and vice versa. If the processing of both first-order and second-order motion were mediated by a shared neural mechanism, then the resulting patterns of cortical activity should be similar across motion types and allow for reliable generalization.

Classification performance for the generalization analysis is shown by the textured bars in Figure 2. Classification accuracy was very similar for generalization from first-order to second-order and second-order to first-order, therefore the results were averaged together. Classification performance was almost as high for generalization as for second-order motion decoding, and did not reliably differ (F(1,60) = 0.63, n.s.). Generalization performance was significantly greater than chance level in all retinotopic areas tested (smallest t₅ = 3.64, p < .05). These results indicate both first- and second-order motion lead to very similar direction-selective activity patterns in the human visual cortex.

We conducted an additional analysis to determine whether these results were generally robust to the number of voxels used for pattern analysis. Because visual areas differ in their size and the number of available voxels, it is conceivable that our use of a fixed number of voxels from each region of interest might have affected the pattern of results found across visual areas. Figure 3 shows that classification performance in all conditions steadily improved when an increasing the number of voxels was used for analysis, with performance saturating by about 50 or so voxels. Moreover, generalization performance closely followed the accuracy of decoding performance for second-order motion, implying that direction-selective activity patterns for second-order motion are sufficiently similar to those evoked by first-order motion to lead to a comparable level of generalization performance. These results were observed for each visual area across a wide range of voxel numbers used for pattern analysis.

Figure 3.

Classification performance of individual visual areas, plotted as a function of the number of voxels used for multivoxel pattern analysis to predict motion direction. First-order motion (blue line), second-order motion (red line), and generalization accuracy across the two motion types (green line). Data points are provided if at least 5/6 participants had a sufficient number of voxels for analysis in a given visual area.

4. DISCUSSION

By using fMRI pattern classification, we were able to isolate direction-selective responses in the human visual cortex and address a long-standing question regarding the neural bases of motion perception. Classification of the motion direction of contrast-modulated texture patterns was lower than that observed for moving luminance-based patterns, consistent with previous studies showing that motion responses are generally weaker or less reliable for second-order motion (O’Keefe & Movshon, 1998; Seiffert et al., 2003). In early visual areas, we did not find evidence to suggest that any of these regions might be specialized for processing a particular type of motion, consistent with some neuroimaging studies (Seiffert et al., 2003; Nishida et al., 2003) but not others (Smith et al., 1998; Dumoulin et al., 2003; Ashida et al., 2007). More importantly, we found strong generalization performance across the different types of motion, suggesting either that these direction-selective responses in the human visual cortex were based on a shared neural substrate or that, at the very least, the direction-selective responses to these two types of motion share very similar spatial distributions in the early visual cortex.

The fact that direction-selective responses could be isolated through the use of pattern analysis may help to explain why our fMRI results are in good agreement with those obtained from neurophysiological recordings of direction-selective responses in individual neurons. We found that direction-selective responses to second-order motion were evident as early as human V1, and parallel findings have been reported in the primary visual cortex of cats and monkeys (Zhou & Baker, 1993, 1994; Chaudhuri & Albright, 1997; O’Keefe & Movshon, 1998; Barraclough et al., 2006). This result agrees with previous neuroimaging work (Seiffert et al., 2003) and fails to support proposals that second-order motion processing should occur at a higher level in visual hierarchy than first-order motion (Lu & Sperling, 1995; Smith et al., 1998; Wilson et al., 1992). According to some theoretical proposals, directional information from a second-order motion stimulus can be extracted only after successful processing of the second-order characteristic by a motion detector sensitive to such patterns (Wilson et al., 1992). If so, reliable decoding of second-order motion direction in V1 could reflect either the processing of second-order stimulus properties at the processing stage of V1, or alternatively, some form of top-down feedback of such information to V1. According to other theories, however, a single system, such as the multi-channel gradient model, can account for the processing of both first and second-order motion (Johnston, McOwen & Buxton, 1992; Johnston & Clifford, 1995). The current results are most consistent with a single-system account, because direction-selective patterns of activity generalized across stimulus type, suggesting a common neural basis for processing both stimuli.

These conclusions, based on neurophysiology and neuroimaging data, might seem inconsistent with published reports of brain-damaged patients who show selective deficits in first-order or second-order motion perception, suggestive of a double dissociation (Vaina & Cowey, 1996; Vaina, Cowey & Kennedy, 1999; Vaina et al., 2000; Vaina & Soloviev, 2004; Vaina & Dumoulin, 2011). These patients had unilateral lesions, providing a within-patient control for performance across the visual fields. The patients with first-order motion deficits, RA and TF, could detect and discriminate second-order motion normally, whereas the patients with second-order motion deficits, FD and JV, showed normal performance for first-order motion. At first glance, these observations seem fundamentally inconsistent with proposals of a common motion system, because focal damage produced specific deficits. However, a closer look at the evidence from multiple studies reveals a more complex picture. First, the location of damage associated with deficits in second-order motion perception was not consistent across cases. Patient FD suffered damage near the posterior tip of the superior-temporal sulcus (STS), in a region that lay just dorsal to the likely location of MT+ in that patient (Vaina & Cowey, 1996; Vaina, Cowey & Kennedy, 1999), whereas patient JV suffered impaired perception of second-order motion after damage to parts of areas V2 and V3 (Vaina & Soloviev, 2004). (Other studies have found that unilateral damage to MT led to impaired perception of second-order motion displays (Plant & Nakayama, 1993).) Second, damage to parts of V2 and V3 has also been observed to produce the opposite result; namely, specific deficits of first-order motion while sparing second-order motion (RA and TF; Vaina et al., 1998; Vaina et al., 2000). It is unclear why the same area would be specialized for one type of motion for one patient and the other type for another patient (Vaina & Soloviev, 2004). Third, studies that have tested a large number of patients have found that those with posterior temporal or parietal damage show quite correlated deficits in their ability to perceive first- and second-order motion, leading to the conclusion that there is substantial overlap between first-order and second-order motion systems (Greenlee & Smith, 1997; Rizzo et al., 2008). Finally, a TMS study of healthy participants found that disruption of activity in either V2/V3 or V5/MT+ areas led to impairments in the perception of both first-order and second-order motion (Cowey, Campana, Walsh & Vaina, 2006). Taken together, although there are suggestions of a possible double dissociation from single case studies, studies of larger groups of patients favor the proposal that many visual areas work in concert to process the motion of both first-and second-order stimuli.

We consider our fMRI results to support the view that first- and second-order motion stimuli are processed by common direction-selective mechanisms at early stages of processing, including the primary visual cortex. However, our results are also consistent with the possibility that second-order motion is processed by a higher-level brain area, which then leads to the feedback of a motion signal to low-level visual areas. The notion that motion perception could be driven by a higher-level mechanism, such as attentionally tracking the change in position of an object or feature over time, is an old one (Al-Haytham, 1056; Wertheimer, 1912; Anstis, 1980; Cavanagh, 1992). Psychophysical studies have measured the contribution of high-level mechanisms, showing that second-order motion perception is strongly driven by feature tracking (Seiffert & Cavanagh, 1998; 1999; Allen & Derrington, 2000; Ukkonen & Derrington, 2000; Ashida, Seiffert & Osaka, 2001; see Derrington, Allen & Delicato 2004, for review). Regions of the posterior parietal cortex, such as the intraparietal lobule, have been implicated in high-level motion perception (Battelli et al., 2001; Claeys et al., 2003; Federspiel et al., 2006; Ruff, et al., 2008), and it is conceivable that such parietal areas could send feedback signals leading to direction-selective responses in early visual areas. Other candidate brain areas implicated in high-level motion perception include area MT+ as well as the posterior superior temporal sulcus, which has been implicated in biological motion perception (Grossman & Blake, 2001; 2002; Noguchi et al., 2005; Vaina & Dumoulin, 2011). The possible role of feedback is an intriguing one that remains to be explored in future studies. To date, it has yet to be shown that feedback mechanisms are necessary to observe direction-selective responses to second-order motion, and such responses have been commonly observed in neurophysiological recordings in anaesthetized animals (O’Keefe & Movshon, 1998; Baker, 1999).

From the perspective of fMRI decoding, the present study illustrates how the pattern classification approach can be extended to measure generalization performance as a means to probe the organization of perceptual or cognitive representations (cf. Tong & Pratte, in press). In earlier work, Kamitani and Tong (2005) showed that activity patterns elicited by single orientations could reliably predict which of two overlapping orientations was being attended by the subject, a novel demonstration of a basic form of visual mind reading. Generalization performance has also been used to investigate the similarity of cortical responses for seen and remembered visual patterns (Harrison & Tong, 2009; Serences et al., 2009), auditory responses to phonemes spoken by different individuals (Formisano et al., 2008), and even generalization of large-scale differences in mental states across individuals (Poldrack et al., 2009). This approach has also been extended to understand the similarity structure of cortical representations to wide-ranging sets of visual stimuli (Kriegeskorte et al., 2008; Naselaris et al., 2009) and semantic stimuli (Mitchell et al., 2008). By considering how fMRI pattern analysis can be used not only to discriminate between a small set of brain states, but further, to investigate the functional similarities and differences between brain states, it may be possible to develop a better understanding of the underlying functional representations that subserve human perception and cognition.

Highlights.

1. fMRI pattern classification reveals common neural substrates for motion perception

2. Direction selectivity for first- and second-order motion found in all visual areas

3. Very similar direction-selective responses found for both types of motion