Activity patterns in human motion-sensitive areas depend on the interpretation of global motion

Abstract

Numerous imaging studies have contributed to the localization of motion-sensitive areas in the human brain. It is, however, still unclear how these areas contribute to global motion perception. Here, we investigate with functional MRI whether the motion-sensitive area hMT⁺/V5 is involved in perceptual segmentation and integration of motion signals. Stimuli were overlapping moving gratings that can be perceived either as two independently moving, transparent surfaces or as a single surface moving in an intermediate direction. We examined whether motion-sensitive area hMT⁺/V5 is involved in mediating the switches between the two percepts. The data show differential activation of hMT⁺/V5 with perceptual switches, suggesting that these are associated with a reconfiguration of cell assemblies in this area.

In the human brain, multiple motion-sensitive areas have been recognized (1–11), and it is important to understand their respective functions. One question is where overlapping, moving contours are parsed into different objects or integrated into single moving objects, respectively.

Neurophysiological research in mammals has been essential in the generation of biologically plausible models of global motion perception (12–16). Primate area MT (middle temporal), for example, has been shown to contain a map for the direction of global motion (Fig. 1B; refs. 17–24). It is, however, unclear how MT integrates and/or segments global motion information and how MT motion maps shift between a single output representation and multiple representations of simultaneously moving objects (22, 24–27).

Figure 1.

(A) Plaid stimuli elicit two alternative perceptual interpretations and allow for the study of bistable perceptual transitions between integration (one-surface, pattern motion) and segmentation (two-surfaces, component motion). Arrows depict direction of perceived movements. Luminance of grating intersections is critical in determining perceptual outcome (see Methods). (B) The scheme depicts a hypothetical global motion map in hMT⁺/V5. The arrows illustrate preferred direction of global movement for neurons in each part of the map. Two dark, large arrows symbolize the possibility that multiple (2) moving surfaces might be simultaneously represented in the hMT⁺/V5 motion map. The small, dark arrow depicts one putative, single, active neural population that represents a single surface. Two-surface percepts are predicted to result in more hMT⁺ activity because two populations of neurons are active instead of one.

In a previous animal study, we applied plaid stimuli to examine neuronal correlates of such switching processes and found that changes in perceptual grouping go along with changes in neuronal synchrony (15). The goal of this whole-brain functional MRI study is to investigate the role of the human hMT⁺/V5 complex in perceptual processes that involve dynamic switching between perceptual integration and segmentation. Plaid stimuli (Fig. 1A) represent a perceptual paradigm that allows the investigation of switches between integration and segmentation of motion vectors (12–16). Two moving gratings (surfaces) are superimposed in the same display. Observers can perceptually segregate these surfaces and then perceive one grating sliding on top of the other (component motion). However, the two component surfaces also can be integrated perceptually into a single surface that moves in a direction intermediate to the motion directions of the component gratings (pattern motion). Certain configurations of plaid stimuli lead to bistable interpretations because of spontaneous switches between perceptual integration (pattern motion) and segregation (component motion).

This bistability permits identification of motion-sensitive areas that mediate perceptual transitions between component and pattern motion. These areas should show activity changes related to the perceptual shifts, even when the stimulus remains constant.

In the first experiment, we took advantage of the fact that perceptual switches can be induced by changing stimulus parameters such as local contrast. In the second experiment, subjects viewed a constant physical stimulus that allowed for bistable, perceptual alternations. In that case, only the internal percept changes and modulations of activity can be related to processes leading to different interpretations of a constant stimulus. Because subjects had to perform motor report on stimulus motion for both conditions, attention is ensured to be stable across conditions in this second experiment. In a third control experiment, we manipulated attention by using tasks that forced the perceiver either to use or ignore specific stimulus information.

Methods

Functional Imaging.

Functional imaging was performed in 10 subjects at 1.5 T (Philips ACS-NT; Philips, Eindhoven, The Netherlands) by using the standard head coil and a gradient echo planar imaging (EPI) sequence (TE = 40 ms, TR = 2,083 ms; FA = 90°, FOV = 224 × 224 mm², 22 slices, voxel size: 3.5 × 3.5 × 5 mm³). A T1-weighted 3D magnetization prepared rapid acquisition gradient echo or fast field echo scan was recorded in the same session as the functional measurements (voxel size = 1.0 × 1.0 × 1.0 mm³).

Some subjects also underwent additional experiments in a different scanner [1.5 T Siemens Magnetom Vision (Siemens, Iselin, NJ); gradient echo EPI sequence, TE = 69 ms, TR = 3,000 ms; FA = 90°, FOV = 210 × 210 mm², 15–20 slices, voxel size = 1.6 × 1.6 × 3–5 mm³ or TE = 38 ms, TR = 1,600 ms; FA = 90°, FOV = 210 × 210 mm², 10 slices, voxel size = 3.3 × 3.3 × 5 mm³]. In these subjects, an additional T1-weighted (T1-FLASH) 3D data set (voxel size = 1.0 × 1.0 × 1.0 mm³) tuned to optimize the contrast between gray vs. white matter was recorded in a separate recording session. These high-resolution, 3D recordings were used for surface reconstruction of the subjects' cortex. In two subjects, eye movement recordings were obtained during two scanning runs by means of an IR light-reflecting eye-tracking system (OBER 2; Iota, Sweden), with a resolution of 0.5°.

Experimental Paradigms.

Plaid stimuli were constructed by superimposing square-wave gratings with 0.3 average duty cycle and angles of approximately ±75° in a circular aperture and manipulating physical transparency by adjusting the luminance of the regions of intersection of the gratings according to previously described rules (28–30). We have used asymmetrical plaids, with opposite contrast polarity of dark and light stripes (≈12 and 36 cd·m⁻²) to intermediate background luminance (≈24 cd·m⁻²). This allowed subjects to perceive both component and pattern motion in alternation for stable periods. Typical intersection luminances were 30 cd·m⁻² for component (described as C1 in Figs. 2 and 3) and 97 cd·m⁻² for pattern stimuli. In the first experiment (see below), we also used a component stimulus with slightly darker intersections (C2 ∼ 10 cd·m⁻²). In the third experiment (see below), pattern-intersection luminance was increased to 147 cd·m⁻². Subjects (n = 10) underwent psychophysical testing before the recording sessions to determine the luminance window for perceptual transparency. The point(s) at which pattern motion occurred in approximate equiprobability with component motion were used for the sessions that required the subject to report the interpretation of a physically constant, ambiguous, bistable stimulus (three subjects). Plaid velocities (pattern velocity) were either 8° or 16°/s and were matched for pattern and component conditions.

Figure 2.

Baseline experiment: manipulation of subject's percepts by changing stimulus parameters (A Upper Right, typical pattern and C1 component stimuli described in Methods). (A) Moving plaids vs. static plaids contrast map (Upper) and component (two-surface) vs. pattern (one-surface) stimulus contrast (Lower). P(cor), corrected P values. In A, all colored voxels have P(cor) < 0.0001 [t(156) > 7.6), and in B, all colored voxels have P(cor) < 0.001 [t(156) > 6.00]. (B) Activity levels across time for two different motion paradigms (using either plaids or flow fields) for left hMT⁺/V5. Symbols for stimuli: St, static; Pt, pattern; C1 and C2, component; Fl, flow field motion; Fix, fixation cross.

Figure 3.

(A) Group analysis of individually defined regions of interest across subjects (each dot depicts data for one subject, except the last one, which shows pooled data). Note the gradient of activity levels depending on whether subjects perceive two moving surfaces (C), one moving surface (P), or no moving surface (S). Average activity is highest when two moving surfaces are perceived. P activity sometimes is blurred with S activity probably because of the initial motion aftereffect upon S presentation. This explanation seems likely because this blurring is diminished if an additional 2-s activity lag is introduced (control plot, Lower Left). (B) Levels of activity in hMT⁺/V5 correlate with number of perceived surfaces, as revealed further by time-resolved averages (each bar depicts overall subject data for time intervals of ≈4 s). Symbol conventions are as in Fig. 2.

Stimuli were generated by using the Direct X graphics library and were back-projected onto a screen positioned at the foot end of the scanner. In the first experiment (six subjects), four stimulus conditions (stationary plaids, component C1, component C2, and pattern), each lasting ≈16 seconds (eight scans), were alternated. Stationary stimuli were repeated six times, and motion stimuli were repeated five times in a balanced manner. In the second experiment (three subjects), only ambiguous plaids were used. Each functional run had the same length as in the first experiment, but now there was a single, physically constant stimulus during motion periods. These periods were alternated with 4 epochs during which stationary plaids were presented (≈16 s/period; see Fig. 4 A and C).

Figure 4.

(A) Ambiguous, bistable plaids (Upper Inset) lead to perceptual alternations between pattern and component motion. (Lower) Typical time courses in hMT⁺/V5 for ambiguous plaids experiments. Conventions are as in previous figures (except that now the colors describe the percept, not the stimulus, which is physically unchanged). Activity seems to peak in dark-gray periods (component). (B) Activity in hMT⁺/V5 is significantly and specifically correlated with activity in left and right motor cortices, as shown by the maps obtained from regression analysis by using motor time courses as predictors. Because motor time courses are correlated very reliably with the subject's perceptual reports, this suggests that hMT⁺/V5 is pivotal in perceptual decision. (C) Cortex-based ICA analyses converge to the same results as regression analysis. (Left) The sensorimotor network found by ICA includes hMT⁺/V5 (regions of interest—blue spots in surface reconstruction of same subject as in B). (Right) Time course of the whole network, showing again dominance of component motion activity. Retinotopic maps are shown as colored stripes in early visual areas. (D) Pattern (P) and component (C) motion responses in hMT⁺/V5 to an identical (ambiguous) stimulus as compared with a common rest baseline (nonrest baselines suffer from possible build-up of adaptation phenomena as well as from the occurrence of irregular perceptual switches). Component motion response is significantly higher than pattern motion response (P < 0.0001, paired t test). (Left) Activity in hMT⁺/V5 significant voxels as identified by analysis by using motor cortex time courses. (Right) Activity in hMT⁺/V5 significant voxels as identified by conventional GLM analysis.

Flow-field experiments were performed in eight subjects. During these measurements, two stationary stimuli (fixation cross and stationary dot pattern, repeated 4 and 10 times) and a motion condition (flow fields, repeated 5 times × 16 s) were alternated. Functional maps from this experiment were used to localize motion-sensitive areas in single subjects by using conventional stimuli (in addition to plaid motion contrast) and served in the data-driven analysis of the second experiment (see section below on data analysis).

In a final experiment addressing the issue of nonspecific arousal effects, subjects viewed unambiguous plaids (component or pattern) separated by a rest condition (fixation, ≈10 s) and performed either an angle or a color task (≈16 s). In the angle task, subjects had to report whether the angle of the fixation cross (changing every 2 s) was larger or smaller than the plaid angle. This task requires subjects to direct equal attention to both pattern and component stimuli, and the subject still must report on a plaid-related feature (angle between grating stripes). In the color task, stable attention was required on features not related to plaid stimuli. In this task, subjects had to report whether the fixation cross (changing every 2 s) had the same or different color from the curve outlining the stimulus aperture. Each task was repeated in eight blocks.

Data Analysis.

Data analysis included preprocessing (3D motion correction and spatial and temporal smoothing), coregistration and Tailarach transformation of scans, volume and surface rendering, cortex reconstruction, and morphing (inflation and flattening). These steps were performed by using brainvoyager 2000 software (31). For multiple regression analysis, a general linear model (GLM) with predictors for each experimental condition, i.e., component motion and pattern motion, was computed. The time courses of individual predictors for experiments that used unambiguous plaids (experiments 1 and 3) were obtained by using a linear model of the hemodynamic response (32). The overall model fit was assessed by using an F statistic. Significant differences between the experimental conditions were assessed by using contrast (t) maps. The obtained P values were corrected for multiple comparisons by using a cortex-based Bonferroni adjustment. Effects were accepted only as significant when P (corrected) < 0.001. In region-of-interest analyses, P values obtained from GLM fits were corrected further for serial correlations.

For the experiment with ambiguous plaids, the time of button press served as the basis of appropriate reference functions. After convolution with a model of the hemodynamic response (32), correlation analysis was applied. Functional MRI time series from this experiment were analyzed further by using a data-driven method [cortex-based, independent component analysis (cbICA)] (33). The voxels lying within a specified range of the white/gray matter border were tagged as “cortex” (34). This reduced data set was decomposed blindly into spatially independent components, each with an associated time course (35). Components presenting peak values in hMT⁺/V5, as defined in individual subjects on the basis of mapping experiments (flowfield mapping, moving vs. static plaids), were selected automatically and independently of their time courses. These latter were analyzed posthoc with reference to the subject's report. This approach does not rely on any a priori specification of the temporal profile of the BOLD response and removes potential interpretation biases caused, e.g., by the choice of a specific hemodynamic delay.

Retinotopic mapping experiments followed procedures described in ref. 36.

Results

Single-subject analysis of activity evoked by plaid motion, as assessed by multiple regression analysis, shows robust and specific activation of areas V3/V3A and hMT⁺/V5 (Fig. 2A). The time course of activation in these regions of interest was highly correlated (P < 0.0001, after correction for serial correlations) with a linear model contrasting responses to plaid motion with responses to a static plaid stimulus. In these clusters, activation was lowest for static stimuli, intermediate for pattern motion, and highest for component motion (Fig. 2B Left). When component motion was contrasted with pattern motion, clusters with a significantly higher response during component motion (P < 0.001, after correction for serial correlations) were observed within these same regions (Fig. 2A Lower Left).

The correspondence between activated clusters and areas V3/V3A and hMT⁺/V5 was confirmed by comparing their location with functional maps obtained with retinotopic and flow-field mapping experiments in the same subjects. Fig. 2B Right shows, for the same cluster identified as hMT⁺/V5, the time course of the response to flow-field motion stimuli.

Average data across subjects confirmed that pattern motion elicited, on average, lower activity in both V3/V3A and hMT⁺/V5 than component motion (Fig. 3A, which demonstrates that our main effect does not come from a single subject). There is a clear dependence of activity levels on the number of perceived moving surfaces: activity is lowest for static plaids, intermediate for pattern motion (one global moving surface), and highest for component motion (two transparently moving surfaces) in regions of interest defined individually for each subject (P < 0.0001 for component vs. pattern in left and right hMT⁺/V5, and P = 0.01 for left V3/V3A, ANOVA, and posthoc tests). Even when averages of activity in hMT⁺/V5 are resolved in finer time slices, the dependence of the BOLD response on the number of perceived surfaces remains clear (Fig. 3B). In area V3/V3A, this trend is not as robust and exhibits greater interindividual variability. Interestingly, the response to a pattern stimulus was, on average, lower if preceded by a static rather than by a component stimulus (P ≪ 0.01 for both left and right hMT⁺/V5). This effect could be a reflection of plaid-motion evoked adaptation phenomena (14, 37).

Based on the finding that activity in hMT⁺/V5 changes in relation with perceptual interpretation, we have used ambiguous bistable plaids to examine further the hypothesis that hMT⁺/V5 is mediating perceptual switches. This extension allowed us to assess whether activity changes in hMT⁺/V5 were due to internal changes of perceptual representations rather than to physical changes of the stimulus. We adapted ambiguous stimuli for each subject individually so that percepts would alternate with even likelihood between pattern (single surface) and component motion (two surfaces). Subjects reported their percept by means of a mouse button press. Having to report with different hands on the two percepts ensures that attention levels are the same in the two cases.

Fig. 4 shows results from these measurements when using ambiguous plaids. Motor time courses related to hand button presses were used in a correlation analysis to select areas whose activity is significantly related to perceptual decision. The highest positive correlation was found in area hMT⁺/V5 (Fig. 4B, with R = 0.6; P < 0.0001 even after correction for serial correlations, with R = 0.4). Typical time courses of perceptual reports and hMT⁺/V5 activity are shown in Fig. 4A. Activity levels are higher during episodes when subjects reported perceiving component rather than pattern motion. This difference is especially evident for the longer perceptual periods, whereas it is less obvious when perceptual periods are very short.

Fig. 4C highlights the value of ICA analysis as an alternative method to analyze data with irregular perceptual periods, as is the case for ambiguous stimuli. By using a selection criterion independent of the time courses of activity (see Methods), a spatial component including hMT⁺/V5 and sensorimotor-related regions could be detected (Fig. 4C Left). The corresponding time course and its correlation with the subject's report (Fig. 4C Right) suggest that this component reflects the activity of a functionally connected network of regions involved in perceptual decision and confirm the results obtained with correlation analysis.

To fairly compare activations induced by component and pattern-motion perception, it is crucial that comparisons are made with respect to a common baseline. Because switches are irregular and state of motion adaptation is likely to vary within hMT⁺/V5 along runs, it is important to compare to which extent pattern and component motion percepts elicit different activity levels when motion responses in hMT⁺/V5 are recruited from a common rest baseline. The reason is that pattern- and component-motion percepts likely involve redistribution of activity within hMT⁺/V5 with likely interactions across cells and complex adaptation effects. This problem is addressed and controlled for by the analysis shown in Fig. 4D. Component-motion activity elicited by the same stimulus after the same baseline rest fixation is higher than average pattern-motion activity (P < 0.0001, paired t test).

Although subjects were forced to report actively on both perceptual epochs in experiment 2, rendering nonspecific arousal effects unlikely, we performed additional experiments in four subjects to examine further the role of selective attention to features related/unrelated to the stimulus (Fig. 5). The first task (angle task; see Methods) forced equal attention to both pattern and component stimuli, and the subject still had to report on a plaid-related feature (angle between grating stripes). The second (color) task required stable attention on features not related to plaid stimuli.

Figure 5.

Average group activity in hMT⁺/V5 (A) and V3/V3A (B) in a control experiment that included different tasks manipulating attention (color and angle tasks; see Methods). In the angle task, which requires subjects to pay equal featural attention to both pattern and component conditions, the response to component motion in hMT⁺/V5 is significantly higher than the response to pattern motion. (ANOVA and post hoc tests, P = 0.02, for comparison between component- and pattern-motion response). C, component stimulus; P, pattern stimulus; F, fixation.

In the angle task, the hMT⁺/V5 response to component motion was significantly higher than pattern-motion response (Fig. 5A, group data). In the color task, there was no significant difference between both conditions. However, the component-motion response in hMT⁺/V5 probably was underestimated because of the higher contrast of the pattern-motion stimuli in this experiment. Unlike experiment 2, stimuli were unambiguous, so that attention could be focused on the task. Because of higher contrast, the pattern stimulus elicited higher responses in early areas (V1/V2) than the component stimulus. Therefore, the similar activation levels in hMT⁺/V5 in the color task still may be taken as an indication for a stronger response to component motion in this area. To test this hypothesis, we performed a group GLM analysis in four subjects to find the areas that had a significant bias for pattern vs. component motion. Only early areas were found [V1/V2, P < 0.001, corrected; t(1,954) = 5.6]. No early areas showed the opposite result (e.g., more modulation for component than for pattern motion). This indicates that in the color task, pattern-motion response also is reduced significantly in hMT⁺/V5 in comparison with the component-motion response. Area V3/V3A also seems to have its activity biased by higher pattern contrast, because activation differences for pattern and component stimuli were much less pronounced than in hMT⁺/V5, even for the angle task (Fig. 5B). This finding is in line with the data shown in Fig. 3.

Taken together, these results exclude a role for nonspecific arousal effects. However, we cannot exclude that feature-specific attention might have modulated motion processing in hMT⁺/V5, because the angle task still required focused attention for a relevant feature of the plaid.

Discussion

Our data provide evidence on the putative site of perceptual grouping operations underlying the switches between fusion and segregation of moving stimuli. Activity in the hMT⁺/V5 complex changes depending on whether subjects integrate all motion signals into the percept of a single surface or whether they segregate signals and perceive two transparent surfaces. Our results agree with microstimulation studies in animals (25, 26) that, depending on experimental strategy, have found evidence for both integration and segregation of signals originating from different loci in the motion map of areas MT/MST (medial superior temporal).

Interestingly, component motion induced higher activity, on average, than pattern motion. One possibility is that the perception of component motion is associated with the formation of two cell assemblies, each of which represents one of the two moving surfaces, whereas the perception of pattern motion requires the formation of only one cell assembly representing a single surface moving in an intermediate direction. Thus, perceiving component motion may be associated with activation of a larger pool of neurons than perceiving pattern motion, and this could account for a higher BOLD signal if one assumes a linear relationship between spike counts and BOLD signal (38, 39).

With ambiguous plaids, our data show no differential activation of V1 in relation to perceptual alternations between pattern and component motion. This agrees with the notion that responses in V1 reflect mainly stimulus contrast and are only weakly modulated by top-down effects (40). The only areas exhibiting high and consistent correlations with perceptual switches were V3/V3A and, in particular, hMT⁺/V5. Activation in V3/V3A is consistent with an earlier positron-emission tomography study that compared responses between single gratings and plaid stimuli (41).

In the second experiment that required a motor response to ambiguous stimuli, we could use the associated activity from motor cortex as an internal marker for the assessment of perception-related activation in visual areas. The finding that hMT⁺/V5 was the only area exhibiting differential activation in relation to the motor response suggests that a redistribution of activity in hMT⁺/V5 probably is causing the perceptual switches between component and pattern motion. A data-driven analysis of these experiments revealed activation patterns closely resembling those obtained with the hypothesis-driven approach, thus providing additional confirmation to the results. Our data provide no evidence that the perceptual switches might have been induced by top-down influences from other regions (42). A possible mechanism for the alternation of response patterns in hMT⁺/V5 could be competition between assemblies representing component and pattern motion, respectively, and direction-specific adaptation of the different populations of motion-sensitive neurons participating in the respective assemblies (14, 37, 43, 44).

How exactly the two surface representations are computed in hMT⁺/V5 is unknown, although animal studies suggest that two assemblies representing the component gratings might be segregated by their differential synchronization (15). Monkey MT contains a depth map (45), and, because the component gratings are perceived as superimposed in depth, it is conceivable that the two representations involve neurons tuned to different depths.

Results were similar regardless of whether subjects were viewing unambiguous or ambiguous stimuli or whether they had to perform a motor response. Attention was matched in ambiguous plaids experiments, because subjects had to report with different hands distinct perceptual conditions. Given the known relevance of attentional effects in motion processing (46–49) and to exclude nonspecific arousal effects as an explanation to our data, we have used an attention-matched task (angle task) in which subjects had to use similar stimulus-specific cues across conditions, with no change in results. In a second task in which subjects ignored plaids, responses differed between component and pattern motion only when normalized for pattern contrast. The difference between the plaid-related angle task and the plaid-unrelated color task suggests that attention specific to task-related features might influence activation state in hMT⁺/V5 and thereby bias perceptual decision.

In conclusion, our results indicate a close relation between activity changes in hMT⁺/V5 and perceptual switches involving differential binding of stimulus components moving in different directions. The suggestion that hMT⁺/V5 is involved in the simultaneous representation of multiple global directions is in agreement with recent physiological and psychophysical data (22–25, 27). It is also in line with observations that coding strategies in MT and medial superior temporal may shift between vector averaging and a winner-take-all mode even when stimulus conditions are constant (27). Such occurrence of dynamic reconfiguration of cell assemblies allows both for segregation and integration of disparate motion signals, respectively.

Abbreviations

cbICA

cortex-based independent component analysis

GLM

general linear model

hMT⁺/V5

human middle temporal complex

MT

middle temporal