Columnar clusters in the human motion complex reflect consciously perceived motion axis

Significance

Existing knowledge of how cortical responses link to conscious content in humans is either inferred from animal models or from human studies limited by lower spatial resolution. While previous studies could relate distinct categorical percepts (faces vs. places) to signal differences across brain areas, measuring responses at submillimeter resolution allowed us to link subcategory conscious percepts (vertical vs. horizontal motion) to amplitude changes of separate populations within the same brain area. Furthermore, preferences for horizontal and vertical motion were organized into columnar clusters. We pave the way for future studies investigating if columnar clusters represent subcategorical distinctions in conscious content different from motion or in high-level perceptual and cognitive phenomena.

Abstract

The specific contents of human consciousness rely on the activity of specialized neurons in cerebral cortex. We hypothesized that the conscious experience of a specific visual motion axis is reflected in response amplitudes of direction-selective clusters in the human motion complex. Using submillimeter fMRI at ultrahigh field (7 T) we identified fine-grained clusters that were tuned to either horizontal or vertical motion presented in an unambiguous motion display. We then recorded their responses while human observers reported the perceived axis of motion for an ambiguous apparent motion display. Although retinal stimulation remained constant, subjects reported recurring changes between horizontal and vertical motion percepts every 7 to 13 s. We found that these perceptual states were dissociatively reflected in the response amplitudes of the identified horizontal and vertical clusters. We also found that responses to unambiguous motion were organized in a columnar fashion such that motion preferences were stable in the direction of cortical depth and changed when moving along the cortical surface. We suggest that activity in these specialized clusters is involved in tracking the distinct conscious experience of a particular motion axis.

An important goal of neuroscience research is to dissociate neural signals pertaining to conscious perception from those related to sensory stimulation (1, 2). Experimentally, this can be achieved by exposing observers to multistable stimuli, that is, stimuli that lead to changes in perception despite unchanging stimulation of the senses (3–5). For example, the bistable motion quartet (Fig. 1A, Top) yields conscious perception of either horizontal or vertical motion (6). The perceived axis of motion switches every few seconds although the retinal stimulation remains constant. Only those neurons that modulate their activity with the perceived axis of motion should be considered content-specific neural correlates of consciousness (NCC) (7, 8).

Fig. 1.

Experimental paradigm and event-related averages. (A) Overview of experimental paradigm. During the “ambiguous motion” experiment (Top), the bistable motion quartet was presented. Presenting two opposed square pairs in temporal alternation gives rise to the conscious percept of either horizontal or vertical apparent motion. While the percept switches about every 10 s, the retinal stimulation remains constant throughout the experiment. During the “physical motion” experiment (Bottom), squares moved physically along the horizontal or vertical motion paths that were previously only perceived during the ambiguous motion experiment. Physical motion led to unambiguous horizontal or vertical percepts. (B) The physical motion stimulus elicited responses in area hMT+, here displayed on the inflated, left hemisphere of a representative subject (Bottom). These responses distinguished horizontal (red) and vertical (green) motion clusters. We studied responses in these clusters to the ambiguous motion experiment (Top); lines show event-related weighted averages across all subjects. Percent signal change is displayed during horizontal (red lines) and vertical (green lines) perceptual periods, as indicated by the subjects via button presses. At time point 0 s subjects reported a perceptual switch specifying whether they now perceived horizontal or vertical motion. Error bars represent the uncertainty of the mean.

Multistable paradigms have revealed strong correlations between the experience of visual motion and responses in both the monkey and human middle temporal visual complex (MT+ and hMT+) (2–4). When macaque monkeys are trained to signal their conscious percept during binocular rivalry, about 40% of MT+ neurons sampled during electrophysiological recordings modulate their spiking with the perceived direction of motion (1, 3). Equally, during the presentation of bistable figures, neurons in MT (9), medial superior temporal (MST), and parietal cortex (10) all show activity reflecting the consciously perceived direction of motion. Monkey areas MT and MST are also spatially organized into columns and clusters of neurons that are each tuned to a particular motion direction (11–13), yet it is unclear whether the NCC for motion direction is localized to these structures.

In humans, the fMRI response in area hMT+ as a whole reflects conscious motion perception. Responses are increased for apparent as opposed to flicker motion (14) and with reversals in perceived motion direction (15). When subjects indicate that they perceive horizontal component motion, the hMT+ amplitude is higher than for vertical pattern motion (16). Furthermore, attended motion directions can be decoded from area hMT+ (17), and hMT+ signals predict perceptual states of clockwise or counterclockwise motion during ambiguous structure from motion (18). Like monkey MT, hMT appears to be organized into columnar clusters (19). To reveal this organization, however, advances in MRI technology (20) and analysis strategies (21, 22) were necessary that allow measuring fMRI responses at submillimeter spatial resolution. Until now fMRI studies on conscious motion perception have used relatively low resolutions (18 to 54 mm³) and could not investigate whether responses in distinct clusters within hMT+ relate to experience of a specific motion axis.

We recorded submillimeter fMRI responses from area hMT+ while humans viewed the bistable motion quartet stimulus and reported the perceived axis of motion via button presses. We hypothesized that (i) hMT+, like monkey MT+, has distinct functional subunits that each modulate with the perceptual state of a particular motion axis and (ii) that these units display a spatial organization into columnar clusters.

Results

We first localized area hMT+ for every subject using standard moving and static dot stimuli (SI Appendix, SI Materials and Methods, Figs. S1 and S2, and Table S1) (23, 24). All subjects showed significant [corrected for multiple comparisons using false discovery rate, q(FDR) < 0.05] bilateral responses to moving dots at the posterior/dorsal limb of the inferior temporal sulcus, in keeping with previous reports (23, 25). We used a disambiguated version of the motion quartet (Fig. 1A, Bottom) to identify clusters within area hMT+ that preferred either horizontal or vertical motion. In this version of the motion quartet (hereafter called the “physical motion quartet”), squares moved physically either along the horizontal or vertical motion paths that were only perceived (i.e., constructed by the brain) in the ambiguous motion quartet. In response to this physical motion, we observed voxels in area hMT+ with a significant [q(FDR) < 0.05] preference for either horizontal or vertical motion [S1: t(1,492) > 2.80; S3: t(1,492) > 3.01; S7: t(1,492) > 2.69; S8: t(1,492) > 2.94; S9: t(1,792) > 2.71]. Based on these responses, we formed horizontal and vertical motion clusters for every subject (SI Appendix, Fig. S3).

Fig. 1B, Top shows amplitude modulations in these clusters during ambiguous motion averaged across participants. Amplitude modulations reflected the consciously perceived motion axis. When subjects indicated a perceptual switch from vertical to horizontal motion, the response increased in horizontal clusters and decreased in the vertical ones. Conversely, when subjects reported a switch from horizontal to vertical motion, signal increased in the vertical clusters and decreased in the horizontal ones. Fig. 2A demonstrates that we consistently found the same pattern of fMRI signal for each subject. Switches to the preferred percept of a given cluster led to increases in fMRI signal. Our permutation testing revealed that in four out of five horizontal clusters (S1: t = 3.86, P = 0.036; S3: t = 9.90, P < 0.001; S7: t = 4.21, P = 0.022; S8: t = 1.63, P = 0.444; S9: t = 2.73, P = 0.042; 1,000-fold permutation) and in four out of five vertical clusters (S1: t = 3.59, P = 0.054; S3: t = 3.15, P = 0.046; S7: t = 4.20, P = 0.018; S8: t = 7.09, P <0.001; S9: t = 6.09, P < 0.001; 1,000-fold permutation) the amplitude modulations reflected perceived motion axis (SI Appendix, Fig. S4). This finding was robust in 7 out of 10 clusters also when we varied the number of voxels included in the horizontal and vertical clusters (SI Appendix, Fig. S5).

Fig. 2.

Single-subject event-related averages for ambiguous and physical motion. (A) Lines show event-related sample means across perceptual periods during ambiguous motion. Error bars represent SD of the mean across perceptual periods. (B) Lines show event-related sample means across cross-validation folds during physical motion. Error bars represent SD of the mean across cross-validation folds. Both panels show responses for either horizontal clusters (on the respective left) or vertical clusters (on the respective right of the panel), separately for every subject (rows). Time courses (normalized to run average) are displayed during horizontal (red lines) and vertical (green lines) perceptual periods. At time point 0 s subjects reported a perceptual switch specifying whether they now perceived horizontal or vertical motion.

To determine to which extent the cluster signals reflected the perceived as opposed to the physical stimulus, we compared signal changes during physical and ambiguous motion. For the physical motion quartet both retinal and perceived motion axis changed, while for the ambiguous motion display only the perceived motion axis changed. Fig. 2B shows the average fMRI time course for every subject during the physical motion experiment. Comparison of responses during the two experiments revealed a similar qualitative pattern. Quantitatively, the amplitude modulations were larger for physical than for ambiguous motion (SI Appendix, Table S2).

Upon visual inspection we found voxel preferences between physical and ambiguous motion to be spatially overlapping (Fig. 3A). Thus, when voxels showed a horizontal preference during physical motion, they often showed the same preference during ambiguous motion. To assess the consistency in preference we computed correlations between preferences during the two experiments and found significant, positive correlations (Fig. 3B) in all subjects (S1: r = 0.25, P < 0.001; S3: r = 0.27, P < 0.001; S7: r = 0.27, P < 0.001; S8: r = 0.21, P < 0.001; S9: r = 0.22, P < 0.001; two-sided test). This consistency often stayed stable even when we varied the number of voxels included in our region of interest (ROI) over a larger range (SI Appendix, Fig. S6).

Fig. 3.

Consistency of preferences between physical and ambiguous motion. (A) Qualitative consistency. Preferences for either horizontal (red) or vertical (green) motion are shown on the transverse slice of a selected subject (S1, left area hMT+) during either physical (left) or ambiguous (right) motion. Perceptually brighter colors indicate a higher degree of preference. Preferences are overlaid on segmentation labels of white (light-gray voxels) and gray matter (dark-gray voxels). (B) Quantitative consistency. Displayed are the median bootstrapped correlation coefficients for t values of the contrast horizontal > vertical found for physical and ambiguous motion. Voxels were selected based on independent data from the hMT+ localizer. Every point represents the result of a single subject (S1, S3, S7, S8, and S9). Error bars represent the 2.5th and 97.5th percentile of the bootstrapped correlation coefficients.

To exclude the possibility that differences in retinotopic preferences are responsible for our observed effects, we conducted several control experiments. First, we showed that the identified clusters display expected axis-of-motion tuning on two independently acquired datasets (SI Appendix, SI Results and Fig. S7 A and B). Second, we estimated the visual field coverage for both horizontal and vertical clusters and did not find a difference in their coverage of the horizontal and vertical motion trajectory [S3: t(205) = 0.066, P = 0.948; S7: t(729) = 1.26, P = 0.209] (SI Appendix, Fig. S8). Third, we selected voxels for horizontal and vertical clusters based on horizontal and vertical motion conditions that were retinotopically matched (SI Appendix, Fig. S7C) and obtained responses in these clusters during ambiguous motion. In three out of three horizontal clusters (S1: t = 4.22, P = 0.002; S3: t = 9.35, P < 0.001; S9: t = 5.26, P < 0.001; 1,000-fold permutation) and in two out of three vertical clusters (S1: t = 3.86, P = 0.048; S3: t = −4.44, P = 0.018; S9: t = 3.44, P = 0.014; 1,000-fold permutation), the amplitude reflected perceived motion axis. This finding was robust to a varying number of voxels included in the clusters (SI Appendix, Fig. S7D).

Finally, we wondered whether preferences for horizontal and vertical motion showed a form of cortical organization akin to columns selective for a particular motion axis. To address this question, we zoomed in to the observed responses (Fig. 4A) by constructing regular, high-resolution grids that covered the identified hMT+ areas (Fig. 4B) (19, 21). The grid allowed us to visualize and quantify how motion preferences change in the direction of cortical depth and along the cortical surface. If motion preferences are organized in a columnar fashion, they should stay stable in the direction of cortical depth and change when moving along the cortical surface. Using Meng’s z-test (26), we found that physical motion preferences at corresponding points along the cortical depth were more correlated to each other than preferences at grid points separated by a similar distance along the cortical plane (Fig. 4C and SI Appendix, Fig. S9) (27) in all subjects (S1: z = 12.76, P < 0.001; S3: z = 11.54, P < 0.001; S7: z = 18.11, P < 0.001; S8: z = 3.80, P < 0.001; S9: z = 8.33, P < 0.001; two-sided test) (Fig. 4D). A comparison of directional spatial autocorrelation corroborated this finding (SI Appendix, Fig. S10). Preferences during ambiguous motion showed smaller and less consistent correlation differences (S1: z = −3.26, P = 0.001; S3: z = 6.09, P < 0.001; S7: z = 3.08, P = 0.002; S8: z = 7.73, P < 0.001; S9: z = −0.21, P = 0.837; two-sided test).

Fig. 4.

Horizontal and vertical columnar-like clusters for physical motion. Based on data from the physical motion experiment, voxels were colored in either red (horizontal motion preference) or green (vertical motion preference). (A) Functional hMT+ clusters shown on an individual brain. (B) Functional responses to physical motion were sampled using a regular 3D grid. Cortex axis 1 and 2 represent the cortical plane; the z axis represents cortical depth. (C) Correlation of axis preference sampled for deep and corresponding superficial depth level (Left) or at different nearby locations in deep depth level only. Results are shown for a selected subject and hemisphere (S7, left hemisphere). (D) Subjectwise 95% confidence intervals for the difference in z-transformed correlation coefficients within and across grid columns. Every interval represents the result of a single subject (S1, S3, S7, S8, and S9). col, column.

Discussion

Our results support findings of previous human fMRI studies (14–18) that area hMT+ makes up part of the content-specific NCC by demonstrating a link between the experience of visual motion and response amplitudes in area hMT+. Most importantly, our findings extend this idea by demonstrating involvement of specific horizontal and vertical hMT+ clusters in tracking conscious experience of a particular motion axis. Earlier human fMRI studies demonstrated a coupling between categorical contents of consciousness and activity in macroscopic human areas (28–30). In those studies the two competing perceptual states pertained to two different categories (e.g., face vs. building or face vs. object) and experience of these categories was shown to be reflected in the amplitude of different category-specialized areas like fusiform face area or parahippocampal place area. By contrast, in our study the two perceptual states fell within the same category (motion) and were distinguishable only by subordinate categorical differences (horizontal vs. vertical).

fMRI studies that have compared subcategory perceptual states (14–16) were limited by spatial resolution and could only distinguish the states by global amplitude differences of the same area. Although studies using multivariate pattern analysis (17, 18) detected small activation differences between two perceptual states, these studies did not unequivocally reveal the underlying spatial organization of such activation patterns (31) since the existence of two types of patterns does not imply the existence of two different neural populations (32). Problematically, this complicates interpretation since coarse-scale biases might account for found differences (33). In comparison, using submillimeter resolution, our study could link subcategory contents of consciousness to dissociative amplitude modulations in distinct populations within the same brain area.

Several studies indicate that not all signals that modulate with a reported conscious state reflect the conscious experience itself. Instead, at least in part, these signals reflect processes that precede or follow the experience, among them monitoring the experimental task, planning, and reporting (8, 34, 35). Although we cannot exclude the possibility that some of the response modulation we observed resulted from button presses to report the perceived axis of motion, this interpretation is not in line with findings that activity for reporting and task monitoring is localized to frontal, executive areas, and not to occipital visual cortex (35). Furthermore, button presses were counterbalanced across the two scanning sessions, making it unlikely that they caused the observed response modulations, which by design needed to be consistent across sessions.

Our study follows recent advances in ultrahigh-field MRI which now allow for probing functional responses at the level of mesoscopic structures such as columns and layers (21, 22, 36). Previous studies have identified columnar-like structures in humans in V1 (37, 38), V2 and V3 (27), V3a (39), and hMT (19), yet these studies did not use stimuli that allowed for dissociating neural signals pertaining to conscious perception from those related to sensory stimulation. Using a multistable stimulus, our study indicates that response modulations in columnar structures relate to subcategory contents of consciousness.

The concept of a cortical column has been subject to substantial debate, starting with the first discovery of columns in primary somatosensory cortex of cats (40), and its functional significance has been doubted (SI Appendix, SI Discussion). Problematically, the term is used in different ways and as a result the various types of columns that have been identified differ in their defining feature (function, cell constellation, connectivity, or myelin content), their extent, and spatial organization (41). Here, we have used an operational definition of columnarity, where functional preferences were required to be more stable along the vertical than the horizontal extent of cortex. To highlight the difference with microcolumns, which would be beyond the available resolution of current fMRI, or idealized hypercolumns, we have used the term “columnar clusters” instead of “columns” throughout.

An alternative explanation of our findings would be that the identified clusters do not reflect a preference for motion axis but for the retinotopic location of the two illusory motion paths. However, our control experiments showed (i) that the selected clusters display characteristic tuning to either the horizontal or vertical motion axis, (ii) that there is no evidence for a difference in population receptive field coverage of horizontal and vertical motion paths, and (iii) that even if voxels are selected based on a stimulus with two retinotopically identical conditions only distinguished by motion axis, we still observe modulations with the conscious percept. We believe that, taken together, the results rule out this alternative explanation.

Based on our findings, we suggest that the hMT+ clusters identified here constitute part of the content-specific anatomical neural correlate for experiencing motion axis. Future studies could examine whether similar, clustered organizations exist for subcategory contents of consciousness other than motion axis. The activity in many cortical areas has been shown to be linked to general stimulus categories, including orientations, body parts, houses, faces, words, bigrams, and letters. Some of these cortical areas are also known to display columnar organization (42). Such research would clarify how the activity of these known functional subunits relates to phenomenal distinctions in conscious content. Furthermore, we pave the way for studies to investigate if columnar clusters represent subordinate dimensions in other high-level phenomena like attention and memory.

Materials and Methods

Additional descriptions of methods and results can be found in SI Appendix. Data as well as scripts for data preprocessing and stimulus presentation are publicly available (43, 44).

Participants.

Nine healthy participants with corrected-to-normal vision were recruited for the study. All participants gave informed, written consent to participate in the experiment. The study was approved by the research ethics committee of the Faculty of Psychology and Neuroscience of Maastricht University.

Functional Data and Statistical Analyses.

To form single-subject event-related averages for the ambiguous motion experiment (Exp. 1), we converted every voxel’s response to percent signal change where the mean of the respective run time series served as baseline. Separately for each cluster, signals were first averaged across voxels and then grouped by time point during horizontal and vertical perceptual periods. This resulted in event-related averages from time point 0 s (subjects indicated a perceptual switch) until 10 s later (average length of a perceptual period), in steps of 2 s (our repetition time). In total, there were four event-related averages per subject (two clusters × two perceptual periods). Event-related averages for the physical motion experiment (Exp. 2) were created in a similar fashion, with the following modification. To avoid circularity, we used a leave-one-run-out cross-validation scheme where all runs but one were used to assign voxels to either the horizontal or vertical cluster and responses from the left-out run were averaged across the cross-validation folds.

To obtain event-related averages across all subjects and to take differences in number of perceptual periods per subject into account, we calculated a weighted mean per time point (45), according to

$${\overline{x}}_{tp}=\frac{\sum _{i=1}^n\frac{1}{{\sigma }_i^2}{x}_i}{\sum _{i=1}^n\frac{1}{{\sigma }_i^2}},$$ [1]

where $n$ is the number of subjects, $x_i$ is the mean per subject per time point, and ${\sigma }_i$ is the SD across perceptual periods per subject per time point. Uncertainty in ${\overline{x}}_{tp}$ for display of the error bars was calculated using error propagation (45) as

$${\sigma }_{tp}=1/\sqrt{\sum _{i=1}^n\frac{1}{{\sigma }_i^2}}.$$ [2]

To test for statistical significance of the amplitude modulations observed during the ambiguous motion experiment, we computed a general linear model (GLM) containing predictors for horizontal and vertical motion and calculated t values for the contrast horizontal > vertical in horizontal clusters and for the contrast vertical > horizontal in the vertical cluster (empirical t values). We then obtained a null distribution of t values by randomly permuting condition labels and rerunning the GLM analysis (1,000-fold permutation testing). If the empirical t value was above the 97.5th percentile of the null distribution, the modulation for a cluster was declared significant. This amounts to a two-sided hypothesis test at 0.05. To test how robust the observed effect was to a varying number of voxels in the clusters, we systematically varied the number of included voxels (100, 200, 300, 400, 500, and 1,000) and redid the analyses described above.

We expected (a subset of) voxels to show consistent preferences across Exps. 1 and 2 for either horizontal or vertical motion. To test this, we selected voxels whose time courses were significantly modulated by moving dots presented in the central aperture [q(FDR) < 0.05] during the hMT+ localizer experiment (SI Appendix). For every selected voxel we ran a GLM containing predictors for horizontal and vertical motion and calculated the t values for the contrast horizontal > vertical, separately for Exps. 1 and 2. Voxels showing a preference for horizontal motion thus had positive t values, while voxels preferring vertical motion had negative values. We calculated the correlation (Pearson’s r) between t values for Exps. 1 and 2, treating every voxel as a data point, and tested for statistical significance at an alpha level of 0.05. We estimated variability of correlation coefficients by bootstrapping the population of voxels included in the calculation (20,000 resamples). From the resulting distribution of 20,000 coefficients, we calculated the 2.5th and 97.5th percentile for the display of error bars. To test how robust correlation was to a varying number of voxels included in the analysis, we also systematically varied the number of voxels and redid the analyses. The number of voxels we varied over was 100, 200, 300, 400, 500, and 1,000.

To visualize and quantify the degree of columnarity, we used BrainVoyager’s grid sampling approach. This approach takes as inputs a cortical thickness map and an ROI definition to create separate regular 2D grids at specified relative cortical depth levels (19, 21). Resulting points in a grid column fall on corresponding points in the cortical depth direction, taking the varying curvature and volume of cortex into account. We used the grid points to sample t values calculated for the contrast horizontal > vertical for the physical motion experiment in our hMT+ ROI. We then calculated the correlation (Pearson’s r) between t values for corresponding deep (0.9) and superficial (0.1) relative depth levels, treating every pair of corresponding points as a data point (27). We restricted the analysis to profiles where cortical thickness exceeded 1.6 mm to prevent that the same voxel contributed to both deep and superficial grid points. Mean cortical thickness values were between 2.56 and 2.97 mm (SI Appendix, Table S3). As a control, we calculated the correlation between t values for each grid point at the deep depth level and a randomly chosen grid point at the same (deep) depth level that was located, on average, 2.0 mm apart from the target grid point (27). We compared the two resulting correlation coefficients for each subject separately using Fisher’s method for comparing correlation coefficients, adjusted for correlated coefficients (26), and evaluated statistical significance at a level of 0.05.