The human visual system preserves the hierarchy of two-dimensional pattern regularity

Abstract

Symmetries are present at many scales in natural scenes. Humans and other animals are highly sensitive to visual symmetry, and symmetry contributes to numerous domains of visual perception. The four fundamental symmetries—reflection, rotation, translation and glide reflection—can be combined into exactly 17 distinct regular textures. These wallpaper groups represent the complete set of symmetries in two-dimensional images. The current study seeks to provide a more comprehensive description of responses to symmetry in the human visual system, by collecting both brain imaging (steady-state visual evoked potentials measured using high-density EEG) and behavioural (symmetry detection thresholds) data using the entire set of wallpaper groups. This allows us to probe the hierarchy of complexity among wallpaper groups, in which simpler groups are subgroups of more complex ones. We find that both behaviour and brain activity preserve the hierarchy almost perfectly: subgroups consistently produce lower-amplitude symmetry-specific responses in visual cortex and require longer presentation durations to be reliably detected. These findings expand our understanding of symmetry perception by showing that the human brain encodes symmetries with a high level of precision and detail. This opens new avenues for research on how fine-grained representations of regular textures contribute to natural vision.

1. Introduction

Symmetries are abundant in natural and man-made environments, due to a complex interplay of physical forces that govern pattern formation in nature. Sensitivity to symmetry has been demonstrated in a number of species, including bees [1], fish [2,3], birds [4,5] and dolphins [6], and may be used as a cue for mate selection in many species [7], including humans [8]. Human cultures have created and appreciated symmetrical patterns throughout history, and since the gestalt movement of the early 20th century, symmetry has been recognized as important for visual perception. Symmetry contributes to the perception of shapes [9,10], scenes [11] and surface properties [12]. This literature is almost exclusively based on stimuli in which one or more symmetry axes are placed at a single point in the image. Focus has been on mirror symmetry or reflection, with relatively few studies including the other fundamental symmetries: rotation, translation and glide reflection [13]—perhaps because reflection has been found to be more perceptually salient [14–18] and produce more brain activity [19–22]. In the current study, we take a different approach by investigating visual processing of regular textures in which combinations of the four fundamental symmetries tile the two-dimensional plane.

In the two spatial dimensions relevant for images, symmetries can be combined in 17 distinct ways, known as the wallpaper groups [23–25]. Previous work on a subset of four of the wallpaper groups used functional MRI to demonstrate that rotation symmetries in wallpapers elicit parametric responses in several areas in the occipital cortex, beginning with visual area V3 [26]. This effect was also robust when symmetry responses were measured with electroencephalography (EEG) using both steady-state visual evoked potentials (SSVEPs) [26] and event-related potentials [27]. The SSVEP technique uses periodic visual stimulation to produce a periodic brain response that is confined to integer multiples of the stimulation frequency known as harmonics. SSVEP response harmonics can be isolated in the frequency domain and depending on the specific design, different harmonics will express different aspects of the brain response. [28]. Here, we expand on previous work by collecting SSVEPs and psychophysical data from human participants viewing the full set of wallpaper groups. We measure responses in visual cortex to 16 out of the 17 wallpaper groups, with the 17th serving as a control stimulus. Our goal is to provide a more complete picture of how wallpaper groups are represented in the human visual system.

A wallpaper group is a topologically discrete group of isometries of the Euclidean plane (i.e. transformations that preserve distance [24]). The wallpaper groups differ in the number and kind of these transformations, and we can uniquely refer to different groups using crystallographic notation. In brief, most groups are notated by PXZ, where X ∈ {1, 2, 3, 4, 6} indicates the highest order of rotation symmetry and Z ∈ {m, g} indicates whether the pattern contains reflection (m) or glide reflection (g). For example, P4 contains rotation of order 4, while P4MM contains rotation of order 4 and two reflections. By convention, many of the groups are given shortened names: for example, P4MM is usually referred to as P4M, as the second reflection can be deduced from the presence of rotation of order 4 alongside a reflection. Two of the groups start with a C rather than a P (CM and CMM), which indicates that the symmetries are specified relative to a cell that itself contains repetition. Full details of the naming convention can be found on Wikipedia [29] and examples of the wallpaper groups are shown in figures 1 and 2.

Figure 1.

Subgroup relationships with indices 2 (solid lines) and 3 (dashed line) are shown in (a). All other relationships can be inferred by identifying the shortest path through the hierarchy, and multiplying the subgroup indices. For example, P1 is related to P6 through P6 → P3 (index 2) and P3 → P1 (index 3) so P1 is a subgroup of P6 with index 3 × 2 = 6. We also show enlarged versions of some of the subgroup relationships involving (b) P6 and (c) PMM, and highlight the symmetries within the subgroups to emphasize how the supergroup can be generated by adding additional transformations to the subgroup. Illustration adapted from Wade [30].

Figure 2.

Examples of each of the 16 wallpaper groups are shown in the left- and right-most column of the figures, next to the corresponding SSVEP (centre-left) and psychophysical (centre-right) data from each group. The SSVEP data are odd-harmonic-filtered cycle-average waveforms. In each cycle, a P1 exemplar was shown for the first 600 ms, followed by the original exemplar for the last 600 ms. Error bars are standard error of the mean. Psychophysical data are presented as boxplots reflecting the distribution of display duration thresholds. The 16 groups are ordered by the strength of the SSVEP response, to highlight the range of response amplitudes.

In mathematical group theory, when the elements of one group are completely contained in another, the inner group is called a subgroup of the outer group [24]. The full list of subgroup relationships is listed in §1.4.2 of the electronic supplementary material. Subgroup relationships between wallpaper groups can be distinguished by their indices. The index of a subgroup relationship is the number of cosets (i.e. the number of times the subgroup is found in the supergroup [24]). As an example, let us consider groups P2 and P6 (see figure 1b). If we ignore the translations in two directions that both groups share, group P6 consists of the set of rotations {0°, 60°, 120°, 180°, 240°, 300°}, in which P2 {0°, 180°} is contained. P2 is thus a subgroup of P6, and P6 can be generated by combining P2 with rotations {0°, 120°, 240°}. Because P2 is repeated three times in P6, P2 is a subgroup of P6 with index 3 [24]. Similarly, PMM contains two reflections and rotations {0°, 180°}. PMM can be generated by adding an additional reflection to both P2 ({0°, 180°}) and PM (one reflection), so P2 and PM are both subgroups of PMM with index 2 (see figure 1c). The 17 wallpaper groups thus obey a hierarchy of complexity where simpler groups are subgroups of more complex ones [31].

The two datasets we present here (data and analysis code has been made available on OSF) make it possible to assess the extent to which both behaviour and brain responses follow the hierarchy of complexity expressed by the subgroup relationships. Based on previous brain imaging work showing that patterns with more axes of symmetry produce greater activity in visual cortex [26,27,32–34], we hypothesized that more complex groups would produce larger SSVEPs. For the psychophysical data, we hypothesized that more complex groups would lead to shorter symmetry detection threshold display durations, based on previous data showing that under a fixed presentation time, discriminability increases with the number of symmetry axes in the pattern [35]. Our results confirm both hypotheses and show that activity in the human visual cortex is remarkably consistent with the hierarchical relationships between the wallpaper groups, with SSVEP amplitudes and psychophysical thresholds following these relationships at a level that is far beyond chance. The human visual system thus appears to encode all of the fundamental symmetries using a representational structure that closely approximates the subgroup relationships from group theory.

2. Results

The stimuli used in our two experiments were generated from random-noise textures, which made it possible to generate multiple exemplars from each of the wallpaper groups, as described in detail elsewhere [26]. We generated control stimuli matched to each exemplar in the main stimulus set, by scrambling the phase but maintaining the power spectrum. All wallpaper groups are inherently periodic because of their repeating lattice structure. Phase scrambling maintains this periodicity, so the phase-scrambled control images all belong to group P1 regardless of group membership of the original exemplar. P1 contains no symmetries other than translation, while all other groups contain translation in combination with one or more of the other three fundamental symmetries (reflection, rotation, glide reflection) [24]. In our SSVEP experiment, this stimulus set allowed us to isolate brain activity specific to the symmetry structure in the exemplar images from activity associated with modulation of low-level features, by alternating exemplar images and control exemplars. In this design, responses to structural features beyond the shared power spectrum, including any symmetries other than translation, are isolated in the odd harmonics of the image update frequency [26,28,36]. Thus, the combined magnitude of the odd harmonic response components can be used as a measure of the overall strength of the visual cortex response.

The psychophysical experiment took a distinct but related approach. In each trial an exemplar image was shown with its matched control, one image after the other, and the order varied pseudo-randomly such that in half the trials the original exemplar was shown first, and in the other half, the control image was shown first. After each trial, participants were instructed to indicate whether the first or second image contained more structure. The duration of both images was controlled by a staircase procedure so that a threshold duration for symmetry detection could be computed for each wallpaper group.

Examples of the wallpaper groups and a summary of our brain imaging and psychophysical measurements are shown in figure 2. For our primary SSVEP analysis, we only considered EEG data from a pre-determined region-of-interest (ROI) consisting of six electrodes over occipital cortex (see electronic supplementary material, figure S1.1). SSVEP data from this ROI were filtered so that only the odd harmonics that capture the symmetry response contribute to the waveforms. While waveform amplitude is quite variable among the 16 groups, all groups have a sustained negative-going response that begins at about the same time for all groups, 180 ms after the transition from the P1 control exemplar to the original exemplar. To reduce the amplitude of the symmetry-specific response to a single number that could be used in further analyses and compared to the psychophysical data, we computed the root-mean-square (RMS) over the odd-harmonic-filtered waveforms. The data in figure 2 are shown in descending order according to RMS. The psychophysical results, shown in box plots in figure 2, were also quite variable between groups, and there seems to be a general pattern where wallpaper groups near the top of the figure, that have lower SSVEP amplitudes, also have longer psychophysical threshold durations.

We now wanted to test our two hypotheses about how SSVEP amplitudes and threshold durations would follow subgroup relationships, and thereby quantify the degree to which our two measurements were consistent with the group theoretical hierarchy of complexity. We tested each hypothesis using the same approach. We first fitted a Bayesian model with wallpaper group as a factor and participant as a random effect. We fit the model separately for SSVEP RMS and psychophysical data and then computed posterior distributions for the difference between supergroup and subgroup. These difference distributions allowed us to compute the conditional probability that the supergroup would produce (a) larger RMS and (b) shorter threshold durations, when compared to the subgroup. The posterior distributions are shown in figure 3 for the SSVEP data and in figure 4 for the psychophysical data, with distributions colour-coded according to conditional probability. For both data sets, our hypothesis is confirmed: For the overwhelming majority of the 63 subgroup relationships, supergroups are more likely to produce larger symmetry-specific SSVEPs and shorter symmetry detection threshold durations, and in most cases, the conditional probability of this happening is extremely high.

Figure 3.

Posterior distributions for the difference in mean SSVEP RMS amplitude. Indices are shown in parentheses next to each subgroup relationship. The shading of the filled distribution relates to the conditional probability that the difference in means is greater than zero. We can see that 55/63 subgroup relationships have p(Δ > 0 | data) > 0.99.

Figure 4.

Posterior distributions for the difference in mean symmetry detection threshold durations. Indices are shown in parentheses next to each subgroup relationship. The shading of the filled distribution relates to the conditional probability that the difference in means is smaller than zero. We can see that 43/63 subgroup relationships have p(Δ < 0 | data) > 0.99.

We also ran a control analysis using (1) odd-harmonic SSVEP data from a six-electrode ROI over parietal cortex (see electronic supplementary material, figure S1.1) and (2) even-harmonic SSVEP data from the same occipital ROI that was used in our primary analysis. By comparing these two control analyses to our primary SSVEP analysis, we can address the specify of our effects in terms of location (occipital cortex versus parietal cortex) and harmonic (odd versus even). For both control analyses (plotted in electronic supplementary material, figures S3.3 and S3.4), the correspondence between data and subgroup relationships was substantially weaker than in the primary analysis. We can quantify the strength of the association between the data and the subgroup relationships, by asking what proportion of subgroup relationships that reach or exceed a range of probability thresholds. This is plotted in figure 5, for our psychophysical data, our primary SSVEP analysis and our two control SSVEP analyses. It shows that odd-harmonic SSVEP data from the occipital ROI and symmetry detection threshold durations both have a strong association with the subgroup relationships such that a clear majority of the subgroups survive even at the highest threshold we consider (p > 0.99). The association is far weaker for the two control analyses.

Figure 5.

Proportion of subgroup relationships that satisfy p(Δ > 0 | data) > x for the SSVEP data and p(Δ < 0 | data) > x for the psychophysical data. If we take x = 0.95 as our threshold, the subgroup relationships are preserved in $56/63=89\text{\%}$ and $48/63=76\text{\%}$ of the comparisons for the primary SSVEP and threshold duration datasets, receptively. This compares to $32/63=51\text{\%}$ and $22/63=35\text{\%}$ for the SSVEP control datasets. (Online version in colour.)

SSVEP data from four of the wallpaper groups (P2, P3, P4 and P6) were previously published as part of our demonstration of parametric responses to rotation symmetry in wallpaper groups [26]. We replicate that result using our Bayesian approach and find an analogous parametric effect in the psychophysical data (see electronic supplementary material, figure S4.1). We also conducted an analysis testing for an effect of index in our two datasets and found that subgroup relationships with higher indices tended to produce greater pairwise differences between the subgroup and supergroup, for both SSVEP RMS and symmetry detection thresholds (see electronic supplementary material, figure S4.2). The effect of index is relatively weak, but the fact that there is a measurable effect can nonetheless be taken as preliminary evidence that representations of symmetries in wallpaper groups may be compositional.

Finally, we conducted a correlation analysis comparing SSVEP and psychophysical data and found a reliable correlation (R² = 0.44, Bayesian confidence interval [0.28, 0.55]). The correlation reflects an inverse relationship: For subgroup relationships where the supergroup produces a much larger SSVEP amplitude than the subgroup, the supergroup also tends to produce a much smaller symmetry detection threshold. This is consistent with our hypotheses about how the two measurements relate to symmetry representations in the brain, and suggests that our brain imaging and psychophysical measurements are at least to some extent tapping into the same underlying mechanisms.

3. Discussion

Here, we show that beyond merely responding to the elementary symmetry operations of reflection [33,34] and rotation [26], the visual system represents the hierarchical structure of the 17 wallpaper groups, and thus every combination of the four fundamental symmetries (rotation, reflection, translation and glide reflection) which comprise the set of regular textures. Both SSVEP amplitudes and symmetry detection thresholds preserve the hierarchy of complexity among the wallpaper groups that is captured by the subgroup relationships [31]. For the SSVEP, this remarkable consistency was specific to the odd harmonics of the stimulus frequency that are known to capture the symmetry-specific response [26] and to electrodes in a region-of-interest (ROI) over occipital cortex. When the same analysis was done using the odd harmonics from electrodes over parietal cortex (electronic supplementary material, figure S3.3) or even harmonics from electrodes over occipital cortex (electronic supplementary material, figure S3.4), the data were substantially less consistent with the subgroup relationships (figure 5).

The current study uses 16 distinct wallpaper groups, while previous neuroimaging studies focused on a subset of 4 [26,27]. This represents a significant conceptual advance, because it makes it possible to investigate the complete subgroup hierarchy among the 17 groups and ask to what extent the hierarchy is reflected in brain activity. Our data provide a description of the visual system’s response to the complete set of symmetries in the two-dimensional plane. We do not independently measure the response to P1, but because each of the 16 other groups produce non-zero odd harmonic amplitudes (see figure 2), we can conclude that the relationships between P1 and all other groups, where P1 is the subgroup, are also preserved by the visual system. The subgroup relationships are in many cases not obvious perceptually, and most participants had no knowledge of group theory. Thus, the visual system’s ability to preserve the subgroup hierarchy does not depend on explicit knowledge of the relationships. Previous brain-imaging studies have found evidence of parametric responses with the number of reflection symmetry folds [32,33,37] and with the order of rotation symmetry [26]. Our study is the first demonstration that the brain encodes symmetry in this parametric fashion across every possible combination of different symmetry types, and that this parametric encoding is also reflected in behaviour. Previous behavioural experiments have shown that although naive observers can distinguish many of the wallpaper groups [38], they tend to sort exemplars into fewer (4–12) sets than the number of wallpaper groups, often placing exemplars from different wallpaper groups in the same set [39]. The two-interval forced choice approach we use in the current psychophysical experiment makes it possible to directly compare symmetry detection thresholds to the subgroup hierarchy, and reveals that not only can the 17 wallpaper groups be distinguished based on behavioural data, behaviour largely follows the subgroup hierarchy.

A large literature exists on the sustained posterior negativity (SPN), a characteristic negative-going waveform that is known to reflect responses to symmetry and other forms of regularity and structure [37]. The SPN scales with the proportion of reflection symmetry in displays that contain a mixture of symmetry and noise [40,41], and reflection, rotation and translation can produce a measurable SPN [19]. It has recently been demonstrated that a holographic model of regularity [42] can predict both SPN amplitude [37] and perceptual discrimination performance [43] for dot patterns that contain symmetry and other types of regularity. The available evidence suggests that the SPN and our SSVEP measurements are two distinct methods for isolating the same symmetry-related brain response: When observed in the time-domain, the symmetry-selective odd-harmonic responses produce similarly sustained waveforms (see figure 2), odd-harmonic SSVEP responses can be measured for dot patterns similar to those used to measure the SPN [36], and the one event-related study on the wallpaper groups also found SPN-like waveforms [27]. Future work should more firmly establish the connection and determine if the SPN can capture similarly precise symmetry responses as the SSVEPs presented here. It would also be worthwhile to ask if and how W can computed for our random-noise based wallpaper textures where combinations of symmetries tile the plane.

We observe a reliable correlation between our brain imaging and psychophysical data. This suggests that the two measurements reflect the same underlying symmetry representations in visual cortex. It should be noted that the correlation is relatively modest (R² = 0.44). This may be partly due to the fact that different individuals participated in the two experiments. It may also be related to the fact that participants where not doing a symmetry-related task during the SSVEP experiment, but instead monitored the stimuli for brief changes in contrast that occurred twice per trial (see Methods). Previous brain imaging studies have found enhanced reflection symmetry responses when participants performed a symmetry-related task [32,33,40]. It is possible that adding a symmetry-related task to our SSVEP experiment would have produced measurements that reflected subgroup relationships to an even higher extent than what we observed. On the other hand, our results are already close to ceiling (see figure 5) and adding a symmetry-related task may simply enhance SSVEP amplitudes overall without improving the discriminality of individual groups, as has been observed for reflection by [32]. Task-driven processing may be important for detecting symmetries that have been subject to perspective distortion, as suggested by SPN measurements [44] and somewhat less clearly in a subsequent functional MRI study [32]. Future work in which behavioural and brain imaging data are collected from the same participants, and task is manipulated in the SSVEP experiment, will help further establish the connection between the two measurements and elucidate the potential contribution of task-related top-down processing to the current results.

We also find an effect of index for both our brain imaging measurements and our symmetry detection thresholds. This means that the visual system not only represents the hierarchical relationship captured by individual subgroups but also distinguishes between subgroups depending on how many times the subgroup is repeated in the supergroup, with more repetitions leading to larger pairwise differences. Our measured effect of index is relatively weak. This is perhaps because the index analysis does not take into account the type of isometries that differentiate the subgroup and supergroup. The effect of symmetry type can be observed by contrasting the measured SSVEP amplitudes and detection thresholds for groups PM and PG in figure 2. The two groups are comparable except PM contains reflection and PG contains glide reflection, and the former clearly elicits higher amplitudes and lower thresholds. An important goal for future work will be to map out how different symmetry types contribute to the representational hierarchy.

The correspondence between responses in the visual system and group theory that we demonstrate here may reflect a form of implicit learning that depends on the structure of the natural world. The environment is itself constrained by physical forces underlying pattern formation, and these forces are subject to multiple symmetry constraints [45]. The ordered structure of responses to wallpaper groups could be driven by a central tenet of neural coding, that of efficiency. If coding is to be efficient, neural resources should be distributed to capture the structure of the environment with minimum redundancy considering the visual geometric optics, the capabilities of the subsequent neural coding stages and the behavioural goals of the organism [46–49]. Early work within the efficient coding framework suggested that natural images had a 1/f spectrum and that the corresponding redundancy between pixels in natural images could be coded efficiently with a sparse set of oriented filter responses, such as those present in the early visual pathway [50,51]. Our results suggest that the principle of efficient coding extends to a much higher level of structural redundancy—that of symmetries in visual images.

The 17 wallpaper groups are completely regular, and relatively rare in the visual environment, especially when considering distortions due to perspective (see above) and occlusion. Near-regular textures, however, abound in the visual world and can be modelled as deformed versions of the wallpaper groups [52]. The correspondence between visual cortex responses and group theory demonstrated here may indicate that the visual system represents visual textures using a similar scheme, with the wallpaper groups serving as anchor points in representational space. This framework resembles norm-based encoding strategies that have been proposed for other stimulus classes, most notably faces [53] and leads to the prediction that adaptation to wallpaper patterns should distort perception of near-regular textures, similar to the aftereffects found for faces [54]. Field biologists have demonstrated that animals respond more strongly to exaggerated versions of a learned stimulus, referred to as ‘supernormal’ stimuli [55]. In the norm-based encoding framework, wallpaper groups can be considered supertextures, exaggerated examples of the near-regular textures common in the natural world. If non-human animals employ a similar encoding strategy, they would be expected to be sensitive to symmetries in wallpaper groups. Recent functional MRI work in macaque monkeys offer some support for that: Macaque visual cortex responds parametrically to reflection and rotation symmetries in wallpaper groups, and the set of brain areas involved largely overlap those observed to be sensitive to symmetry in humans [56]. In human societies, visual artists may consciously or unconsciously create supernormal stimuli to capture the essence of the subject and evoke strong responses in the audience [57]. Wallpaper groups are visually compelling, and symmetries have been widely used in human artistic expression going back to the Neolithic age [58]. If wallpapers are in fact supertextures, this prevalence may be a direct result of the strategy the human visual system has adopted for texture encoding.

4. Methods

(a) . Participants

Twenty-five participants (11 females, mean age 28.7 ± 3.3) took part in the EEG experiment. Their informed consent was obtained before the experiment under a protocol that was approved by the Institutional Review Board of Stanford University. Eleven participants (8 females, mean age 20.73 ± 1.21) took part in the psychophysics experiment. All participants had normal or corrected-to-normal vision. Their informed consent was obtained before the experiment under a protocol that was approved by the University of Essex’s Ethics Committee. There was no overlap in participants between the EEG and psychophysics experiments.

(b). Stimulus generation

Exemplars from the different wallpaper groups were generated using a modified version of the methodology developed by Clarke and colleagues [39] that we have described in detail elsewhere [26]. Briefly, exemplar patterns for each group were generated from random-noise textures, which were then repeated and transformed to cover the plane, according to the symmetry axes and geometric lattice specific to each group. The use of noise textures as the starting point for stimulus generation allowed the creation of an almost infinite number of distinct exemplars of each wallpaper group. To make individual exemplars as similar as possible we replaced the power spectrum of each exemplar with the median across exemplars within a group. We then generated control exemplars that had the same power spectrum as the exemplar images by randomizing the phase of each exemplar image. The phase scrambling eliminates rotation, reflection and glide-reflection symmetries within each exemplar, but the phase-scrambled images inherent the spectral periodicity arising from the periodic tiling. This means that all control exemplars, regardless of which wallpaper group they are derived from, are transformed into another symmetry group, namely P1. P1 is the simplest of the wallpaper groups and contains only translations of a region whose shape derives from the lattice. Because the different wallpaper groups have different lattices, P1 controls matched to different groups have different power spectra. Our experimental design takes these differences into account by comparing the neural responses evoked by each wallpaper group to responses evoked by the matched control exemplars.

(c) . Stimulus presentation

For the EEG experiment, the stimuli were shown on a 24.5” Sony Trimaster EL PVM-2541 organic light emitting diode (OLED) display at a screen resolution of 1920 × 1080 pixels, 8-bit colour depth and a refresh rate of 60 Hz, viewed at a distance of 70 cm. The mean luminance was 69.93 cd/m2 and contrast was 95%. The diameter of the circular aperture in which the wallpaper pattern appeared was ${13.8}^{\circ }$ of visual angle presented against a mean luminance grey background. Stimulus presentation was controlled using in-house software. For the psychophysics experiment, the stimuli were shown on a 48 × 27 cm VIEWPixx/3D LCD Display monitor, model VPX-VPX-2005C, resolution $1920\times 1080\hspace{0.17em}\mathrm{pixels}$, with a viewing distance of approximately 40cm and linear gamma. Stimulus presentation was controlled using MatLab and Psychtoolbox-3 [59,60]. The diameter of the circular aperture for the stimuli was ${21.5}^{\circ }$.

(d) . EEG procedure

Visual-evoked potentials were measured using a steady-state design, in which P1 control images alternated with exemplar images from each of the 16 other wallpaper groups. Exemplar images were always preceded by their matched P1 control image. A single 0.83 Hz stimulus cycle consisted of a control P1 image followed by an exemplar image, each shown for 600 ms. A trial consisted of 10 such cycles (12 s) over which 10 different exemplar images and matched controls from the same rotation group were presented. For each group, the individual exemplar images were always shown in the same order within the trials. Participants initiated each trial with a button-press, which allowed them to take breaks between trials. Trials from a single wallpaper group were presented in blocks of four repetitions, which were themselves repeated twice per session, and shown in random order within each session. To control fixation, the participants were instructed to fixate a small white cross in the centre of display. To control vigilance, a contrast dimming task was employed. Two times per trial, an image pair (control P1 plus exemplar) was shown at reduced contrast. Participants were instructed to press a button on a response pad whenever they noticed a contrast change. Reaction times were not taken into account and participants were told to respond at their own pace while being as accurate as possible. We adjusted the reduction in contrast such that average accuracy for each participant was kept at 85% correct, in order to keep the difficulty of the vigilance task at a constant level.

(e) . Psychophysics procedure

The experiment consisted of 16 blocks, one for each of the wallpaper groups (excluding P1). We used a two-interval forced choice approach. In each trial, participants were presented with two stimuli (one of which was the wallpaper group for the current block of trials, the other being P1), one after the other (inter-stimulus interval of 700 ms). After each stimulus had been presented, it was masked with white noise for 300 ms. After both stimuli had been presented, participants made a response on the keyboard to indicate whether they thought the first or second image contained more symmetry. Each block started with 10 practice trials, (stimulus display duration of 500 ms) to allow participants to familiarize themselves with the current block’s wallpaper pattern. If they achieved an accuracy of 9/10 in these trials they progressed to the rest of the block, otherwise they carried out another set of 10 practice trials. This process was repeated until the required accuracy of 9/10 was obtained. The rest of the block consisted of four interleaved staircases (using the QUEST algorithm [61], full details given in the SI) of 30 trials each. On average, a block of trials took around 10 min to complete.

(f) . EEG acquisition and preprocessing

Steady-state visual evoked potentials (SSVEPs) were collected with 128-sensor HydroCell Sensor Nets (Electrical Geodesics, Eugene, OR) and were band-pass filtered from 0.3 to 50 Hz. Raw data were evaluated off line according to a sample-by-sample thresholding procedure to remove noisy sensors that were replaced by the average of the six nearest spatial neighbours. On average, less than 5% of the electrodes were substituted; these electrodes were mainly located near the forehead or the ears. The substitutions can be expected to have a negligible impact on our results, as the majority of our signal can be expected to come from electrodes over occipital, temporal and parietal cortices. After this operation, the waveforms were re-referenced to the common average of all the sensors. The data from each 12s trial were segmented into five 2.4 s epochs (i.e. each of these epochs was exactly 2 cycles of image modulation). Epochs for which a large percentage of data samples exceeded a noise threshold (depending on the participant and ranging between 25 and 50 μV) were excluded from the analysis on a sensor-by-sensor basis. This was typically the case for epochs containing artefacts, such as blinks or eye movements. Steady-state stimulation will drive cortical responses at specific frequencies directly tied to the stimulus frequency. It is thus appropriate to quantify these responses in terms of both phase and amplitude. Therefore, a Fourier analysis was applied on every remaining epoch using a discrete Fourier transform with a rectangular window. The use of two-cycle epochs (i.e. 2.4 s) was motivated by the need to have a relatively high resolution in the frequency domain, δf = 0.42 Hz. For each frequency bin, the complex-valued Fourier coefficients were then averaged across all epochs within each trial. Each participant did two sessions of eight trials per condition, which resulted in a total of 16 trials per condition.

(g) . SSVEP analysis

Response waveforms were generated for each group by selective filtering in the frequency domain. For each participant, the average Fourier coefficients from the two sessions were averaged over trials and sessions. The SSVEP paradigm we used allowed us to separate symmetry-related responses from non-specific contrast transient responses. Previous work has demonstrated that symmetry-related responses are predominantly found in the odd harmonics of the stimulus frequency, whereas the even harmonics consist mainly of responses unrelated to symmetry, that arise from the contrast change associated with the appearance of the second image [26,36]. This functional distinction of the harmonics allowed us to generate a single-cycle waveform containing the response specific to symmetry, by filtering out the even harmonics in the spectral domain, and then back-transforming the remaining signal, consisting only of odd harmonics, into the time-domain. For our main analysis, we averaged the odd harmonic single-cycle waveforms within a six-electrode region of interest (ROI) over occipital cortex (electrodes 70, 74, 75, 81, 82, 83). These waveforms, averaged over participants, are shown in figure 2. The same analysis was done for the even harmonics and for the odd harmonics within a six electrode ROI over parietal cortex (electrodes 53, 54, 61, 78, 79, 86; see electronic supplementary material, figure S1.1). The root-mean square values of these waveforms, for each individual participant, were used to determine whether each of the wallpaper subgroup relationships were preserved in the brain data.

(h) . Defining the list of subgroup relationships

In order to get the complete list of subgroup relationships, we digitized table 4 from Coxeter [31] (shown in electronic supplementary material, table S1.2). After removing identity relationships (i.e. each group is a subgroup of itself) and the three pairs of wallpapers groups that are subgroups of each other (e.g. PM is a subgroup of CM, and CM is a subgroup of PM), we were left with a total of 63 unambiguous subgroups that were included in our analysis.

(i) . Bayesian analysis of SSVEP and psychophysical data

Bayesian analysis was carried out using R (v. 3.6.1) [62] with the brms package (v. 2.9.0) [63] and rStan (v. 2.19.2 [64]). The data from each experiment were modelled using a Bayesian generalized mixed effect model with wallpaper group being treated as a 16-level factor and random effects for participant. The SSVEP data and symmetry detection threshold durations were modelled using log-normal distributions with weakly informative, $\mathcal{N}(0,2)$, priors. After fitting the model to the data, samples were drawn from the posterior distribution of the two datasets, for each wallpaper group. These samples were then recombined to calculate the distribution of differences for each of the 63 pairs of subgroup and supergroup. These distributions were then summarized by computing the conditional probability of obtaining a positive (or for the psychophysical data, negative) difference, p(Δ|data). For further technical details, please see the electronic supplementary material where the full R code, model specification, prior and posterior predictive checks, and model diagnostics, can be found.

Ethics

Informed consent was obtained from all participants prior to data collection. For the EEG experiment the protocol was approved by the Institutional Review Board of Stanford University. For the psychophysics experiment, the protocol was approved by the University of Essex’s Ethics Committee.

Data accessibility

Data from the EEG and psychophysics experiments have been made available with the electronic supplementary material on OSF. In accordance with best practices of Open Science, the electronic supplementary material includes all of the data and code required to run our analyses, as well as additional helpful figures and tables (see: https://osf.io/f3ex8/).

Authors' contributions

P.J.K.: Conceptualization, Investigation, Methodology, Supervision, Visualization, Writing-original draft, Writing-review & editing; A.D.F.C.: Conceptualization, Data curation, Investigation, Methodology, Visualization, Writing-review and editing.

Competing interests

We declare we have no competing interests.

Funding

This work was supported by the Vision Science to Applications (VISTA) program funded by the Canada First Research Excellence Fund (CFREF, 2016–2023) and by a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada awarded to P.J.K. The work was also partially supported by a National Science Foundation INSPIRE grant 1248076 awarded to Yanxi Liu, Anthony M. Norcia and Rick O. Gilmore.