Skip to main content
NCBI home page
Elsevier Sponsored Documents logoLink to Elsevier Sponsored Documents
. 2026 Aug;245:None. doi: 10.1016/j.visres.2026.108825

From pixels to perception: A benchmark for human-like symmetry detection☆

Gonzalo Muradás Odriozola a,b,c,⁎, Lisa Koßmann a,c, Tinne Tuytelaars b,c, Johan Wagemans a,c
PMCID: PMC13282018  PMID: 42090885

Abstract

Symmetry, a fundamental concept in nature, science and art, has challenged computer vision researchers because it occurs in various forms and human symmetry perception can deviate from the mathematical definition. Previous symmetry detection datasets are limited by the number of annotators and by missing the nuances of human perception. We introduce PIX2PER, a novel dataset for reflection symmetry in natural scenes and artworks. We also introduce WF1, a modified version of the widely-used F1 detection performance score, by adding weights to precision and recall to accommodate for the perceived symmetry strength. Created by adding weights to precision and recall to accommodate for the perceived symmetry strength. We perform a comparative analysis of existing models for symmetry detection on this human-centric dataset. Additionally, we present a fully synthetic dataset for pretraining symmetry detection models. When finetuning this pretrained model with human data, performance increases significantly. This research introduces and evaluates ways of improving symmetry detection and contributes to the development of computer vision models that more effectively represent human perception.

Keywords: Human symmetry perception, Benchmark dataset, Synthetic data, Computer vision, Natural images, Art images

Graphical abstract

Graphical abstract Image 1

Open in a new tab

Highlights

  • •

    We introduce PIX2PER dataset and WF1 metric for symmetry in images and artworks.

  • •

    We provide a comprehensive comparison of state-of-the-art models on the new benchmark

  • •

    We propose a hybrid training strategy, using synthetic data pretraining and human based fine-tuning.

1. Introduction

Symmetry is the elegant harmony that manifests itself in various aspects of the world. For centuries it has captivated the minds of scholars and artists alike, manifesting in phenomena ranging from natural formations to mathematical abstractions. In human vision, symmetry serves as a fundamental Gestalt principle, supporting figure–ground segmentation, object recognition, and shape detection (Machilsen et al., 2009, Wagemans, 1995). This study will focus on reflectional (mirror) symmetry. This form, particularly along the vertical axis, is the most salient and efficiently detected form of symmetry, but humans are also remarkably tolerant of distortions due to noise (Csatho et al., 2004, van der Helm, 2010), and skewing (Saunders and Knill, 2001, Sawada and Pizlo, 2008, Wagemans, 1993) that cause deviations from strict mathematical definitions. In addition, human symmetry perception is remarkable versatile, being able to handle nested symmetries (Nucci & Wagemans, 2007) and wide range of stimuli, such as landscapes, faces, shapes or flowers (Bertamini et al., 2019). This ability to perceive “approximate” or imperfect symmetry highlights the importance of studying symmetry as a perceptual phenomenon rather than merely a geometric one.

The exploration of human symmetry perception traces its origins back to the 19th century (Allen, 1879). Many of these investigations have involved presenting participants with stimuli containing varying degrees of symmetry and soliciting binary responses indicating the presence or absence of symmetry. Investigations have examined human capacity for detecting symmetry within random dense patterns (Pombo et al., 2023, Tyler, 1999, Tyler, 2001), under conditions of stress induced by alterations applied to symmetric structures (Atadjanov & Lee, 2018) and on the time taken for symmetry detection (Royer, 1981). Other studies have shown that symmetry is more salient when two orthogonal axes are present (Locher and Wagemans, 1993, Nucci and Wagemans, 2007), even under conditions of skewing (Wagemans et al., 1991). Additionally, some studies have explored the relationship between symmetry and other Gestalt principles, particularly figure–ground organization, suggesting that this connection enhances its salience (Machilsen et al., 2009). In our study, we expand upon this research by presenting over 900 participants with rich images of natural scenes and artworks and allowing them to freely indicate symmetric regions using bounding boxes and identify symmetry axes. This way of collecting symmetry detection data is quite different from the traditional experiments on human detection of symmetry (e.g., yes/no responses) in isolated patterns (e.g., dot patterns) or shapes (e.g., polygons).

In computer vision, symmetry detection remains a significant challenge, primarily due to the inherent variations and ambiguities present in real images (Liu et al., 2010, Seo et al., 2021). One of the explicit challenges with reflection symmetry arises from its potential manifestation across a broad spectrum of orientations, scales, and distortions. Furthermore, not all instances of mathematically symmetric structures in images warrant classification as symmetrical by humans. For example, an expanse solely comprising a perfect blue sky within a landscape may technically exhibit perfect symmetry based on quantitative metrics, yet it would not necessarily be classified as such by humans (Muradas et al., 2026).

Early studies employed feature detection techniques to extract keypoints, which were subsequently matched based on local features to generate pairs of symmetric points (Cho and Lee, 2009, Loy and Eklundh, 2006, Ogawa, 1991).

Recent advancements in symmetry detection methods primarily revolve around computing the likelihood of each pixel belonging to a symmetry axis (heatmaps), often leveraging machine learning techniques. In this context, early approaches introduced four-layer neural networks capable of generating dense heatmaps of symmetry axes for intricate patterns and line drawings (Fukushima, 2005, Fukushima and Kikuchi, 2006). Other methodologies employ feature detection techniques such as color, texture, and spectral clustering, followed by multiple instance learning to discern the scale and orientation of symmetric structures (Tsogkas & Kokkinos, 2012). Structured random forests (SRF) have also been employed to predict curved reflection symmetry, particularly when trained on patch-based symmetric features (Teo et al., 2015). Furthermore, some studies have drawn inspiration from how the visual cortex processes planar symmetry detection, employing a two-stage prediction approach involving the extraction of consistently oriented candidate and ranking them based on their surroundings (Gnutti et al., 2021).

Funk and Liu pioneered the application of deep neural networks (DNN) for symmetry detection by training versions of VGG-16 (Simonyan & Zisserman, 2014) and ResNet-101 (He et al., 2016), and comparing their accuracy against existing computer vision algorithms (Funk & Liu, 2017). Subsequently, polar matching convolutions were incorporated, which conducted feature matching to compute symmetry scores (Seo et al., 2021). Building upon this, EquiSym was proposed, a group-equivariant convolutional network, which achieved state-of-the-art results in symmetry detection (Seo et al., 2022). In this study, we compare these last two on our new benchmark.

The subsequent integration of DNNs into this domain (Funk & Liu, 2017) marked a significant advancement, enabling models to be trained using human-labeled data, thereby facilitating predictions more aligned with human perception. Nonetheless, this advancement introduced its own set of challenges, particularly the necessity for extensive datasets to adequately train state-of-the-art neural networks (Seo et al., 2021). Despite efforts to create datasets containing thousands of human-labeled images, there is often insufficient emphasis on the amount and diversity of agents labeling each instance. Consequently, each image typically possessed a solitary ground truth, derived from either a single individual or an average consensus among a group of individuals. Moreover, symmetrical entities were assigned binary strength values, irrespective of their perceptual salience or robustness as perceived by human agents.

Several symmetry detection datasets have been developed to facilitate research in this domain. While invaluable contributions to the investigation of symmetry, the first datasets were rather small in image and annotation volume (Cicconet et al., 2017, Funk et al., 2017, Lin et al., 2014, Liu et al., 2013, Tsogkas and Kokkinos, 2012).

Subsequent reflection symmetry datasets aimed to address the shortage of instances by significantly increasing the number of samples. The Beyond Planar Symmetry (BPS) dataset (Funk & Liu, 2017), consists of 1199 images sourced from MS-COCO. Reflection symmetries in this dataset are annotated by an average of 29.18 agents per image. The Learning to Discover Reflection Symmetry (LDRS) dataset (Seo et al., 2021), adhered to the protocol established by BPS. LDRS comprises 1110 training, 127 validation, and 240 testing instances, each featuring at least one symmetry axis. Additionally, LDRS-synth was introduced, augmenting the dataset by introducing synthetic symmetric elements to real images. This augmentation resulted in a substantial increase in the dataset size, providing a training split consisting of 83,000 images. Continuing this trend of dataset expansion, the dataset landscape was enhanced further with the creation of DENDI (Seo et al., 2022). DENDI consists of 2483 images, showcasing reflection symmetries with a diverse array of orientations and sizes of symmetry axes.

Most of the efforts towards symmetry detection have been directed towards real images (Funk et al., 2017, Funk and Liu, 2017, Liu et al., 2013, Seo et al., 2022). This concentration has left the detection of symmetry in artworks and other visual media relatively underexplored.

Variations exist regarding the number of annotators per dataset and instance. Furthermore, limited emphasis has been placed on the diversity and quantity of agents contributing to symmetry annotations, resulting in symmetries being uniformly regarded as equal irrespective of human perception. We address this gap by integrating the influence of human perception into both the dataset and its corresponding evaluation metric.

In this study, we gather data from over 900 participants, encompassing a novel compilation of images sourced from both natural scenes and artworks. Employing clustering algorithms on the collected data, we derive a comprehensive symmetry dataset characterized by symmetry axes and bounding boxes delineating symmetric elements within the images (Fig. 1). Crucially, our dataset incorporates assessments of perceived symmetry strength for each instance. With this dataset, we conduct a benchmarking analysis of state-of-the-art models, to compare them to human perceptual judgments of symmetry. Fig. 2 presents a visualization of the methodology of this study. Our main contributions are:

Fig. 1.

Fig. 1

Open in a new tab

Creation of PIX2PER’s instances and comparison with a single annotator dataset.

Fig. 2.

Fig. 2

Open in a new tab

Visualization of the two main studies performed in this paper. Creation of the benchmark dataset PIX2PER and pretraining a symmetry detection model with a novel synthetic dataset.

  • •

    Introducing a novel benchmark dataset, PIX2PER, and metric, WF1, tailored for evaluating symmetry detection models across a diverse range of images, including both natural scenes and artworks.

  • •

    Conducting a comparison of state-of-the-art deep learning architectures for symmetry detection against the newly introduced symmetry benchmark dataset.

  • •

    Proposing a hybrid training strategy pretraining on large-scale synthetic symmetry data followed by fine-tuning on PIX2PER to enable modern deep learning models to benefit from high-quality but relatively small perceptual datasets.

PIX2PER, as well as all data and code used to create it, including the synthetic dataset and clustering algorithm, will be made publicly available.

2. Methodology

2.1. Dataset

2.1.1. Experimental design

Data was collected through an online behavioral task. The Social and Societal Ethics Committee of our institution approved the study. Through the platform (Prolific, 2025), 901 participants (310 female, 578 male, 13 other) were recruited. Participants were asked to place a rectangular bounding box around an image region they perceived to contain reflection symmetry and to indicate the axis of symmetry, they had the freedom to adjust the size of the box to best fit the region. Participants could place as many bounding boxes per image as they saw fit. For each box, they also rated the perceived salience of the region (i.e., how much it popped-out from the background) and the strength of the symmetry (i.e., from rather imperfect to almost perfect symmetry). Additionally, we asked participants to rate the aesthetic pleasure of the entire image. Note that, for now, we did not use perceived salience, strength of symmetry and aesthetic pleasure although they could be useful for future in-depth analyses. The symmetry detection task and the rating task were presented in two different blocks the order of which was counterbalanced between participants. One participant saw either 20 (art-condition) or 25 (natural scenes condition) images, with the whole study encompassing 200 images of natural scenes and 200 images of artworks. Participants were randomly assigned to each condition first. Each image was rated by 25 participants per experiment for a total of 50 ratings. Basic demographic data such as gender, age, native language, colorblindness, education level, art-education, art-history education was also collected. The symmetry detection block took the average participant 30 min to complete. Participants annotated an average of 1.96 symmetry regions per image (1.46 standard deviation), around 49% of images were labeled with 2 or more boxes.

A limitation of the current framework is that symmetry is represented as a 2D axis, which cannot fully capture symmetries arising from three-dimensional structures under perspective distortions. Future work could address this by including more expressive annotation tools that allow participants to represent such symmetries more accurately.

Images were selected and prescreened for regions of symmetry from established image datasets by the authors. Two hundred natural images were chosen from the following databases: AADB (Kong et al., 2016), BSD (Martin et al., 2001), CAT2000 (Borji & Itti, 2015), COCO (T. Lin et al., 2014), EVA (Kang et al., 2020), FLICKR-AES (Ren et al., 2017), Oxford 102 Flower (Nilsback & Zisserman, 2010), MemCat (Goetschalckx & Wagemans, 2019), MIRFLICKR25 (Huiskes & Lew, 2008), VIU (Koehler et al., 2014). Two hundred artworks were taken from the LAPIS dataset (Maerten et al., 2025).

2.1.2. Clustering

The dataset was compiled from the contributions of at least 50 participants per image, each tasked with labeling a minimum of one symmetry region per instance. While participants often annotate similar symmetric regions, the drawn bounding boxes are not identical. To streamline the dataset and ensure coherence, the collected data underwent a clustering process, grouping them into varying numbers of symmetries per image. Each cluster retained information regarding the number of participants who identified a particular area as exhibiting symmetry.

Following data normalization, all bounding boxes were represented as six-dimensional vectors constrained within the range [−1, 1]. These vectors encoded the center coordinates, height, and width of the box, along with two values (cos, sin) denoting the rotation angle. This representation ensured that each bounding box could be compared in a standardized manner, capturing both spatial and orientational aspects of symmetry. The inclusion of trigonometric components for rotation made the representation robust to angle wrapping (e.g., distinguishing between 0°and 360°), which is crucial for consistent clustering in a high-dimensional space.

After some experimentation we opted for the density-based algorithm DBSCAN (Ester et al., 1996) for clustering. This method obviates the need for specifying the number of clusters a priori and may designate outlier points as noise if they lie beyond a certain distance threshold from any cluster. It remains a method of choice when clustering high dimensional data (Schubert et al., 2017). The algorithm required at least 3 instances for a group of instances to be considered a cluster. We set this minimum at 3 after testing the algorithm with different hyperparameters: this value provided enough variety in the resulting symmetric regions while still ensuring that each region was supported by a non-trivial number of participants.

The resulting clustered annotations are then used to train the models in a segmentation framework. In this setting, each clustered symmetry axis is treated as an individual segment from which the model learns and attempts to generalize. The output of the trained model is a mask of the same spatial dimensions as the input image, where each pixel value represents the probability of that location belonging to a symmetry axis.

2.1.3. Dataset statistics

After clustering the collected data, we created a dataset with an average of 6.49 individual symmetries per image. About 24.01% of the annotated bounding boxes were not clustered by the algorithm and were labeled as outliers. Fig. 3 illustrates the distribution of orientations for the symmetry axes in the collected data (a), as well as the distribution of the length of symmetry axes when normalized with the image diagonal (b).

Fig. 3.

Fig. 3

Open in a new tab

Statistical analysis of PIX2PER. (a) orientation distribution of the symmetry axes (b) distribution of axis length (c) distribution of number of symmetries.

2.2. Metric

To assess the accuracy of the models in alignment with the clustering-derived data, we introduce a slight modification of the F1-score employed by convention in most symmetry detection studies (Funk and Liu, 2017, Seo et al., 2022, Seo et al., 2021). Recall that the standard F1 score is the harmonic mean of precision and recall, providing a single measure that balances the trade-off between false positives and false negatives. In our modified version, symmetries annotated by a larger number of annotators carry a higher weight (i.e., proportional to the number of instances in the cluster). This numerical value is then used to compute a weight multiplier (Eq. (1)), ranging from 1 to a predefined maximum multiplier—2 in this study. The multiplier increases the score for predictions in regions with higher annotator agreement, effectively rewarding the model for correctly predicting symmetries that are more consistently perceived by humans.

We adapt the equations for calculating precision (Eq. (2)) and recall (Eq. (3)) by aggregating the product of each distinct axis with its corresponding weight multiplier (Eq. (1)) for true positives and false negatives.1 Consequently, to calculate the weighted F1-score (WF1) the equation remains intact, utilizing the weighted precision and weighted recall (Eq. (4)).

WM(NIC)=NIC−MSCNL−MSC∗(MM−1)+1 (1)
WP=∑i=1nTPi∗WM(NICi)∑i=1n[TPi∗WM(NICi)]+FP (2)
WR=∑i=1nTPi∗WM(NICi)∑i=1n(TPi+FNi)∗WM(NICi) (3)
WF1=2∗WP∗WRWP+WR (4)

In segmentation, True Positives (TP) are the pixels correctly classified as belonging to the target class, False Positives (FP) are pixels incorrectly classified as such. In PIX2PER the target symmetry axes are represented as segments of 10 pixels width. This should be modified to fit lower resolution studies. The metric is designed to accommodate predictions presented as probability maps that maintain the same dimensions as the original image, featuring normalized values spanning from 0 to 1 across all coordinates.

2.3. Generation of synthetic data

With the collected data we aimed to train a DNN based on human perception. However, PIX2PER, consisting of a total of 400 multiple symmetry images, proved insufficient for adequately training most current models. To address this limitation, we devised a novel tool for generating synthetically labeled symmetry datasets.

When developing symmetry detection models, a primary concern with DNNs pertains to their tendency for object recognition, potentially leading models to identify and label structures that are commonly perceived as symmetric, such as trees or houses, rather than strictly following the mathematical definition of symmetry. Seo et al. previously introduced synthetic data within their LDRS-synth dataset. However, this dataset involved the insertion of symmetric structures—derived from real images into other real images. In contrast, our objective was to create a fully synthetic dataset, generating entirely abstract symmetric and asymmetric figures. This approach necessitates the model to grasp the fundamental concept of symmetry, thereby mitigating the risk of over-reliance on recognizable structures.

To create a simple single symmetry, a digit from the MNIST (Deng, 2012) dataset is rotated and positioned atop a mirrored copy, maintaining a random distance within a range given by the desired output image size (20 pixels in our study) between them. Subsequently, both digits undergo additional rotation and resizing operations. During dataset generation, parameters for each step — including rotation angles, distances, and resizing ratios — are randomly selected. Similarly, generating a single asymmetry follows analogous steps, albeit utilizing two distinct digits instead of a mirrored copy of the same digit.

Additionally, we also generated double symmetries (structures with 2 perpendicular symmetry axis) by overlaying in an orthogonal orientation two single symmetries, separated by a random distance within a range. To introduce variability in perspective, skewing was applied by simulating distortions from random viewing angles. This was achieved by computing a transformation that mapped the original symmetric structure into a trapezoidal shape, followed by morphological operations to preserve structural coherence under the new viewpoint. We also included rotational asymmetries as additional noise structures composed of non-similar elements arranged in a circular configuration. Color gradients were applied by generating random gradient maps, which were overlaid on top of the structures to create smooth transitions of hue and intensity, adding further visual diversity. Sample textures from high-resolution images were used as backgrounds.

Using the tool for generating synthetic images, we generated a 10,000 instance dataset along with its masks for the labeled symmetries. The elements, positions, color gradients and backgrounds were randomly generated. Fig. 4 shows some examples of instances of the synthetic dataset. The distribution of the types of elements generated for the dataset follow the frequencies described on Table 1.

Fig. 4.

Fig. 4

Open in a new tab

Examples of the synthetic dataset, the symmetries on the two examples on the left have been labeled.

Table 1.

Frequency of appearance of elements in the synthetic database. Frequencies were determined experimentally, aiming to generalize to real examples.

Element Weight Frequency
Single reflection symmetry 8 21.62%
Double reflection symmetry 2 5.41%
Single skewed reflection symmetry 4 10.81%
Double skewed reflection symmetry 1 2.70%
Reflection asymmetry 20 54.05%
Rotational asymmetry 2 5.41%
Open in a new tab

Although the synthetically generated dataset provides a useful starting point, it only covers local structures inherent in handwritten strokes, which may bias the model towards simpler, more “handcrafted” symmetry patterns. Other more rich human-labeled images are still required to successfully train symmetry detection models.

3. Experimental analysis

3.1. Benchmarking deep learning models

We evaluate the performance of several state-of-the-art models on PIX2PER. We also include a ResNet-50 baseline (He et al., 2016) trained on the DENDI dataset (Seo et al., 2022). All models are evaluated with the set of 200 art images and the set of 200 real images (see Table 2).

Table 2.

Weighted F1-score comparison on our PIX2PER dataset when tested with the subset art images, natural images and the whole benchmark dataset.

Network Trained On Art Img. Natural Img. All Img.
ResNet-50 DENDI (Seo et al., 2022) 0.228 0.257 0.242
PMCNet (Seo et al., 2021) LDRS (Seo et al., 2021) 0.235 0.264 0.250
EquiSym (Seo et al., 2022) DENDI (Seo et al., 2022) 0.282 0.314 0.298
Human performance – 0.388 0.397 0.392
Open in a new tab

All models exhibited a performance improvement (approximately 3%) on natural images compared to the subset of artworks. This is likely due to the training datasets primarily containing real-world images. However, the networks presented in this study demonstrated an ability to generalize to artworks without a significant accuracy decline. PMCNet achieved marginally better scores than the baseline across all categories, with an average improvement of 0.74%. Notably, EquiSym emerged as the top performer on both categories, surpassing the baseline by 5.57%. See Fig. 5 for a visual comparison of predictions by the different models. It is noteworthy that the highest reported F1-scores for reflection symmetry detection on datasets utilized to train and test each respective model rarely exceed 0.65 (Seo et al., 2022, Seo et al., 2021). Consequently, the relatively low scores obtained on this novel metric with an unseen dataset are to be expected.

Fig. 5.

Fig. 5

Open in a new tab

Examples of predictions presented as heatmaps of the ResNet-50 (He et al., 2016) trained with DENDI (Seo et al., 2022), PMCNet (Seo et al., 2021), EquiSym (Seo et al., 2022) and the U-Net (Ronneberger et al., 2015).

Additionally we selected a random annotator for all images to measure their efficacy in symmetry detection within PIX2PER and the WF1 metric. Human evaluation on the benchmark dataset revealed minimal variation in performance between participants (around 0.4 WF1 score) and across image categories. PIX2PER encompasses a collection of the perception of multiple human participants, the different sensibilities for symmetry as well as the accuracy when labeling bounding boxes on symmetric objects could explain the obtained scores. Consequently, the task requires models to capture general perceptual patterns across annotators rather than reproducing the judgment of any one individual. Human annotators achieved significantly higher WF1 scores compared to the state-of-the-art models on the same dataset. This highlights the ongoing challenges in training deep learning models for robust symmetry detection.

The qualitative results of ResNet-50 (He et al., 2016), PMCNet (Seo et al., 2021), EquiSym (Seo et al., 2022) and U-Net (Ronneberger et al., 2015) trained with PIX2PER, with and without pretraining with synthetic data, are shown in Fig. 5. All the examples shown belong to the subset of testing data for the U-Net, hence, they are images it has never seen before.

3.2. Synthetically trained model

Utilizing the synthesized dataset, we pre-trained a U-Net (Ronneberger et al., 2015). Following training, the model achieved an accuracy of 99.16% and a Dice score of 0.82652 on the synthetic test set. Fig. 6 displays the ability for the model to generalize to real data in a wide range of symmetry axis orientations and lengths having been trained only on synthetic data.

Fig. 6.

Fig. 6

Open in a new tab

U-Net’s normalized predictions after being trained only on synthetic data.

After pretraining a U-Net with the synthetic dataset, it underwent fine-tuning on PIX2PER. 80% of the data was allocated for training, while the remaining 20% was used for testing and validation. All models were trained for 100 epochs with a learning rate of 0.0001, a batch size of 32, and utilizing the Adam optimizer (Kingma & Ba, 2014), see Table 3 for the performance results. Note that the U-Net was trained with only symmetry axes, consistent with all other standard deep learning approaches. Meaning that parallel overlapping symmetry axes in the same region are treated as a single axis in both training and prediction.

Table 3.

WF1 of a U-Net trained with the synthetic dataset and fine tuned with the PIX2PER.

Pretrained FineTuned WF1 on Art Img. WF1 on Natural Img. WF1 on all Img.
Yes No 0.011 0.026 0.019
No Yes 0.289 0.311 0.300
Yes Yes 0.403 0.430 0.416
Open in a new tab

There is a significant jump in performance between the model only trained with the benchmark dataset and the model that was previously pretrained with the synthetic data, around 10%. However, it is essential to note that comparing this model to current state-of-the-art models is not entirely fair. State-of-the-art models have been trained on different datasets with distinct labeling conventions and dataset statistics, such as the average number of symmetries per image. Therefore, these results primarily demonstrate the advantages of pretraining models with synthetic data to simulate real symmetries, rather than directly comparing performance to existing models.

Furthermore, the success of this hybrid training strategy suggests a scalable pathway for modeling other Gestalt principles beyond symmetry, such as continuity, closure, or similarity. Deep learning models often struggle to separate high-level semantic object recognition from low-level perceptual grouping; however, pretraining on abstract, synthetically generated geometric primitives forces the network to learn the underlying ‘grammar’ of visual organization first. By separating the learning of fundamental perceptual rules from specific real-world imagery during the initial phase, this methodology facilitates the development of generalized feature extractors that can be easily adapted to other structural detection tasks where obtaining large-scale human-annotated ground truth is challenging.

3.3. Number of annotators analysis

While not directly obvious from the quantitative results, we want to stress the importance of combining results from multiple annotators, via the proposed clustering pipeline. In this section we qualitatively compare results obtained when using our clustered inputs from multiple annotators as ground truth, vs. those obtained by just a single annotator. We generate a new dataset with 400 images labeled by random annotators from the experiments participant pool. All labeled symmetries were assigned a weight of 1, meaning uniform symmetry strength across the dataset. We then visually compared the prediction errors of a fine-tuned U-Net on the original benchmark dataset and the new single-annotator dataset. Fig. 7 illustrates these errors.

Fig. 7.

Fig. 7

Open in a new tab

Comparison of prediction error on images labeled by one person and the PIX2PER labeled with 50 people per image. Art images on the left, real images on the right. Green pixels represent true positives, red pixels false positives and yellow false negatives.

Critically, the deployment of a large pool of annotators mitigates the subjectivity inherent in individual human perception, ensuring a more robust ground truth than traditional single-annotator methods. As demonstrated by the prediction errors in the single-annotator dataset, relying on a single observer leads to incomplete labeling, where valid symmetrical elements are overlooked simply because they were not the most salient to that specific individual. This omission creates a flaw in evaluation, as models are penalized with false positives for detecting legitimate symmetries that the single annotator missed, such as the secondary axis in double symmetries. By aggregating the judgments of 50 participants, PIX2PER effectively filters out anecdotal bias and constructs a “consensus” perception, ensuring that models are evaluated against a comprehensive representation of symmetry that accounts for the diverse visual priorities of the general population.

4. Conclusion

In this paper, we collected human perception data to create a novel benchmark dataset, PIX2PER, tailored for reflection symmetry detection, focusing on capturing the diverse strengths of symmetry across various regions within images. Additionally, we introduce a new evaluation metric, WF1, to complement this dataset. We utilize it to assess the performance of state-of-the-art deep learning models in symmetry detection tasks. Furthermore, we propose a novel method for generating synthetic datasets to pretrain symmetry detection models, experiments with a base-line model show a substantial increase in performance.

CRediT authorship contribution statement

Gonzalo Muradás Odriozola: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Lisa Koßmann: Writing – review & editing, Visualization, Methodology, Investigation, Data curation, Conceptualization. Tinne Tuytelaars: Writing – review & editing, Validation, Supervision, Methodology, Investigation, Conceptualization. Johan Wagemans: Writing – review & editing, Supervision, Project administration, Methodology, Funding acquisition, Conceptualization.

Acknowledgments

Funded by the European Union (ERC AdG, GRAPPA , 101053925). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.

☆

This article is part of a Special issue entitled: ‘Models in Vision II’ published in Vision Research.

1

n: number of clusters in image, WM: Weight Multiplier, NIC: Number of Instances in Cluster, MSC: Minimum Samples per Cluster, NL: Number of Labelers, MM: Max Multiplier, WP: Weighted Precision, TP: True Positives, FP: False Positives, FN: False Negatives, WR: Weighted Recall, WF1: Weighted F1-Score.

References

  1. Allen G. The origin of the sense of symmetry. Mind. 1879;4(15):301–316. [Google Scholar]
  2. Atadjanov I.R., Lee S. Robustness of reflection symmetry detection methods on visual stresses in human perception perspective. IEEE Access. 2018;6:63712–63725. [Google Scholar]
  3. Bertamini M., Rampone G., Makin A.D., Jessop A. Symmetry preference in shapes, faces, flowers and landscapes. PeerJ. 2019;7 doi: 10.7717/peerj.7078. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Borji A., Itti L. 2015. CAT2000: A large scale fixation dataset for boosting saliency research. arXiv:1505.03581. [Google Scholar]
  5. Cho M., Lee K.M. BMVC. 2009. Bilateral symmetry detection via symmetry-growing; pp. 1–11. [Google Scholar]
  6. Cicconet M., Birodkar V., Lund M., Werman M., Geiger D. A convolutional approach to reflection symmetry. Pattern Recognition Letters. 2017;95 [Google Scholar]
  7. Csatho A., van der Vloed G., van der Helm P.A. The force of symmetry revisited: Symmetry-to-noise ratios regulate (a) symmetry effects. Acta Psychologica. 2004;117(3):233–250. doi: 10.1016/j.actpsy.2004.06.004. [DOI] [PubMed] [Google Scholar]
  8. Deng L. The mnist database of handwritten digit images for machine learning research [best of the web] IEEE Signal Processing Magazine. 2012;29(6):141–142. [Google Scholar]
  9. Ester M., Kriegel H.-P., Sander J., Xu X., et al. Kdd. vol. 96. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise; pp. 226–231. [Google Scholar]
  10. Fukushima K. Use of non-uniform spatial blur for image comparison: symmetry axis extraction. Neural Networks. 2005;18(1):23–32. doi: 10.1016/j.neunet.2004.08.001. [DOI] [PubMed] [Google Scholar]
  11. Fukushima K., Kikuchi M. Symmetry axis extraction by a neural network. Neurocomputing. 2006;69(16–18):1827–1836. [Google Scholar]
  12. Funk, C., Lee, S., Oswald, M. R., Tsogkas, S., Shen, W., Cohen, A., Dickinson, S., & Liu, Y. (2017). 2017 ICCV challenge: Detecting symmetry in the wild. In Proceedings of the IEEE international conference on computer vision workshops (pp. 1692–1701).
  13. Funk, C., & Liu, Y. (2017). Beyond planar symmetry: Modeling human perception of reflection and rotation symmetries in the wild. In Proceedings of the IEEE international conference on computer vision (pp. 793–803).
  14. Gnutti A., Guerrini F., Leonardi R. Combining appearance and gradient information for image symmetry detection. IEEE Transactions on Image Processing. 2021;30:5708–5723. doi: 10.1109/TIP.2021.3085202. [DOI] [PubMed] [Google Scholar]
  15. Goetschalckx L., Wagemans J. MemCat: a new category-based image set quantified on memorability. PeerJ. 2019;7 doi: 10.7717/peerj.8169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).
  17. van der Helm P.A. Weber-Fechner behavior in symmetry perception? Attention, Perception, & Psychophysics. 2010;72(7):1854–1864. doi: 10.3758/APP.72.7.1854. [DOI] [PubMed] [Google Scholar]
  18. Huiskes, M. J., & Lew, M. S. (2008). The mir flickr retrieval evaluation. In Proceedings of the 1st ACM international conference on multimedia information retrieval (pp. 39–43).
  19. Kang, C., Valenzise, G., & Dufaux, F. (2020). Eva: An explainable visual aesthetics dataset. In Joint workshop on aesthetic and technical quality assessment of multimedia and media analytics for societal trends (pp. 5–13).
  20. Kingma D.P., Ba J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [Google Scholar]
  21. Koehler K., Guo F., Zhang S., Eckstein M.P. What do saliency models predict? Journal of Vision. 2014;14(3):14. doi: 10.1167/14.3.14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kong S., Shen X., Lin Z., Mech R., Fowlkes C. In: Computer vision – ECCV 2016. Leibe B., Matas J., Sebe N., Welling M., editors. Springer International Publishing; Cham: 2016. Photo aesthetics ranking network with attributes and content adaptation; pp. 662–679. [Google Scholar]
  23. Lin T., Maire M., Belongie S.J., Bourdev L.D., Girshick R.B., Hays J., Perona P., Ramanan D., Dollár P., Zitnick C.L. 2014. Microsoft COCO: Common objects in context. CoRR abs/1405.0312 arXiv:1405.0312 URL http://arxiv.org/abs/1405.0312. [Google Scholar]
  24. Lin T.-Y., Maire M., Belongie S., Hays J., Perona P., Ramanan D., Dollár P., Zitnick C.L. Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13. Springer; 2014. Microsoft coco: Common objects in context; pp. 740–755. [Google Scholar]
  25. Liu Y., Hel-Or H., Kaplan C.S., Van Gool L., et al. Computational symmetry in computer vision and computer graphics. Foundations and Trends® in Computer Graphics and Vision. 2010;5(1–2):1–195. [Google Scholar]
  26. Liu, J., Slota, G., Zheng, G., Wu, Z., Park, M., Lee, S., Rauschert, I., & Liu, Y. (2013). Symmetry detection from realworld images competition 2013: Summary and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 200–205).
  27. Locher P.J., Wagemans J. Effects of element type and spatial grouping on symmetry detection. Perception. 1993;22(5):565–587. doi: 10.1068/p220565. [DOI] [PubMed] [Google Scholar]
  28. Loy G., Eklundh J.-O. Computer vision–ECCV 2006: 9th European conference on computer vision, graz, Austria, May 7-13, 2006. proceedings, part II 9. Springer; 2006. Detecting symmetry and symmetric constellations of features; pp. 508–521. [Google Scholar]
  29. Machilsen B., Pauwels M., Wagemans J. The role of vertical mirror symmetry in visual shape detection. Journal of Vision. 2009;9(12):11. doi: 10.1167/9.12.11. [DOI] [PubMed] [Google Scholar]
  30. Maerten, A.-S., Chen, L.-W., De Winter, S., Bossens, C., & Wagemans, J. (2025). LAPIS: A novel dataset for personalized image aesthetic assessment. In Proceedings of the computer vision and pattern recognition conference (pp. 6302–6311).
  31. Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. vol. 2, In Proceedings eighth IEEE international conference on computer vision. ICCV 2001 (pp. 416–423). vol.2.
  32. Muradas G., Koßmann L., Wagemans J. 2026. The relationships between figure-ground organization, objectness and symmetry. Manuscript submitted for publications. [Google Scholar]
  33. Nilsback M.-E., Zisserman A. Delving deeper into the whorl of flower segmentation. Image and Vision Computing. 2010;28(6):1049–1062. [Google Scholar]
  34. Nucci M., Wagemans J. Goodness of regularity in dot patterns: Global symmetry, local symmetry, and their interactions. Perception. 2007;36(9):1305–1319. doi: 10.1068/p5794. [DOI] [PubMed] [Google Scholar]
  35. Ogawa H. Symmetry analysis of line drawings using the Hough transform. Pattern Recognition Letters. 1991;12(1):9–12. [Google Scholar]
  36. Pombo M., Aleem H., Grzywacz N.M. Multiple axes of visual symmetry: detection and aesthetic preference. Symmetry. 2023;15(8):1568. [Google Scholar]
  37. Prolific . 2025. Many projects, one interface. https://www.prolific.com/. (Accessed 17 September 2025) [Google Scholar]
  38. Ren, J., Shen, X., Lin, Z., Mech, R., & Foran, D. J. (2017). Personalized image aesthetics. In Proceedings of the IEEE international conference on computer vision (pp. 638–647).
  39. Ronneberger O., Fischer P., Brox T. Medical image computing and computer-assisted intervention–mICCAI 2015: 18th international conference, munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer; 2015. U-net: Convolutional networks for biomedical image segmentation; pp. 234–241. [Google Scholar]
  40. Royer F.L. Detection of symmetry. Journal of Experimental Psychology: Human Perception and Performance. 1981;7(6):1186. doi: 10.1037//0096-1523.7.6.1186. [DOI] [PubMed] [Google Scholar]
  41. Saunders J.A., Knill D.C. Perception of 3D surface orientation from skew symmetry. Vision Research. 2001;41(24):3163–3183. doi: 10.1016/s0042-6989(01)00187-0. [DOI] [PubMed] [Google Scholar]
  42. Sawada T., Pizlo Z. Detection of skewed symmetry. Journal of Vision. 2008;8(5):14. doi: 10.1167/8.5.14. [DOI] [PubMed] [Google Scholar]
  43. Schubert E., Sander J., Ester M., Kriegel H.P., Xu X. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Transactions on Database Systems. 2017;42(3):1–21. [Google Scholar]
  44. Seo, A., Kim, B., Kwak, S., & Cho, M. (2022). Reflection and rotation symmetry detection via equivariant learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9539–9548).
  45. Seo, A., Shim, W., & Cho, M. (2021). Learning to discover reflection symmetry via polar matching convolution. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 1285–1294).
  46. Simonyan K., Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. [Google Scholar]
  47. Teo, C. L., Fermuller, C., & Aloimonos, Y. (2015). Detection and segmentation of 2d curved reflection symmetric structures. In Proceedings of the IEEE international conference on computer vision (pp. 1644–1652).
  48. Tsogkas S., Kokkinos I. Computer vision–ECCV 2012: 12th European conference on computer vision, florence, Italy, October 7-13, 2012, proceedings, part VII 12. Springer; 2012. Learning-based symmetry detection in natural images; pp. 41–54. [Google Scholar]
  49. Tyler C.W. Human symmetry detection exhibits reverse eccentricity scaling. Visual Neuroscience. 1999;16(5):919–922. doi: 10.1017/s0952523899165118. [DOI] [PubMed] [Google Scholar]
  50. Tyler C.W. The symmetry magnification function varies with detection task. Journal of Vision. 2001;1(2):7. doi: 10.1167/1.2.7. [DOI] [PubMed] [Google Scholar]
  51. Wagemans J. Skewed symmetry: a nonaccidental property used to perceive visual forms. Journal of Experimental Psychology: Human Perception and Performance. 1993;19(2):364. doi: 10.1037//0096-1523.19.2.364. [DOI] [PubMed] [Google Scholar]
  52. Wagemans J. Detection of visual symmetries. Spatial Vision. 1995;9:9–32. doi: 10.1163/156856895x00098. [DOI] [PubMed] [Google Scholar]
  53. Wagemans J., Van Gool L., D’ydewalle G. Detection of symmetry in tachistoscopically presented dot patterns: Effects of multiple axes and skewing. Perception & Psychophysics. 1991;50(5):413–427. doi: 10.3758/bf03205058. [DOI] [PubMed] [Google Scholar]

RESOURCES