Abstract
Eye tracking data are highly promising in revealing novel and valuable evidence on human behavior and decision making. Data descripted in this article were collected in fourteen experiments with SMI eye tracking glasses in individual and social decision making conditions. The dataset is available on Harvard Dataverse and include data of 14 subjects with 4,180 visual behavior metrics summary and 3,744 eye moment records in decision-related areas of attention. Data may be applicable in computational models of oculomotor activity to explain decision process and predict its outcomes.
Subject terms: Human behaviour, Social behaviour
Background & Summary
Vision is a synergistic effect of eye sensing and brain processing mechanisms. Extensive fundamental research considered a tight coupling between visual perception and cognition1,2. Studies linked visual perception with higher cognitive functions such as visual spatial attention and memory3,4. Decades of eye tracking research revealed that eye movements and decision making are tightly related5,6. Eye movement metrics has been useful in providing evidence of mechanisms that drive reward-driven attention, information search and choice7,8. Investigations on eye movements enable a fine-grained analysis of the behavior reflecting an escalatory decision value9,10, a decision in preference formation11–13, and multi-alternative decision making14.
Some farther evidence suggested a great potential of neuroscience-based metrics in investigating decision making in social environments15,16. Together with a functional perspective, studies employed electroencephalography17, fMRI18, neurochemical19, and, more recently, eye tracking measures to understand a social modulating role on decisions and social contexts of decision making16. Eye movement data considered to be highly promising in revealing novel and valuable evidence on social influence on attention, behavior, and choices20–22.
Considering these two insightful research directions in Visual and Decision Neuroscience, I present data of visual behavior and its dynamics in experimental investigation for decision making in individual and social conditions.
Methods
Data collection was performed in fourteen experiments with the SMI eye tracking glasses (by SensoMotoric Instruments GmbH), a mobile eye tracking device with a 120 Hz binocular sampling rate capturing eye movements with 0.5° gaze tracking accuracy and 80° horizontal, 60° vertical gaze tracking range.
The sample is available on Harvard Dataverse and include data of 14 subjects (7 women) aged between 20 and 40 years (M 23.7, SD 6.2) with 4,180 visual behavior metrics and 3,744 eye moment records in decision-related areas of attention. All experiments were conducted in both individual and group conditions, and applied data filters covered individual versus group decision making conditions as well as pro-self subject behavior versus prosocial behavior. The study procedures involving human participants were approved by the Ethics Council of Tomsk State University (Approval 101–2019 on 2 September, 2019) and adhered to the tenets of the Declaration of Helsinki. The methods in the study were in accordance with relevant guidelines, and a written informed consent was obtained from all participants.
As an experimental framework, Prisoner’s Dilemma (hereinafter PD), an essential example of a social dilemma widely used in behavioral and decision making studies21,23–26, was employed to investigate pro-self and prosocial decision making. In PD, individuals can either cooperate or defect by making prosocial or pro-self decisions. Both would benefit from mutual cooperation but defecting or making egoistic pro-self choices would be more rewarding in terms of payoff as a PD game outcome.
In experimental procedure, twelve participants were invited to a computer classroom where they took their seats at the computer desks and were instructed and then completed the participant consent form.
Stage 1. One participant of twelve was equipped with the mobile eye tracking glasses device (hereinafter ETG). The ETG were calibrated with a standard 1-point calibration procedure to ensure the high quality of the participant’s eye movements tracking. Then, all the experiment participants proceeded to a computer-based PD game (20 trials). The participants were able to move to the next trials only after all of them had made their choices. No one knew who the opponents were, and in each round the pairs changed randomly. The current trial result and the overall game result were displayed on computer screens after each trial. Eye recording started from the beginning of the first trial of the PD game. After the game was completed, the participant with ETG took off the ETG device.
Stage 2. Participants were involved in a social interaction through the communication and group formation, when two groups of six individuals each were created by participants based on their common interests, hobbies, and etc. This laboratory model combines the classic social psychology minimal group paradigm with group manipulations that cause a sense of social attachment27,28.
Stage 3. Participants took their seats at the computers. They were instructed that they would be asked to play PD again, but this time their partner would be a random member from the newly formed group of six people. The participant who was equipped with ETG at the Stage 1 put on the ETG and the standard calibration procedure were held once again. Then all the participants proceeded to PD, which consisted of 22 trials. The result of each trial and the total personal result for the game were displayed on the participants’ computer screens after each trial of the game.
The research design description presented graphically with Fig. 1.
Fig. 1.

Research Design employed for generation the data reported in this article.
All the trials during Stage 1 and Stage 2 employed the same PD game with unchanged payoff matrix (presented with Fig. 2).
Fig. 2.

PD’s Areas of Interest employed for AOIs-based aggregating the eye movement indicators.
Data Records
The Semantic Gaze Mapping tool was used for aggregating each participant’s eye movement data. After Semantic Gaze Mapping-based data aggregation, eye movement quantitative metrics were exported with BeGaze software with the following variables:
Revisits – the amount of eye gaze returns to the already scanned area;
Fixation Count – the number of eye gaze fixations in the specified area per second;
Average Fixation Duration – the mean value of the fixation time in the specified area;
Fixation Time – the time of eye gaze fixation in the specified area, as a percentage of the time that the respondent looked at the specified area;
Saccade Count – the number of eye saccades in the specified area per second;
Saccade Frequency (count/s) – the number of saccades per second;
Saccade Latency Average (ms) – the average reaction time of a visually guided saccade;
Dwell Time – the total time that the subject looked at the specified area, as a percentage of the time of the demonstration of the whole area;
Scanpath Length (px) is the total length of all saccades in gaze trajectory.
In addition, cells of a payoff matrix in the PD game were employed in analysis as so-called Regions of Interest (ROIs)29 or Areas of Interest (AOIs)21 (Fig. 2). Markup was done before the experiments were conducted. Information on AOI Name, AOI Order, AOI Size (px), and AOI Coverage (%) is available in the dataset. Alike the above, aggregate measures such as fixation counts, saccades, dwell times, and others were registered on each PD’s Area of Interest and then exported.
Generated and deposited data are divided into two datasheets – eye movement records summary and AOIs-based eye movement records. Both are available in a deposited data file on Harvard Dataverse30 and are ready for data processing.
Technical Validation
All data were collected in one laboratory using the unified protocols and tasks. The PD game interface as well as the PD game payoff matrix structure was the same during all experiments and experimental stages. All the participants recruited in the experiments had normal or corrected-to-normal vision. A standard 1-point eye calibration procedure and SMI software of eye movement data pre-processing were in accordance with relevant technical guidelines and ensured reliability of all the variables and technical quality of the dataset. Together, these homogeneities minimize the variation of the experimental environment, tasks, procedures, and participants.
Usage Notes
As a description of data potential for reuse, I highlight the important data application directions. First analytical glance at eye movement data revealed that, in general, eye behaviour changed throughout the experimental stages. By the Stage 2, when participants made their decisions in PD game under social condition, their eye fixation and saccade frequencies were increased. Time delays between saccades were reduced, and the total length of gaze path became consistently shorter. Such a dynamic may be interpreted by perceptual learning effect as participants had already experienced all the visual environment of the PD game by the experimental Stage 2 and basic stimuli elements were unaltered across the PD task. However, further interpretation demands a bit deeper and comprehensive look into the data, and social versus individual decision-making conditions as well as their impact on decisions that were made should be considered.
In addition, eye movement data on visual environment of the PD game, which was set as regions of interest (particularly, PD matrix cells), deliver a unique clue on eye movement patterns during pre-decision visual information processing. For example, reported data showed that longer fixations were more common for participants with pro-self decisions than for those who made prosocial ones. While eye fixation is universally considered to be linked to a task processing model, which assumes that the eye fixates the referent of the symbol being operated on31, the obtained evidence can be explained by the consideration that pro-self strategy probably demands more cognitive effort. However, this consideration should be confirmed with other eye metrics of participants who showed ‘egoism’ in decision making task.
Along with that, further directions of data reuse may include sophisticated modelling of pro-self and prosocial decision making based on eye movement data. These computational models may employ neural networks and deep learning to enrich current knowledge of eye-brain decision making mechanisms.
Acknowledgements
The author acknowledges the project No. FSWM-2020-0040 within the assignment of the Ministry of Education and Science of the Russian Federation.
Author contributions
Anastasia Peshkovskaya solely contributed to the article preparation and submission.
Code availability
No custom code was used to generate or process the data described in the manuscript.
Competing interests
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Footnotes
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Beatty, J. & Kahneman, D. Pupillary changes in two memory tasks. Psychon Sci5, 371–372, 10.3758/BF03328444 (1966). [Google Scholar]
- 2.Goldstone, R. L. & Barsalou, L. W. Reuniting perception and conception. Cognition65(2), 231–262, 10.1016/S0010-0277(97)00047-4 (1998). [DOI] [PubMed] [Google Scholar]
- 3.Basso, M. A., Bickford, M. E. & Cang, J. Unraveling circuits of visual perception and cognition through the superior colliculus. Neuron109(6), 918–937, 10.1016/j.neuron.2021.01.013 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Simion, C. & Shimojo, S. Early interactions between orienting, visual sampling and decision making in facial preference. Vision research46(20), 3331–3335, 10.1016/j.visres.2006.04.019 (2006). [DOI] [PubMed] [Google Scholar]
- 5.König, P. et al Eye movements as a window to cognitive processes. Journal of Eye Movement Research, 9(5) 10.16910/jemr.9.5.3 (2016).
- 6.Spering, M. Eye Movements as a Window into Decision-Making. Annual Review of Vision Science8(1), 427–448, 10.1146/annurev-vision-100720-125029 (2022). [DOI] [PubMed] [Google Scholar]
- 7.Bird, G. D., Lauwereyns, J. & Crawford, M. T. The role of eye movements in decision making and the prospect of exposure effects. Vision Research60, 16–21, 10.1016/j.visres.2012.02.014 (2012). [DOI] [PubMed] [Google Scholar]
- 8.Wedel, M., Pieters, R. & van der Lans, R. Modeling Eye Movements During Decision Making: A Review. Psychometrika88, 697–729, 10.1007/s11336-022-09876-4 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Shimojo, S., Simion, C., Shimojo, E. & Scheier, C. Gaze bias both reflects and influences preference. Nature Neuroscience6(12), 1317–1322, 10.1038/nn1150 (2003). [DOI] [PubMed] [Google Scholar]
- 10.Pearson, D., Watson, P., Albertella, L. & Le Pelley, M. E. Attentional economics links value-modulated attentional capture and decision-making. Nat Rev Psychol1, 320–333, 10.1038/s44159-022-00053-z (2022). [Google Scholar]
- 11.Carpenter, R. & Williams, M. Neural computation of log likelihood in control of saccadic eye movements. Nature377, 59–62, 10.1038/377059a0 (1995). [DOI] [PubMed] [Google Scholar]
- 12.Krajbich, I., Armel, C. & Rangel, A. Visual fixations and the computation and comparison of value in simple choice. Nat Neurosci13, 1292–1298, 10.1038/nn.2635 (2010). [DOI] [PubMed] [Google Scholar]
- 13.Rojas, J. C., Marín-Morales, J., Ausín Azofra, J. M. & Contero, M. Recognizing Decision-Making Using Eye Movement: A Case Study with Children. Frontiers in psychology11, 570470, 10.3389/fpsyg.2020.570470 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gluth, S., Kern, N., Kortmann, M. & Vitali, C. L. Value-based attention but not divisive normalization influences decisions with multiple alternatives. Nature human behaviour4(6), 634–645, 10.1038/s41562-020-0822-0 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.van den Bos, R., Jolles, J. W. & Homberg, J. R. Social modulation of decision-making: a cross-species review. Front. Hum. Neurosci.7, 301, 10.3389/fnhum.2013.00301 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Peshkovskaya, A. & Myagkov, M. Eye Gaze Patterns of Decision Process in Prosocial Behavior. Frontiers in Behavioral Neuroscience14, 525087, 10.3389/fnbeh.2020.525087 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Billeke, P., Zamorano, F., Cosmelli, D. & Aboitiz, F. Oscillatory brain activity correlates with risk perception and predicts social decisions. Cereb. Cortex23(12), 2872–2883, 10.1093/cercor/bhs269 (2013). [DOI] [PubMed] [Google Scholar]
- 18.Kuss, K. et al. Neuronal correlates of social decision making are influenced by social value orientation-an fMRI study. Frontiers in behavioral neuroscience9, 40, 10.3389/fnbeh.2015.00040 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Edmiston, E. K., Merkle, K. & Corbett, B. A. Neural and cortisol responses during play with human and computer partners in children with autism. Soc. Cogn. Affect. Neurosci.10, 1074–1083, 10.1093/scan/nsu159 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jiang, T., Potters, J. A. & Funaki, Y. Eye‐tracking Social Preferences. Journal of Behavioral Decision Making29, 157–168, 10.1002/bdm.1899 (2016). [Google Scholar]
- 21.Peshkovskaya, A. G. et al. The socialization effect on decision making in the Prisoner’s Dilemma game: An eye-tracking study. PLoS ONE12(4), e0175492, 10.1371/journal.pone.0175492 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Chen, W., Ruan, R., Deng, W. & Gao, J. The effect of visual attention process and thinking styles on environmental aesthetic preference: An eye-tracking study. Frontiers in psychology13, 1027742, 10.3389/fpsyg.2022.1027742 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Axelrod, R. & Hamilton, W. D. The evolution of cooperation. Science211, 1390–1396, 10.1126/science.7466396 (1981). [DOI] [PubMed] [Google Scholar]
- 24.Andreoni, J. & Miller, J. H. Rational cooperation in the finitely repeated prisoner’s dilemma: Experimental evidence. The economic journal103, 570–585, 10.2307/2234532 (1993). [Google Scholar]
- 25.Peshkovskaya, A., Myagkov, M., Babkina, T. & Lukinova, E. Do women socialize better? Evidence from a study on sociality effects on gender differences in cooperative behavior. CEUR Workshop Proceeding1968, 41–51 (2017). [Google Scholar]
- 26.Kozitsina, T. S., Kozitsin, I. V. & Menshikov, I. S. Quantal response equilibrium for the Prisoner’s Dilemma game in Markov strategies. Sci Rep12, 4482, 10.1038/s41598-022-08426-3 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Peshkovskaya, A., Babkina, T., & Myagkov, M. In-Group Cooperation and Gender: Evidence from an Interdisciplinary Study. In: Kaz, M., Ilina, T., Medvedev, G. (eds) Global Economics and Management: Transition to Economy 4.0. Springer Proceedings in Business and Economics 10.1007/978-3-030-26284-6_17 (Cham: Springer, 2019).
- 28.Babkina, T. et al. Choice of the group increases intra-cooperation. CEUR Workshop Proceeding1627, 13–23 (2016). [Google Scholar]
- 29.Holmqvist, K. et al Eye Tracking: A Comprehensive Guide to Methods and Measures. http://ukcatalogue.oup.com/product/9780199697083.do (Oxford Univ. Press, 2011).
- 30.Peshkovskaya, A. Data for: Eye Gaze Patterns of Decision Process in Prosocial Behavior. Harvard Dataverse, V110.7910/DVN/TF2C4Q (2020). [DOI] [PMC free article] [PubMed]
- 31.Just, M. A. & Carpenter, P. A. Eye fixations and cognitive processes. Cognitive Psychology8, 441–480, 10.1016/0010-0285(76)90015-3 (1976). [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
No custom code was used to generate or process the data described in the manuscript.
