Temporal Structure of Human Gaze Dynamics Is Invariant During Free Viewing

Colleen A Marlow; Indre V Viskontas; Alisa Matlin; Cooper Boydston; Adam Boxer; Richard P Taylor

doi:10.1371/journal.pone.0139379

. 2015 Sep 30;10(9):e0139379. doi: 10.1371/journal.pone.0139379

Temporal Structure of Human Gaze Dynamics Is Invariant During Free Viewing

Colleen A Marlow ^1,^2,^*, Indre V Viskontas ³, Alisa Matlin ³, Cooper Boydston ⁴, Adam Boxer ³, Richard P Taylor ⁴

Editor: Jesus Malo⁵

PMCID: PMC4589360 PMID: 26421613

Abstract

We investigate the dynamic structure of human gaze and present an experimental study of the frequency components of the change in gaze position over time during free viewing of computer-generated fractal images. We show that changes in gaze position are scale-invariant in time with statistical properties that are characteristic of a random walk process. We quantify and track changes in the temporal structure using a well-defined scaling parameter called the Hurst exponent, H. We find H is robust regardless of the spatial complexity generated by the fractal images. In addition, we find the Hurst exponent is invariant across all participants, including those with distinct changes to higher order visual processes due to neural degeneration. The value we find for H of 0.57 shows that the gaze dynamics during free viewing of fractal images are consistent with a random walk process with persistent movements. Our research suggests the human visual system may have a common strategy that drives the dynamics of human gaze during exploration.

Introduction

The fovea is the region of the retina with the highest visual acuity. To allow the human neural system to sample its visual environment, the gaze, or line of sight from the fovea to the position of interest in the visual field, is adjusted over time. The movements of the gaze within the visual field form a trajectory, known as the scan path, which is composed of a series of rapid changes in gaze position. Eye tracking technology can measure the position of the gaze in the visual field, allowing participants’ scan paths to be tracked while they engage in visual tasks in real-time. Using this technology, the human eye’s scan path is observed to have a complex structure both spatially and temporally. The underlying mechanisms in the neural system which modulate how the eye navigates the visual field to generate this complex structure is a topic of much discussion [1]. The structure of human gaze changes is thought to arise from the interplay of bottom-up factors such as the saliency of image features and top-down higher order cognitive factors such as those affecting attention. Historically, models tend to focus on the role of visual attention to predict where localizations of the gaze, fixations, occur within a visual scene [2]. However, in order to understand the complex dynamics of the human visual system as it navigates the sensory field it is necessary also to take into account how the gaze shifts occur in time. Recent models take into account the importance of the temporal dynamics of gaze by modelling gaze shifts using random walk processes [3–5].

A random walk involves a series of steps for which the root mean square displacement,〈d ²〉, of the walker scales with the elapsed time, t, following the relationship 〈d ²〉~t ^2H. The specific value of the Hurst exponent H determines how the random walk evolves in time and is related to the probability distribution of a random walk process. It determines how the trajectory of the walk fills space over time and the diffusive properties of the process. For example, a value of H = 1/2 is the signature of a Gaussian distribution yielding Brownian motion with normal diffusion. In contrast, values of H > 1/2 are linked to long-tailed distributions (i.e. Lévy Flights [6]) with superdiffusive behavior and values of H < 1/2 are associated with subdiffusive behavior.

Models of gaze shifts based on random-walk processes predict scale invariance in gaze dynamics. Recent experiments confirmed scale invariance in the temporal structure of the larger shifts in gaze position [6–9], referred to as saccades. Additionally, scale invariance has also been observed in so-called fixational eye movements [10, 11] which are the microscopic shifts in gaze that occur while the eye is fixating on a localized region in the visual field. Recent models incorporating the prevalent feature of temporal scale invariance show promise in uncovering the underlying dynamics employed by the visual system during fixation [12] and have provided a better understanding of the function of fixational movements in seeing [13].

The success of the temporal analysis of gaze shifts during fixation motivates our study on the temporal qualities of changes in gaze which occur during free viewing. We apply a time-series analysis method that reliably identifies and quantifies any changes in the temporal scale invariance of gaze shifts. This allows us to describe, characterize and compare the temporal properties of the movements across individual participants engaged in visual exploration during free viewing of computer-generated fractal images.

The prevalence of fractals in our natural environment has motivated a number of investigations of human behavioral and neurophysiological responses to their visual characteristics. Fractals feature patterns that repeat at increasing fine size scales, building immense visual complexity. Examples of these scale invariant patterns include tree branches, coastlines, clouds, and mountain profiles. In our study, we use computer-generated fractal images to establish the basic temporal gaze structure during free viewing and then present preliminary data indicating that this behavior extends to the more subtle stimuli of natural scenes.

This particular viewing scenario was chosen such that higher order processes are not strongly engaged during visual exploration. Low level pre-attentive image features dominate gaze control during free viewing of static images [14, 15]. Within this context, we vary the spatial complexity of the salient feature of our fractal images- which are the fractal edges of the black and white regions.

We identify scale invariance of the gaze shifts and determine the Hurst exponent associated with the time series using the variation method, as described in the Analysis section. We then track the Hurst exponents to determine whether the overall strategy of the scan path depends on the spatial complexity of the fractal image or solely on the viewing mode the individual is engaged in. Participants with neural degenerative disease were included to assess whether physical changes in the visual system due to the disease affect the overall gaze dynamics. By identifying when and if the temporal dynamics of the scan path change with neural degeneration will help determine if the gaze shift strategy is controlled for at a higher level in the visual system or at the oculomotor level. We will contrast our results, in which participants were not intentionally fixating, with studies in which participants were actively fixating. Then we discuss this contrast in the context of the function of the visual system.

Materials and Methods

Participants

Gaze position data was collected for 20 participants, 11 of whom were healthy participants (H) and 9 were participants with neural degenerative (ND) diseases. ND participants comprised 3 participants with Alzheimer’s disease (AD), 5 participants with degeneration in the frontal and anterior temporal lobes (FTD) and 1 participant with progressive supranuclear palsy (PSP). Of the H group, 8 were young normal (YN) participants (ages ranging from 18 to 35) and 3 were age-matched normal control (NC) participants for comparison to ND patients. Participants were recruited and evaluated through the Memory and Aging Center at University of California, San Francisco (UCSF). Neurodegenerative participants were diagnosed by a team of neurologists, neurophysiologists and nurses after extensive examination. Participants with red/green colorblindness or who had undergone eye surgery were excluded from the study.

All participants involved in this research gave informed written consent to participate in the experimental procedures. In cases where participants had a compromised capacity to provide consent, a legally authorized representative gave informed written consent on behalf of the participant. This study was approved by the University of California, San Francisco Institutional Review Board.

Experiment Procedure

Each participant viewed from 9 to 93 images including at least one complete set of fractal images of varying complexity (Fig 1). The images, along with versions of the image where the black and white regions were inverted, were presented in random order. Each image was viewed for 5 seconds and a fixation screen was viewed for 1 second between each image (to re-center the gaze). The viewing protocol is shown in the lower left section of Fig 1. To increase the likelihood the participant remained engaged throughout the entire viewing time, participants were asked to answer (recorded with a press of key on a keyboard) whether or not they found the image aesthetically appealing as compared to previous images. The x and y positions of the participant’s left pupil’s gaze were recorded at 120 Hz using an ASL infrared eye tracker (504HS).

It is important to note that gaze position data collected in this study is therefore limited to changes on times scales of 8.3 ms and above. As a result, we do not probe the entire temporal range in which fixational movements extend, typically down to 2ms [10–12]. However, we are still able to observe shifts over a significant range of measured timescales, from 8.3ms to 5s.

Images were displayed on a LCD monitor (30.48 cm by 40.64 cm) which participants viewed at a distance of 67.3 cm. A chin rest and head strap were used to keep the participant’s head steady and the screen to pupil distance fixed throughout viewing. All images were 768 x 768 pixels subtending the visual angles of 24° vertically and 24° horizontally (one image pixel subtended an angel of 0.034° in the visual field). The upper value of the noise in the measured gaze position was 0.03°, just under the pixel resolution.

Using our computer-generated fractal images we can control the spatial complexity easily by altering the fractal dimension, D, of the image. We speculate that images with a large range of spatial complexity will induce changes in the gaze dynamics responsible for exploration. In particular, we hypothesize that as image complexity is increased the Hurst exponent will increase to accommodate. Images with higher D have higher ratios of fine to coarse spatial structure which in turn generates a higher visual spatial complexity. Fig 1 shows examples of typical fractal images used (top row) along with Euclidean (non-fractal) example images. We quantify the complexity, C, of the fractal images as C = D– 1. A non-fractal image has a value of D = 1 and therefore a low complexity of C = 0. A fractal image with D = 1.9 has a high complexity of C = 0.9. Images ranged from C = 0 to 0.9 in increments of 0.1. The non-fractal images were either simple geometric shapes, abstract images or patterns of geometric shapes (see Fig 1 top left for examples).

Analysis

All raw gaze position data files were processed after collection to eliminate files with either multiple or long events where the gaze was lost by the eye tracker and cases for which the raw gaze position exceeded the image boundaries (i.e. the gaze position was off-screen) for over 110 ms (the average length of a blink). The magnitude of the change in gaze position with time, l, is calculated from the vertical and horizontal coordinates of the gaze position of two successive points in time (x _i+1, y _i+1 and x _i, y _i) and is defined as $l = \sqrt{{(x_{i + 1} - x_{i})}^{2} + {(y_{i + 1} - y_{i})}^{2}}$ .

Variation Method

We use the variation method to identify whether the gaze data exhibits the scale invariance characteristic of random walk processes. The variation method is a well-established spectral analysis method appropriate for time-series data [16]. It not only reliably detects scale invariance but will allow us to quantify the Hurst exponent associated with the random walk process and detect any changes to this value. In the variation method, the data trace is covered with columns of spacing Δt and then the minimum number of non-overlapping boxes, N, needed to cover the data structure is calculated. This process is repeated for increasingly fine time scale until the data resolution limit is reached. If the time series is scale invariant then N(Δt) follows a power law of the form

N (Δ t) ~ {Δ t}^{- (2 - H)}

over a significant range of timescales, at least one order of magnitude. Scale invariance and the associated Hurst exponent H value can be determined from a plot of-log N(Δt) vs. log (Δt). A linear plot demonstrates scale invariance and H is determined from the slope of the line. Shallower slopes (H closer to 1) indicate a bias toward low frequency movements and steeper slopes (H closer to 0) indicate a bias toward high frequency movements.

The variation method differs from basic box counting methods [17] because the grid height is normalized to account for the different scaling factors implicit in each of the coordinates defining the structure (i.e. the variable l does not have the same units as t). This important difference is what makes it possible for the variation method to assess the scale-invariant behavior of the times series. The variation method is also optimized to avoid over-counting of the structure while maximizing counting statistics leading to a more accurate value of the scaling exponent. Our approach is similar in concept to determining the power spectrum where one looks at the frequency content of time series. Using power spectrum analysis, scale invariance occurs when a data trace’s power is an inverse function of frequency, S(f)~1/f ^β. For a finite set of data points statistics are limited. In this case, the power spectrum analysis leads to greater scatter in the output function when compared to the variation method, rendering the variation method the more accurate of the two methods [16]. A computational algorithm was developed and performed in MATLAB to apply the variation method analysis to the change in gaze position data with time, l(t), collected in this study.

Results

Data from the healthy participant Y1 is used as a typical example of observations. Fig 2A and 2B show the participant’s vertical and horizontal gaze position along with the accompanying scan path. The magnitude of the change in gaze position with time, l(t), for this data is shown in Fig 3A. The variation method output N(Δt) is plotted as a function of Δt, and follows an inverse power law. This can be clearly observed in the linearity of the plot of–log N(Δt) with log Δt (black circles in Fig 3B). The data is plotted within the observational time limits of the experiment and the linear fit to the data is shown as a solid line. The scaling exponent for this particular gaze data is found to be 1.44 by a linear regressive fit of the scaling plot, giving a Hurst exponent value of H = 0.56. The plot is linear over the range marked by the solid arrows indicating l (t) is scale invariant across 1.5 orders of magnitude, consistent with observations of scale invariance in other physical systems in nature [18]. The observational time cut-offs are set by the length of time participants viewed the image (5 seconds) and the time resolution (8.3 ms) of the data collection. The deviations we see in the data from the linear, scale invariant behavior occur at the positions along the x-axis, marked by the arrows, where we would expect them to be based on the statistical criteria of our analysis. In order to reliably detect scale invariant structure we need at least a grid size of 1/5 of the maximum trace length (5 seconds). The coarse-scale observational cut-off in our study is thus 1 s, as marked by the arrow on Fig 3B. We also cannot detect structure below times which are 3 times the resolution (8.3 ms) of the data collected. This fine-scale cut-off is also marked on Fig 3B.

Fig 3 — Actual data is shown as black circles and surrogate data in red squares. Note time axis in insets have been cropped in order to see details of plot. The associated plot of–log N(Δt) vs. log Δt for (B) the actual and surrogate data for participant Y1, and the data for FTD5, (D) and (F). The linear behavior observed in the actual data demonstrates the scale invariance of *l (t)* within the measured range. The Hurst exponent H is listed on the plots and was obtained from the slope of the linear fit (slope = 2-H). Insets shows residuals of the linear fits.

To ensure that the observed scale invariance is not an artifact of the measurement system we created surrogate data by randomly shuffling the actual gaze position data in time. The scaling plot associated with the surrogate data is plotted as red squares in Fig 3B (bottom plot). The failure of the surrogate data to follow a power law across the observable range is best illustrated by the plot of the residuals in the inset of Fig 3B. The plot shows a U-shape consistent with a nonlinear log-log plot. This demonstrates that the scale invariance is removed through random shuffling and is therefore not an artifact of the measurement system but rather a signature of human gaze dynamics. This temporal scale invariance across participants is consistent with previous results [7–10, 17]. Equivalent plots were generated for all gaze position data collected and linearity was seen for all images and for all participants. Fig 2C and 2D and Fig 3C and 3D show the equivalent plots for participant FTD5 while also looking at a computer-generated fractal.

To assess the effect that image complexity has on the gaze dynamics, we tracked the value of H for each of the participants across all the images they viewed. The resulting H values are shown for participant Y1 in Fig 4A. Box plots of H are grouped by image complexity. The accuracy of our analysis method was determined by generating several different artificial times series of known H and performing the variation method algorithm on them. The precision for determining H was found to be ± 0.03. To determine any significant differences between the image groups, a two-sample doubled sided t-test (α-level of 0.01) was performed with the Bonferroni correction for multiple comparisons. The Bonferroni correction adjusts the significance threshold to α/m where m is the number of groups being compared. In this case, the correction sets the significance threshold to 0.001. In all cases, there were no significant differences between H values across image complexity, with all instances giving p-values of at least 0.1 and the majority giving p-values larger than 0.5. The specific values for this participant (Y1) are listed in Table 1 as an example. Counter to our expectations, all participants showed no significant difference in the observed H value across image complexity indicating we see no change in the diffusive properties of gaze with image. These results suggest that the overall spatial complexity of an image does not affect the gaze dynamics during free viewing. Furthermore, scale invariance is seen in all cases of the Euclidean (non-fractal) images, even the simpler geometric images which have very low saliency. This suggests that scale invariant gaze dynamics prevail as long as participants are not actively fixating.

Table 1. p-values from double comparison two-sided t-test for image complexity groups for participant Y1.

The value C represents the level of spatial complexity in the image, where C = 0 is the lowest and C = 0.9 is the highest. The significance threshold was determined by the Bonferroni comparison as 0.005.

	n = 19	n = 5	n = 7	n = 6	n = 7	n = 8	n = 8	n = 6	n = 7	n = 7
C	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
0	-	0.21	0.73	0.79	0.32	0.13	0.11	0.83	0.22	0.20
0.1	0.21	-	0.54	0.26	0.85	0.86	0.86	0.47	0.97	0.88
0.2	0.73	0.54	-	0.69	0.68	0.45	0.41	0.94	0.57	0.53
0.3	0.79	0.26	0.69	-	0.41	0.24	0.18	0.76	0.31	0.35
0.4	0.32	0.85	0.68	0.41	-	0.73	0.73	0.63	0.88	0.77
0.5	0.13	0.86	0.45	0.24	0.73	-	0.96	0.41	0.84	0.98
0.6	0.11	0.86	0.41	0.18	0.73	0.96	-	0.36	0.85	0.94
0.7	0.83	0.47	0.94	0.76	0.63	0.41	0.36	-	0.52	0.51
0.8	0.22	0.97	0.57	0.31	0.88	0.84	0.85	0.52	-	0.85
0.9	0.20	0.88	0.53	0.35	0.77	0.98	0.94	0.51	0.85	-

Open in a new tab

All of the participants’ H values were tested for normality using a two sample double-sided t-test (α-level of 0.05) for comparison with a normal distribution of the same mean and standard deviation (s.d.) of the data. All p-values resulting from normality testing are listed in Table 2 along with the mean value of H, H _ave, and s.d. for each participant (p-values > 0.05 indicate no significant difference between the distributions and normality in the data). In the cases where the sample size was too small to reliably test for normality (n < 10), we also report the range of H values for the participant. Next specific Hurst exponents were compared across individuals within a given participant group. Fig 4B shows the box plots of the H values observed across all healthy participants (H).

Table 2. Hurst exponent, H, statistics listed for each participant.

H _ave is the mean value per participant, s.d is the standard deviation and the p-values are those found from normality tests (p > 0.05 indicates normally distributed data). For small sample sizes (n <10), the range of H values for that participant is also reported.

Participant	n	H _ave	s.d.	H range	p
Y1	83	0.62	0.03		0.87
Y2	21	0.49	0.06		0.79
Y3	72	0.56	0.03		0.95
Y4	8	0.53	0.06	0.48–0.56	0.68
Y5	19	0.58	0.03		0.56
Y6	9	0.58	0.02	0.56–0.61	0.90
Y7	10	0.50	0.04		0.63
Y8	6	0.50	0.03	0.46–0.56	0.40
C1	10	0.51	0.05		0.86
C2	18	0.54	0.03		0.56
C3	3	0.47	0.02	0.45–0.48	0.47
FTD1	2	0.62	0.03	0.60–0.64	0.92
FTD2	17	0.55	0.04		0.43
FTD3	6	0.58	0.02	0.56–0.60	0.64
FTD4	4	0.56	0.03	0.53–0.59	0.86
FTD5	14	0.52	0.04		0.84
AD1	16	0.54	0.02		0.55
AD2	5	0.56	0.03	0.51–0.59	0.47
AD3	3	0.48	0.04	0.45–0.52	0.22
PSP1	14	0.55	0.06		0.82

Open in a new tab

The observed Hurst exponents for individuals within a given participant group were compared using a two-sample doubled sided t-test with the Bonferroni correction for multiple comparisons. No significant differences were seen in the H values between individuals within the H group and individuals with degeneration in the frontal and anterior temporal lobes (FTD) and Alzheimer Disease (AD) also show no significant differences within each group. Lastly, H values were compared across all participant groups, including the participant with progressive supranuclear palsy (PSP), and reveal no significant difference. This is shown in the boxplot of Fig 4C. All p-values fall above the significance threshold determined from the Bonferroni correction for multiple comparisons (a significance threshold of 0.0125 in this case), see Table 3.

Table 3.

a) Hurst exponent, H, statistics listed for each participant group. H _ave is the mean value per group, s.d is the standard deviation and the p-values are those found from normality tests (p > 0.05 indicates normally distributed data) on the groups data. b) p-values from double comparison two-sided t-test between participant groups. The significance threshold determined by the Bonferroni comparison is 0.0125.

Group	H _ave	s.d.	p	Group	H	AD	FTD	PSP
H	0.57	0.06	0.80	H	-	0.20	0.72	0.36
AD	0.54	0.02	0.79	AD	0.20	-	0.20	0.22
FTD	0.57	0.06	0.75	FTD	0.72	0.20	-	0.49
PSP	0.55	0.06	0.41	PSP	0.36	0.22	0.49	-

Open in a new tab

Lastly, when we incorporate data from all participants in this study, we find that the gaze dynamics are quantified by a mean Hurst exponent of H = 0.57 (s.d. 0.06). This observed mid-range value suggests the visual system balances between a bias towards low and high frequency movements. Further, these results indicate that the statistics of gaze position for free viewing is consistent across all participants, even those with neural degeneration. This suggests that this behavioral feature may not be controlled for at the cognitive level of the visual system but is inherent to oculomotor processes occurring at an early stage of visual processing.

Discussion

Our results indicate that gaze dynamics during free viewing are consistent with a random walk process where the movements come from a long-tailed Lévy distribution. This is in contrast with the dynamics of fixational eye movements observed when individuals were engaged in fixating on a given location in the visual field [10, 19]. Interestingly, during fixation a Hurst exponent of 0.35 was observed for movements on timescales larger than 20ms [10]. This value of H indicates antipersistent, or subdiffusive, movements. These types of movements are consistent with the demand of the task at hand- the need to keep the gaze localized on the intended fixation position. However, when an individual is engaged in the free viewing scenario of our study, H values indicate superdiffusive movements-the type necessary for visual exploration and that are suppressed during fixation. Within this context it appears that the visual system has different oculomotor strategies for different functional modes of viewing, which are uncovered by looking at the overall temporal structure of gaze dynamics.

Otero-Millan and colleagues have recently proposed a fixation-exploration continuum model in which eye movements have a common generator for exploration and fixation [20]. The persistence of scale invariant structure that we see in the gaze shift temporal dynamics at both ends of the spectrum from exploration to fixation is consistent with this model. Nevertheless, we see a change in the Hurst exponent with a change in task, from super-diffusive to sub-diffusive. An interesting question along these lines is whether the Hurst value varies continuously from exploration to fixation, as is seen with the saccadic measures [20], or whether it changes abruptly as the viewing mode is changed.

The H > 1/2 values revealed in our study indicate human gaze dynamics follow a strategy one would expect to be beneficial in exploration. Animals engaged in foraging exhibit super-diffusive behavior with H values which range from just above 0.5 to 1.24 [21]. Presumably, this range of values exists due to the wide range of conditions involved in the exploration in each case. The Hurst exponents we measure are at the lower end of superdiffusive behavior. Perhaps, just as is witnessed in animal foraging, the visual system takes on a range of H values depending on the particular viewing scenario (scene and task). In the case of the viewing paradigm of this study, the images used had relatively low visual content and the behavioral task was simply to look at the image and judge its aesthetic quality. The H value we observe could indicate that the visual system only engages weakly in exploration leading to a weak case of super-diffusion.

Mathematically, a higher value of H results in more rapid diffusion. It is surprising that increasing the fine scale spatial structure of the image had no effect the gaze dynamics. One might imagine a case where the resulting super-diffusive gaze dynamics could result from normal or Brownian diffusion interacting with a complex visual scene. However, our results show that the scaling is independent of the complexity of the visual scene. Images of high spatial complexity were included with the expectation that gaze dynamics would have a larger H value to effectively explore the more complex image. Even so, we find that image complexity does not induce changes to gaze dynamics.

To expand the ecological relevance of our study, we did a preliminary study in which 5 of our participants (C2, FTD1, FTD4, FTD5 and AD3) viewed photographs of natural landscapes. Example images are shown in Fig 1. Previous studies of human response to fractals employed two types of visual stimuli: computer simulations of fractals [17, 22–27] and photographs of natural landscapes [23, 24, 28–32]. So far in this study we have used the former and benefitted from the ability to adjust fractal complexity in a systematic, controlled fashion. We now incorporate the latter which display additional scale-invariant properties of natural scenery beyond the fractal shapes of the individual objects: these include the size distribution of the objects and the luminance contours established by the surface textures of the objects.

The measured gaze position data show scale invariant gaze dynamics in all cases consistent with the computer-generated fractal images, as shown in Fig 3E and 3F for participant FTD5. Furthermore, we observe no significant difference in mean Hurst exponent between the two sets of data, with a mean H = 0.54 (s.d. 0.06) for the natural landscapes.

It is also surprising that observed H values are robust across individuals. Individuals with Alzheimer Disease exhibit distinct differences in their visual processing, including impairments to visual cognition which impact visual memory and attention and affect visual search processes [33–35]. Patients with degeneration in the frontal and anterior temporal lobes (which includes the behavioral variant of FTD, semantic dementia and progressive nonfluent aphasia) and progressive supranuclear palsy often do not show impairment to higher level visual cognitive processes [36] while PSP patients show significant abnormalities in visually guided (reflexive) saccades and damage to the brainstem oculomotor system [37]. The fact that the AD, FTD and PSP participants all show no significant difference in gaze dynamics from the H participants indicates that this behavioral feature is fundamental to the human visual system and may not interact with higher level search processes or the control of reflexive eye movements. Universality suggests a common mechanism exists in the visual system which controls the generation of the shifts in gaze position and that this mechanism may be controlled at an early stage in visual processing.

How the human visual system responds to natural stimuli is a complicated question with multiple components at play. A scene and the observer can be either static or in motion, the scene itself can contain multiple levels of spatial complexity, and there are also a wide range of tasks during viewing. For this study, our goal was to simplify the natural viewing scenario to static black and white fractal images in which we controlled the spatial complexity of the natural pattern. Free viewing of static images while judging image appeal is only one of the many real world tasks the human visual system engages in and is just a first step in understanding the overall function of the visual system.

Future studies should apply times series analysis methods to gaze position data for participants in a variety of viewing scenarios such as active search to further determine the role task plays in gaze dynamics. In addition, analysis of gaze position data for participants in a variety of human developmental stages, in particular investigations of the gaze dynamics in infants, may tell us something about whether the gaze strategies develop or are robust to different stages of cognitive development.

Rucci and Victor recently addressed the functional importance that the temporal statistics of the class of fixational eye movements known as ocular drift have in image processing. They argue that these statistics play a significant role in the ability to see [13]. The dynamics of gaze position during visual exploration in free viewing of images dictates how the visual world is spatially represented over time to the neural system. We argue that the specific statistics of this strategy establishes a fundamental basis under which the visual system extracts information about the visual word. Efforts, such as ours, to understand what these statistics are and under what conditions they change can provide useful insight into how the human neural system acquires information about its environment.

Acknowledgments

We thank Bruce Miller, Sam Sober and Steve Lisberger for useful discussion.

Data Availability

All raw eye data files, subject and image information are available from the Dryad database (accession number doi:10.5061/dryad.b54gp).

Funding Statement

C. A. Marlow was funded as a Sloan-Swartz Fellow during the data collection phase of this work and during the analysis phase by the Institutional Development Award (IDeA) from the National Institutes of Health under grant number 5 P20 GM103430-13. The experiment was funded through A. Boxer’s National Institutes of Health grants, R01AG038791, R01AG031278, P01AG019724 and the Tau Research Consortium. Swartz Foundation: http://www.theswartzfoundation.org/research_a.asp, Institutional Development Award (IDeA) from the National Institutes of Health: http://web.uri.edu/inbre/ri-inbre/about-ri-inbre/, National Institutes of Health: http://www.nih.gov/, and Tau Research Consortium: http://tauconsortium.org/research. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1. Kowler E. Eye movements: the past 25 years. Vision Research. 2011;51(13):1457–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
2. Itti L, Koch C. Computational modelling of visual attention. Nature Reviews Neuroscience. 2001;2(3):194–203. [DOI] [PubMed] [Google Scholar]
3. Brockmann D, Geisel T. The ecology of gaze shifts. Neurocomputing. 2000;32–33:643–50. [Google Scholar]
4. Boccignone G, Ferraro M. Modelling gaze shift as a constrained random walk. Physica A. 2004;331:207–18. [Google Scholar]
5. Boccignone G, Ferraro M. Ecological sampling of gaze shifts. IEEE Transactions on Cybernetics. 2014;44(2):266–79. 10.1109/TCYB.2013.2253460 [DOI] [PubMed] [Google Scholar]
6. Fairbanks MS, Taylor RP. Scaling analysis of spatial and temporal patterns: from the human eye to the foraging albatross In: Guastello S, Gregson R, editors. Non-linear Dynamical Analysis for the Behavioral Sciences Using Real Data Boca Raton: CRC Press, Taylor and Francis Group; 2011. p. 341–66. [Google Scholar]
7. Stephen DG, Mirman D, Magnuson JS, Dixon JA. Levy-like diffusion in eye movements during spoken-language comprehension. Physical Review E. 2009;79:056114. [DOI] [PMC free article] [PubMed] [Google Scholar]
8. Credidio HF, Teixeira EN, Reis SDS, Moreira AA, Andrade JS Jr. Statistical patterns of visual search for hidden objects. Scientific Reports. 2012;2 10.1038/srep00920 [DOI] [PMC free article] [PubMed] [Google Scholar]
9. Shinde DP, Mehta A, Mishra RK. Searching and fixating: Scale-invariance vs. characteristic timescales in attentional processes. Europhysics Letters. 2011;94(6):68001. [Google Scholar]
10. Engbert R, Kliegl R. Microsaccades keep the eyes' balance during fixation. Psychological Science. 2004;15(6):431–6. [DOI] [PubMed] [Google Scholar]
11. Liang J-R, Moshel S, Zivotofsky AZ, Caspi A, Engbert R, Kliegl R, et al. Scaling of horizontal and vertical fixational eye movements. Physical Review E. 2005;71. [DOI] [PubMed] [Google Scholar]
12. Engbert R, Mergenthaler K, Sinn P, Pikovsky A. An integrated model of fixational eye movements and microsaccades. Proceedings of the National Academy of Sciences. 2011;108(39):E765–E70. [DOI] [PMC free article] [PubMed] [Google Scholar]
13. Rucci M, Victor JD. The unsteady eye: an information-processing stage, not a bug. Trends in Neurosciences. 2015;38(4):195–206. 10.1016/j.tins.2015.01.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
14. Torralba A, Oliva A. Statistics of natural image categories. Network: Computation in Neural Systems. 2003;14(3):391–412. [PubMed] [Google Scholar]
15. Tatler BWH, Mary M. Land Michael F. Ballard Dana H. Eye guidance in natural vision: Reinterpreting salience. Journal of Vision. 2011;11(5):1–23. 10.1167/11.5.5 [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Dubuc B, Quiniou JF, Roques–Carmes C, Tricot C, Zucker SW. Evaluating the fractal dimension of profiles. Physical Review A. 1989;39(3):1500–12. [DOI] [PubMed] [Google Scholar]
17. Taylor RP, Fairbanks MS. Scaling analysis of spatial and temporal patterns: from the human eye to the foraging albatross In: Guastello S, Gregson R, editors. Non-linear Dynamical Analysis for the Behavioral Sciences Using Real Data Boca Raton: CRC Press, Taylor and Francis Group; 2011. p. 341–66. [Google Scholar]
18. Avnir D. Is the geometry of nature fractal? Science. 1998;279(5347):39–40. [Google Scholar]
19. Mergenthaler K, Engbert R. Modeling the control of fixational eye movements with neurophysiological delays. Physical Review Letters. 2007;98:138104 [DOI] [PubMed] [Google Scholar]
20. Otero-Millan J, Macknik SL, Langston RE, Martinez-Conde S. An oculomotor continuum from exploration to fixation. Proceedings of the National Academy of Sciences of the United States of America. 2013;110(5):6175–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
21. Viswanathan GM, Raposo EP, da Luz MGE. Lévy flights and superdiffusion in the context of biological encounters and random searches. Physics of Life Reviews. 2008;5(3):133–50. 10.1016/j.plrev.2008.03.002 [DOI] [Google Scholar]
22. Hagerhall CM, Laike T, Kuller M, Marcheschi E, Boydston C, Taylor RP. Human physiological benefits of viewing nature: EEG response to exact and statistical fractal patterns. Nonlinear Dynamics, Psychology, and Life Sciences. 2015;19:1–12. [PubMed] [Google Scholar]
23. Spehar B, Clifford CWG, Newell BR, Taylor RP. Universal aesthetic of fractals. Computers and Graphics. 2003;27(5):813–20. [Google Scholar]
24. Taylor RP, Spehar B, Wise JA, Clifford CWG, Newell BR, Hagerhall CM, et al. Perceptual and physiological response to the visual complexity of fractals. Nonlinear Dynamics, Psychology, and Life Sciences. 2005;9(1):89–114. [PubMed] [Google Scholar]
25. Rogowitz BE, Voss RF, editors. Shape perception and low dimensional fractal boundary contours Proc SPIE 1249, Human Vision and Electronic Imaging: Models, Methods, and Applications; 1990; Santa Clara, Ca. [Google Scholar]
26. Cutting JE, Garvin JJ. Fractal curves and complexity. Perception and Psychophysics. 1987;42(4):365–70. [DOI] [PubMed] [Google Scholar]
27. Aks D, Sprott J. Quantifying aesthetic preference for chaotic patterns. Empirical Studies of the Arts. 1996;14(1):1–16. [Google Scholar]
28. Field DJ. Relationships between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America. 1987;4(12):2379–94. [DOI] [PubMed] [Google Scholar]
29. Field DJ, Brady N. Visual sensitivity, blur and the sources of variability in the amplitude spectra of natural scenes. Vision Research. 1997;37(23):3367–83. [DOI] [PubMed] [Google Scholar]
30. Billock VA. Neural acclimation to 1/f spatial frequency spectra in natural images transduced by the human visual system. Physica D: Nonlinear Phenomena. 2000;137(3–4):379–91. [Google Scholar]
31. Billock VA, de Guzman GC, Kelso JAS. Fractal time and 1/f spectra in dynamic images and human vision. Physica D: Nonlinear Phenomena. 2001;148(1–2):136–46. [Google Scholar]
32. Redies C. A universal model of esthetic perception based on sensory coding of natural stimuli. Spatial Vision. 2007;21:97–117. [DOI] [PubMed] [Google Scholar]
33. Rizzo M, Anderson SW, Dawson J, Nawrot M. Vision and cognition in Alzheimer’s disease. Neuropsychologia. 2000;38:1157–69. [DOI] [PubMed] [Google Scholar]
34. Ko PC, Ally BA. Visual Cognition in Alzheimer’s Disease and Its Functional Implications In: De La Monte S, editor. The Clinical Spectrum of Alzheimer's Disease-The Charge Toward Comprehensive Diagnostic and Therapeutic Strategies,: InTech; 2011. [Google Scholar]
35. Viskontas IV, Boxer AL, Fesenko J, Matlin A, Heuer HW, Mirsky J, et al. Visual search patterns in semantic dementia show paradoxical facilitation of binding processes. Neuropsychologia. 2011;49(3):468–78. 10.1016/j.neuropsychologia.2010.12.039 [DOI] [PMC free article] [PubMed] [Google Scholar]
36. Viskontas IV, Miller BL. Frontotemporal dementia In: Miller BL, Boeve B, editors. Behavioral Neurology of Dementia: Cambridge Press; 2009. p. 45–55. [Google Scholar]
37. Boxer A, Garbutt S, Seeley W, Jafari A, Heuer H, Mirsky J, et al. Saccade abnormalities in autopsy-confirmed frontotemporal lobar degeneration and Alzheimer disease. Archives of Neurology. 2012;69(4):509–17. 10.1001/archneurol.2011.1021 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

All raw eye data files, subject and image information are available from the Dryad database (accession number doi:10.5061/dryad.b54gp).

[pone.0139379.ref001] 1. Kowler E. Eye movements: the past 25 years. Vision Research. 2011;51(13):1457–83. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref002] 2. Itti L, Koch C. Computational modelling of visual attention. Nature Reviews Neuroscience. 2001;2(3):194–203. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref003] 3. Brockmann D, Geisel T. The ecology of gaze shifts. Neurocomputing. 2000;32–33:643–50. [Google Scholar]

[pone.0139379.ref004] 4. Boccignone G, Ferraro M. Modelling gaze shift as a constrained random walk. Physica A. 2004;331:207–18. [Google Scholar]

[pone.0139379.ref005] 5. Boccignone G, Ferraro M. Ecological sampling of gaze shifts. IEEE Transactions on Cybernetics. 2014;44(2):266–79. 10.1109/TCYB.2013.2253460 [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref006] 6. Fairbanks MS, Taylor RP. Scaling analysis of spatial and temporal patterns: from the human eye to the foraging albatross In: Guastello S, Gregson R, editors. Non-linear Dynamical Analysis for the Behavioral Sciences Using Real Data Boca Raton: CRC Press, Taylor and Francis Group; 2011. p. 341–66. [Google Scholar]

[pone.0139379.ref007] 7. Stephen DG, Mirman D, Magnuson JS, Dixon JA. Levy-like diffusion in eye movements during spoken-language comprehension. Physical Review E. 2009;79:056114. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref008] 8. Credidio HF, Teixeira EN, Reis SDS, Moreira AA, Andrade JS Jr. Statistical patterns of visual search for hidden objects. Scientific Reports. 2012;2 10.1038/srep00920 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref009] 9. Shinde DP, Mehta A, Mishra RK. Searching and fixating: Scale-invariance vs. characteristic timescales in attentional processes. Europhysics Letters. 2011;94(6):68001. [Google Scholar]

[pone.0139379.ref010] 10. Engbert R, Kliegl R. Microsaccades keep the eyes' balance during fixation. Psychological Science. 2004;15(6):431–6. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref011] 11. Liang J-R, Moshel S, Zivotofsky AZ, Caspi A, Engbert R, Kliegl R, et al. Scaling of horizontal and vertical fixational eye movements. Physical Review E. 2005;71. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref012] 12. Engbert R, Mergenthaler K, Sinn P, Pikovsky A. An integrated model of fixational eye movements and microsaccades. Proceedings of the National Academy of Sciences. 2011;108(39):E765–E70. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref013] 13. Rucci M, Victor JD. The unsteady eye: an information-processing stage, not a bug. Trends in Neurosciences. 2015;38(4):195–206. 10.1016/j.tins.2015.01.005 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref014] 14. Torralba A, Oliva A. Statistics of natural image categories. Network: Computation in Neural Systems. 2003;14(3):391–412. [PubMed] [Google Scholar]

[pone.0139379.ref015] 15. Tatler BWH, Mary M. Land Michael F. Ballard Dana H. Eye guidance in natural vision: Reinterpreting salience. Journal of Vision. 2011;11(5):1–23. 10.1167/11.5.5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref016] 16. Dubuc B, Quiniou JF, Roques–Carmes C, Tricot C, Zucker SW. Evaluating the fractal dimension of profiles. Physical Review A. 1989;39(3):1500–12. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref017] 17. Taylor RP, Fairbanks MS. Scaling analysis of spatial and temporal patterns: from the human eye to the foraging albatross In: Guastello S, Gregson R, editors. Non-linear Dynamical Analysis for the Behavioral Sciences Using Real Data Boca Raton: CRC Press, Taylor and Francis Group; 2011. p. 341–66. [Google Scholar]

[pone.0139379.ref018] 18. Avnir D. Is the geometry of nature fractal? Science. 1998;279(5347):39–40. [Google Scholar]

[pone.0139379.ref019] 19. Mergenthaler K, Engbert R. Modeling the control of fixational eye movements with neurophysiological delays. Physical Review Letters. 2007;98:138104 [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref020] 20. Otero-Millan J, Macknik SL, Langston RE, Martinez-Conde S. An oculomotor continuum from exploration to fixation. Proceedings of the National Academy of Sciences of the United States of America. 2013;110(5):6175–80. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref021] 21. Viswanathan GM, Raposo EP, da Luz MGE. Lévy flights and superdiffusion in the context of biological encounters and random searches. Physics of Life Reviews. 2008;5(3):133–50. 10.1016/j.plrev.2008.03.002 [DOI] [Google Scholar]

[pone.0139379.ref022] 22. Hagerhall CM, Laike T, Kuller M, Marcheschi E, Boydston C, Taylor RP. Human physiological benefits of viewing nature: EEG response to exact and statistical fractal patterns. Nonlinear Dynamics, Psychology, and Life Sciences. 2015;19:1–12. [PubMed] [Google Scholar]

[pone.0139379.ref023] 23. Spehar B, Clifford CWG, Newell BR, Taylor RP. Universal aesthetic of fractals. Computers and Graphics. 2003;27(5):813–20. [Google Scholar]

[pone.0139379.ref024] 24. Taylor RP, Spehar B, Wise JA, Clifford CWG, Newell BR, Hagerhall CM, et al. Perceptual and physiological response to the visual complexity of fractals. Nonlinear Dynamics, Psychology, and Life Sciences. 2005;9(1):89–114. [PubMed] [Google Scholar]

[pone.0139379.ref025] 25. Rogowitz BE, Voss RF, editors. Shape perception and low dimensional fractal boundary contours Proc SPIE 1249, Human Vision and Electronic Imaging: Models, Methods, and Applications; 1990; Santa Clara, Ca. [Google Scholar]

[pone.0139379.ref026] 26. Cutting JE, Garvin JJ. Fractal curves and complexity. Perception and Psychophysics. 1987;42(4):365–70. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref027] 27. Aks D, Sprott J. Quantifying aesthetic preference for chaotic patterns. Empirical Studies of the Arts. 1996;14(1):1–16. [Google Scholar]

[pone.0139379.ref028] 28. Field DJ. Relationships between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America. 1987;4(12):2379–94. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref029] 29. Field DJ, Brady N. Visual sensitivity, blur and the sources of variability in the amplitude spectra of natural scenes. Vision Research. 1997;37(23):3367–83. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref030] 30. Billock VA. Neural acclimation to 1/f spatial frequency spectra in natural images transduced by the human visual system. Physica D: Nonlinear Phenomena. 2000;137(3–4):379–91. [Google Scholar]

[pone.0139379.ref031] 31. Billock VA, de Guzman GC, Kelso JAS. Fractal time and 1/f spectra in dynamic images and human vision. Physica D: Nonlinear Phenomena. 2001;148(1–2):136–46. [Google Scholar]

[pone.0139379.ref032] 32. Redies C. A universal model of esthetic perception based on sensory coding of natural stimuli. Spatial Vision. 2007;21:97–117. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref033] 33. Rizzo M, Anderson SW, Dawson J, Nawrot M. Vision and cognition in Alzheimer’s disease. Neuropsychologia. 2000;38:1157–69. [DOI] [PubMed] [Google Scholar]

[pone.0139379.ref034] 34. Ko PC, Ally BA. Visual Cognition in Alzheimer’s Disease and Its Functional Implications In: De La Monte S, editor. The Clinical Spectrum of Alzheimer's Disease-The Charge Toward Comprehensive Diagnostic and Therapeutic Strategies,: InTech; 2011. [Google Scholar]

[pone.0139379.ref035] 35. Viskontas IV, Boxer AL, Fesenko J, Matlin A, Heuer HW, Mirsky J, et al. Visual search patterns in semantic dementia show paradoxical facilitation of binding processes. Neuropsychologia. 2011;49(3):468–78. 10.1016/j.neuropsychologia.2010.12.039 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0139379.ref036] 36. Viskontas IV, Miller BL. Frontotemporal dementia In: Miller BL, Boeve B, editors. Behavioral Neurology of Dementia: Cambridge Press; 2009. p. 45–55. [Google Scholar]

[pone.0139379.ref037] 37. Boxer A, Garbutt S, Seeley W, Jafari A, Heuer H, Mirsky J, et al. Saccade abnormalities in autopsy-confirmed frontotemporal lobar degeneration and Alzheimer disease. Archives of Neurology. 2012;69(4):509–17. 10.1001/archneurol.2011.1021 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Temporal Structure of Human Gaze Dynamics Is Invariant During Free Viewing

Colleen A Marlow

Indre V Viskontas

Alisa Matlin

Cooper Boydston

Adam Boxer

Richard P Taylor

Roles

Abstract

Introduction