Yarbus, eye movements, and vision

Abstract

The impact of Yarbus's research on eye movements was enormous following the translation of his book Eye Movements and Vision into English in 1967. In stark contrast, the published material in English concerning his life is scant. We provide a brief biography of Yarbus and assess his impact on contemporary approaches to research on eye movements. While early interest in his work focused on his study of stabilised retinal images, more recently this has been replaced with interest in his work on the cognitive influences on scanning patterns. We extended his experiment on the effect of instructions on viewing a picture using a portrait of Yarbus rather than a painting. The results obtained broadly supported those found by Yarbus.

1. Introduction

Al'fred Luk'yanovich Yarbus (Iarbus) is one of the founders of modern eye movement research; two portraits of him are shown in figure 1. His book Eye Movements and Vision, published in Russian in 1965 and translated into English by Basil Haigh in 1967, has had a profound influence on recent approaches to the study of eye movements and vision. The impact has been widespread across a range of disciplines, and his book now stands as one of the single most cited publications in the area. It is therefore somewhat surprising that so little is known about the man himself and his career in science. In this paper we present some new details about Yarbus, consider the impact of his eye movement studies on contemporary vision research, and revisit and extend Yarbus's now-classic exploration of how the instructions given to an observer can influence the observer's eye movement behaviour. Our report differs from Yarbus's work by exploring inspection behaviour when looking at a photographic image of a single individual (Yarbus himself) rather than a painted scene containing several individuals and a range of objects (Ilya Yefimovich Repin's The Unexpected Visitor⁽¹⁾).

Figure 1.

Al'fred Luk'yanovich Yarbus (1914–86). (a) Yarbus during his military service, taken around 1944. (b) Yarbus photographed during the 1980s (photograph courtesy of Galina Rozhkova).

2. Yarbus

The name Yarbus is known to almost every student of experimental psychology. Moreover, figures from his book are reproduced in most textbooks on perception. Despite this widespread exposure to his name and to his research, very little is known to English readers about Yarbus himself. Almost all that is available can be gleaned from the few lines printed inside the English edition of Eye Movements and Vision:

Al'fred Luk'yanovich Yarbus was born in Moscow in 1914. He was graduated from the Faculty of Physics of Moscow University in 1941 and was a scientific assistant at the Institute of Biophysics of the Academy of Sciences of the USSR until 1963. He is presently a senior scientist at the Institute for Problems of Information Transmission of the Academy. In 1964 he received the degree of Doctor of Biological Sciences for his work on ‘The Role of Eye Movements in Vision’ (Yarbus, 1967, page iv).

No further biographical details appear in the book; nor were we able to trace any further substantive information until recently. Thanks to personal communications with a number of researchers, we have been able to piece together the following brief outline of Yarbus's professional and personal life.

Yarbus was born on April 3, 1914, in Moscow to Polish parents who had immigrated to Russia some years earlier because of parental disapproval of their marriage. His natural father died when he was young, and his mother remarried Lukian Yarbus, from whom Al'fred Luk'yanovich took his name. Yarbus studied in the Department of Physics at the Moscow Lomonosov State University between 1935 and 1941. After graduating, he worked as an engineer in Moscow, before his military service, which spanned from 1942 to 1946 and included time in the Far East on the Japanese front (see figure 1a for an image of Yarbus during this period). During this time, Yarbus was married to Shalgovsakaya Yevgnia Yohanesovna, who was of German extraction. They had a daughter Francheska, born in 1942 while Yarbus was at the front on war service. Francheska has subsequently followed a successful and distinguished career as an artist. She married Yuri Norstein, and the two are internationally famous for their animated cartoons.

Following the war, Al'fred Luk'yanovich worked as a researcher at the Institute of Crystallography before completing a PhD on visual illusions in under the supervision of Professors Kravkov (sensory physiology) and Rubenstein (psychology). Following this, Yarbus obtained a position as a researcher and then (from 1957) as a senior research fellow at the Biophysics Institute of the USSR Academy of Sciences, which in 1963 became part of the newly established Institute for Problems in Information Transmission. His work on visual illusions subsequently led to interest in eye movements and to the psychological and biophysical effects of stabilised images. At that time he became one of a group of talented scientists that included mathematicians, physicists, and physicians. The leaders of this interdisciplinary laboratory were Michael Moiiseevich Bongard (the founder of Russian research in artificial intelligence and pattern recognition) and Michael Sergeevich Smirnov (an expert in sensory physiology and communication). The technician of the group, Vadim Ivanovich Chernishev, was the first to make suction cups to attach to the eyes. Initially these were largely impractical due to discomfort for the wearer; it was Yarbus who proposed refinements to the technique, which, after a number of trials and errors, led to the development of a range of suction devices for eye tracking (see below and figure 2b). These devices were used successfully throughout his research and were adopted by other laboratories for similar research studies. For his work using suction devices to measure eye movements, Yarbus was awarded his second academic degree, a PhD in Biological Science in 1964.

Figure 2.

(a) dust cover of the first English edition of Eye Movements and Vision (1967); (b) two of Yarbus's suction caps, together with the recording device. (With kind permission from Springer Science and Business Media.)

Yarbus's eye movement work progressed in the 1950s and 1960s, with regular journal publications. In 1965 he wrote the book which, in its 1967 English translation (Eye Movements and Vision), has become a well-known classic. Basil Haigh was a noted translator, and reviews of Yarbus's book often commended the quality and the clarity of the translation. Haigh was trained in medicine and learned Russian when he served in the British embassy in Moscow between 1950 and 1953. After he took a position at Cambridge University in 1959, Haigh translated many texts by Russian neuroscientists, particularly the books of A R Luria.

Yarbus's work was already known to Western scientists, in part because many of the articles were published in Biofizika, a Russian journal whose first issue appeared in 1955, and which maintained an English version, Biophysics. Pick (1964) wrote a review of Russian research on perception in which Yarbus was mentioned briefly, but wider recognition was to follow. Sporadic citations appear in articles from most of the leading laboratories working on eye movements (Barlow 1963; Clowes and Ditchburn 1959; Krauskopf and Riggs 1959; Rashbass and Westheimer 1961a, 1961b; Robinson 1964), although not to the extent that would be the case after the book was published: see, for example, the extensive attention that his work is given in the books by Alpern (1969) and Lévy-Schoen (1969).

Most of Yarbus's early work appears as single-author publications, although his name can be found on multi-author works, including some in the Western European literature coauthored with the distinguished Soviet psychologist and neuroscientist Alexander Romanovich Luria (Karpov, Luria, and Yarbus 1968; Luria, Karpov, and Yarbus 1966; Luria, Pravdina-Vinskaya, and Yarbus 1963). These few publications are the first in the literature considering highly diagnostic features of oculomotor behaviour after differently localised instances of brain damage. Despite the wide international and interdisciplinary recognition of Yarbus's work (or perhaps due to it), this was evidently not an easy period in his life, both professionally and personally. Some of his co-workers appear to have resented their exclusion from the credit, and relations in the laboratory became difficult. Also, other work in the laboratory gave rise to debate about the generalisability of some of his findings (see Rozhkova, Nikolaev, and Shchadrin 1982).

Yarbus divorced and remarried at the age of 53, and a son, Anton (currently a prose writer living in Moscow), was born to this marriage. He subsequently pursued a more lone individual course of research and a series of ten papers entitled “Work of the human visual system” appeared in Biofizika between 1975 and 1980. He was interested in a kind of analogue electrical modelling of human vision during the last two decades of his life, conducting numerous experiments on subthreshold summation of brightness and colour stimuli. In 1986 Yarbus developed cancer and died. Figure 1b shows Yarbus in a photograph thought to have been taken during the 1980s. Yarbus's career spanned a sensitive period for Soviet science and its support; we refer the reader to Krementsov (1997) for a survey of Soviet science during this period.

2.1. Eye movements and vision

The original Russian version of Yarbus's book was entitled , the direct translation being The Role of Eye Motion in Vision Processes (DeAngelus and Pelz 2009). This monograph was the culmination of several years of research by Yarbus, during which he developed a novel set of devices for recording and compensating for eye movements. An English translation of the monograph was published in 1967 (figure 2a) and reprinted in 1971.

The cornerstone of Yarbus's research summarised in his monograph was the development of a method for accurately recording eye movements, using suction caps on the eyes (figure 2b shows two of the eight discussed in the book). The caps developed by Yarbus allowed stable recordings of eye position over extended periods of recording. Importantly, he developed devices that allowed images to be presented that moved with the eyes such that a stabilised retinal image could be presented. That this was his principal concern is evident from the opening sentences of his preface: “This book deals with the perception of images which are strictly stationary relative to the retina, the principles governing human eye movements, and the study of their role in the process of vision” (page ix).

The first two chapters, constituting half the book, are devoted to describing the methods developed by Yarbus and his colleagues and the use of these to study the perception of stabilised retinal images, respectively. Yarbus himself makes clear that these are the two key chapters of the book. In his preface to the English version he dwells primarily on the suction cap methods and his hopes that these techniques might be adopted by other laboratories: “In the author's opinion, this technique is suitable for use in studying a wide range of phenomena. It would be a source of great satisfaction if this technique were to be adopted in the research laboratory and new and interesting results obtained by its use” (page vii). In the introductory chapter Yarbus states that chapter two is “the most important in the book” (page 2). Similar emphasis on this first half of the book is placed by the publishers in the inside dust cover of the book, and in Gerald Westheimer's (1968) review of the book in Science:

Yarbus's significant contribution is the development of tightly fitting contact lenses that do not move at all with respect to the eye. As a result he achieves stabilization good enough and lasting long enough that there is no reappearance of even very bright lights for several minutes…In what must surely be the most important segment of this book, Yarbus describes the appearance of stabilized visual fields of various colors to which are added fields of different colors, stabilized and unstabilized (page 657).

In the early 1950s, the issue of image stabilisation attracted several groups of researchers, most notably Ditchburn and Ginsborg at Reading University, Riggs and Ratliff at Brown University, and Barlow at Cambridge University. Ratliff and Riggs (1950) employed an optical lever system and photography to record the involuntary motions of the eye during fixation. Barlow (1952) placed a droplet of mercury on the cornea and photographed the eye during motion and fixation; the instabilities during fixation were small but measurable. Ditchburn and Ginsborg (1952) used an optical system with a disc subtending 1° separated into halves vertically. Luminance differences between the two parts were visible initially but disappeared after 2 or 3 s; the diffuse disc remained visible with occasional reappearances of the bipartite disc (see Ditchburn 1973). Yarbus developed a fuller-fitting contact lens which retained its position by suction. With his optical system, he found that “in any test field, unchanging and stationary with respect to the retina, all visible differences disappear after 1–3 sec, and do not reappear in these conditions” (Yarbus 1967, page 59). The research was concerned with determining the visual consequences of compensating for involuntary eye movements, but there were some differences between the various studies. The source of these was examined by Barlow (1963), who used a full-fitting contact and a suction cap after the manner of Yarbus in order to compare their possible slippage; an afterimage was used as a perfectly stabilised target. The suction cap was more stable than the full-fitting contact lens, but neither was free from some slippage. He concluded, “Good-quality images stabilized as well as possible ‘blur’ and lose contrast rapidly: detail and texture cease to be visible, and do not reappear” (Barlow 1963, page 50).

The suction caps developed by Yarbus were not only for the purpose of presenting stabilised retinal images but also for recording eye position while inspecting external stimuli. A series of suction caps were developed for this purpose, two of which are shown in figure 2b. The variety of designs was employed in order to meet different experimental requirements and to offer a set of devices that would be able to meet a wide range of experimental paradigms.

The second half of Eye Movements and Vision is much broader in scope and contains a large number of eye movement recordings exploring a variety of topics. As Westheimer (1968) explains, in this second half of the book “[t]he approach, an indigenous mix of psychology and cybernetics, is interesting and obviously productive, but it is not by any means superior to the research strategies with which we are more conversant” (page 657). Here Yarbus considers miniature movements during fixation (chapter 3), detailed kinematics of individual saccades (chapter 4), vergence (chapter 5), and pursuit (chapter 6). However, it is the final chapter of the book, entitled “Eye movements during perception of complex objects”, that has become Yarbus's key contribution to the recent history of eye movement research. This chapter is dense with plots of eye movement records and considers an impressively wide range of questions about how people inspect complex objects and scenes. Of particular note in this chapter is a series of records of observers viewing Ilya Repin's The Unexpected Visitor (figure 3). Here Yarbus showed that when different people viewed the same painting, the patterns of eye movements were similar but not identical. When a single individual was shown the same painting a number of times, with between one and two days separating the recording sessions, the eye movement records from successive viewings were again very similar but not identical. It was evident, however, that similarity between the inspection patterns for a single observer was greater than it was between observers. Yarbus also considered how viewing behaviour changes over extended periods of time, looking at eye movement behaviour in selected sections of a long (3 min) recording session, and then exploring in great detail the first 35 s of a viewing period. Early in the viewing period, fixations were particularly directed to the faces of the individuals in the painted scene. The general trend for fixations when viewing scenes to fall preferentially on persons within the scene had been known previously (Buswell 1935)—although, rather surprisingly, it has only recently been demonstrated that this trend is present on the very first fixation (Fletcher-Watson et al 2008).

Figure 3.

The Unexpected Visitor. Oil on canvas painting by Ilya Repin, 1884–88. (Source: Courtesy ofwww.ilyarepin.org.)

Dissecting a long viewing period into a series of sections suggested to Yarbus that when we view a complex scene for an extended period of time, we show repeated cycles of inspection behaviour, where repeated 25 s samples from 3 min recording sessions revealed the following: “analysis of these separate records shows that each of them, roughly speaking, corresponds to a cycle during which the eye stops and examines the most important elements of the picture” (1967, page 194). This cyclic behaviour was also found when an observer viewed a photographic portrait showing only a face. In two such recordings, the observer was found to cycle periodically through the triangle describing the eyes, nose, and mouth of the pictured subject (figure 4). While it was a relatively minor contribution to the chapter, this report of face scanning has assumed a prominent place in the history of face research. This study of face viewing provided two key insights into how we look at faces. First, it shows there is a strong preference to look at the eyes more than any other feature of the face. Second, for extended viewing there is a clear tendency to make repeated cycles of fixations between the key features of a face. Yarbus's work has become widely cited in the face research community for both of these key early insights into face viewing (see Kingstone 2009).

Figure 4.

Cyclic fixation behaviour while viewing faces. (a) “Girl from the Volga”, viewed with no instructions for 3 min. (b) a girl's face viewed with no instruction for 1 min. From Yarbus (1967, figures 114 and 115). (With kind permission from Springer Science and Business Media.)

The observation of cyclic scanning behaviour, and indeed of cognitive influences on viewing (see below), has had far-reaching consequences. Notably, this work underpinned the Scanpath Theory developed by Noton and Stark (1971). This theory proposes that the paths taken by the eye during extended viewing are an integral part of our perception of complex images. Furthermore, Noton and Stark suggest that the replicability of eye movement patterns for repeated viewings of a stimulus or during extended viewing reflects this sensorimotor aspect of perception. While the theory has subsequently been criticised, it is clear that Yarbus's work profoundly influenced the development of these ideas.

However, despite the richness of this chapter, the vast majority of citations of Yarbus's book draw upon the data presented in just one of the figures in the series of recordings of observers viewing The Unexpected Visitor and the surrounding discussion of these data. Yarbus's invaluable contribution in this chapter was to ask the same individual to view the painting seven times, each time with a different instruction before starting to view the image. These instructions asked the viewer to make a series of judgements about the scene depicted, to remember aspects of the scene, or simply to look at it freely. The data illustrated compellingly that simply altering the instructions given to the observer, and thus their task while viewing, had a profound effect on the inspection behaviour of the observer (figure 5). As Yarbus observed: “Depending on the task in which a person is engaged, ie, depending on the character of the information which he must obtain, the distribution of the points of fixation on an object will vary correspondingly, because different items of information are usually localized in different parts of an object” (page 192).

Figure 5.

Examining a picture (The Unexpected Visitor) with different questions in mind. Each record lasted 3 min. (a) Free examination. (b) Estimate the material circumstances of the family in the picture. (c) Give the ages of the people. (d) Surmise what the family had been doing before the arrival of the ‘unexpected visitor’. (e) Remember the clothes worn by the people. (f) Remember the position of the people and objects in the room. (g) Estimate how long the unexpected visitor had been away from the family. (Illustration adapted from Yarbus 1967, figure 109, for Land and Tatler 2009).

3. Questions raised by Yarbus's work

As we have seen above, the final chapter of Eye Movements and Vision included two key themes, which have been revisited frequently in the literature over the years since the book was published and for which Yarbus's book is often cited. First, the chapter was concerned with scanning behaviour while viewing faces. Second, the chapter was concerned with how the instructions given to an observer influenced inspection behaviour when viewing a complex social scene (The Unexpected Visitor). In the present paper we return to these two central themes of Yarbus's work. The portrait of the young Yarbus (figure 1a) provides a convenient (and fitting) stimulus to allow us to extend and combine the themes of the final chapter of Eye Movements and Vision. First, we can use this portrait to consider face viewing when more of the person than just the face is present, an issue that has received little attention in the face viewing literature. Second, we can consider the influence of instructions on viewing a less contrived image than The Unexpected Visitor. Moreover, this second issue allows us to consider how instructions influence the manner in which we view faces. Both of these questions were addressed by conducting a single experiment as detailed below. In the sections that follow we take the same approach as Yarbus of combining qualitative pictorial representations of the data and quantitative analyses to explore our two questions.

3.1. Participants

Twenty participants from the University of Dundee took part on a voluntary basis. All had normal or corrected-to-normal vision, had no prior knowledge of Yarbus or his work, and were naive to the purposes of the study.

3.2. Procedure

Participants each viewed the portrait of Yarbus, which was embedded in a 1600 by 1200 pixel white background screen such that the portion of the portrait shown in figure 1a measured 1036 pixels high by 734 pixels wide. Displayed on a 21 in SVGA Iiyama monitor, positioned 60 cm from the observer, the portrait subtended approximately 26 by 18.4 deg in the observer's field of view.

The same stimulus was shown seven times in random order for each participant. Prior to each presentation of the portrait, a set of written instructions was displayed on the screen for 10 s. Table 1 shows the seven different instructions given to participants. After 10 s, the instructions disappeared and were replaced by the portrait of Yarbus, which was presented for 50 s. After 50 s, the portrait was replaced with a blank, white screen and the observers were asked to provide a verbal answer to the question asked in the instructions, where appropriate. These verbal responses were not analysed.

Table 1.

The seven sets of written instructions shown during the experiment.

Condition	Instructions When the picture appears…
1	Simply look at it in whatever way you want.
2	Estimate how wealthy this person was and what his social position was.
3	Estimate how old the person in the picture was when it was taken.
4	Estimate what the person had been doing just before this picture was taken.
5	Remember as much as you can about the clothes the person is wearing.
6	Try to remember the positions and details of everything in the picture.
7	Try to estimate how long this person had been away from home when this picture was taken and why he had been away.

3.3. Eye movement recording

Eye movements were recorded as participants viewed the portrait, using an SR Research EyeLink 2 eye tracker set to record gaze position at 500 Hz using only the pupil detection method. Calibration and validation of the eye tracker involved a randomly permuted display of nine circular targets on the monitor. If the validation indicated an average error in excess of 0.5° or a maximum error in excess of 1°, the calibration and validation procedures were repeated.

Fixations and saccades were classified using the software supplied by SR Research. This detects saccades when eye position changes by more than 0.1°, with a minimum velocity of 30° s⁻¹ and a minimum acceleration of 8000° s⁻², over a period of at least 4 ms. No minimum fixation duration or saccade amplitude thresholds were applied. Note that data from two participants were lost, one due to a technical error as the data were being saved, the other due to clear vertical slippage of the eye tracker during the recording session.

4. Face viewing in presence of a body

Studies of face viewing, including Yarbus's own work, typically involve the presentation of isolated faces, devoid of the context of the subject's body (eg Bindemann et al 2009; Butler et al 2005; Guo et al 2010). Yarbus's report of cyclic looking around the eyes and mouth of the face to produce the triangular eye movement records seen in figure 4 has become a widely accepted tenet in face research. However, it is less certain whether such cyclic viewing of the facial features is seen when the face is viewed in the context of the subject's body.

In Yarbus's consideration of face scanning there was no instruction to the viewer before they were presented with the face. We therefore use only data from the free viewing condition in the present analysis. By combining the data from all participants while they viewed the portrait of Yarbus with no explicit instructions (the ‘free viewing’ condition), we can consider whether there were general tendencies to fixate on particular parts of the face.

In order to consider whether there was an overall tendency to look more at the eyes and mouth of the face than other parts of the face, we constructed an overall distribution of the allocation of gaze to the portrait. This distribution was constructed by iteratively adding a Gaussian centred around each sample from the eye tracker (each sample being separated by 2 ms). Each Gaussian had full width at half maximum of 0.5° to reflect estimates of foveal extent. The resulting 3-dimensional landscape contains peaks, the heights of which reflect the cumulative viewing time across all participants. For ease of interpretation, this distribution is presented as a ‘heat map’ overlaid upon the portrait of Yarbus (figure 6). Only the region around the face is shown in this figure to make comparisons to Yarbus's studies of isolated face viewing (figure 4) easier. There is a strong tendency to look at one of the eyes and the mouth of the face in the portrait. The other eye (Yarbus's right eye), which is in relative shadow on the face, received only a small fraction of the overall allocation of gaze time.

Figure 6.

Overall fixation distribution of people free viewing the portrait of Yarbus. The intensity of the coloured overlay indicates the amount of fixation time received across all observers.

While the plot in figure 6 shows an overall tendency to look at the mouth and one of the eyes, it does not allow us to consider whether participants employed the cyclic viewing behaviour reported by Yarbus. For this we must look at the sequences of eye movements made by individual observers. In our dataset we found that the scan patterns over the face in the portrait varied considerably. Four representative plots are shown in figure 7. In some cases, scan patterns resembling those found by Yarbus can be seen (the right hand panels of figure 7). Here we see portions of the scan patterns that involved looks between the key features of the face. Thus, there was some cyclic looking behaviour evident in these trials. However, in many cases (for example, the plots in the left-hand panels of figure 7) no such cyclic patterns of looking around the features were present. In these cases the typical scanning behaviour described by Yarbus was absent. It should be noted that all of the plots in figure 7 include a large number of scans to locations outside the sections of the portrait shown in the plots—these are looks to other parts of the portrait such as the body. We will return to the issue of looks at non-face regions of the portrait in the next section of this paper.

Figure 7.

Scan patterns for four of the observers in the present study as they freely viewed the portrait of Yarbus. As in figure 6, for comparisons to Yarbus's work on face viewing (figure 4), only the facial region of the portrait is shown here.

The individual differences in scan patterns over the facial region of the portrait are consistent with Yarbus's study of individual differences in viewing a complex scene (The Unexpected Visitor). In figure 108 of the final chapter of Eye Movements and Vision, Yarbus plotted the eye movement behaviour for seven different observers, each viewing The Unexpected Visitor for 3 min under free-viewing instructions. We further consider individual differences suggested in figure 7 of the present study by looking at differences across the entire spatial extent of the portrait. Figure 8 shows eye movement data for seven randomly selected participants viewing the portrait of Yarbus. Yarbus's corresponding figure of seven participants looking at the Unexpected Visitor is shown alongside these data for comparison. In both sets of data participants viewed the picture with no instructions other than to simply look at the image. In Yarbus's data, participants viewed the painting for 3 min. In our data each participant viewed the image for 50 s. Yarbus noted that there were global similarities in the patterns of eye movements made by the different subjects when viewing the painting but that there were also clear differences between these observers. In our data the same general observations can be made. However, in some cases (compare panels 1 and 2 in figure 8b) the differences between participants appear quite considerable.

Figure 8.

Seven participants freely viewing pictures. (a) Yarbus's data originally published in figure 107 of Eye Movements and Vision (with kind permission from Springer Science and Business Media). (b) Data from seven randomly selected participants in the present study.

5. Instructions and face viewing

The issue of how a high-level task, such as the instruction given prior to viewing a scene, influences eye movement behaviour has become a question of central interest in eye movement research. The debate about the extent to which fixations are allocated on the basis of internally generated priorities has generated a large volume of research. For an overview of the current state of the continuing debate about the relative contributions of low- and high-level factors in targeting eye movements during scene viewing, we refer the reader to the recent special issue of Visual Cognition on this topic (Tatler 2009) and to Land and Tatler (2009). However, the literature concerned with the relative roles of low- and high-level factors in scene viewing rarely engages with that on face viewing. Indeed, the stimuli employed in scene perception studies are often devoid of faces, comprising natural but unpopulated scenes. Conversely the stimuli employed in face research are often devoid of scenes (or indeed anything other than the face). Yarbus's study of The Unexpected Visitor is a rare example of a scene stimulus that contains human faces, but like other scene perception studies that have included faces (eg Birmingham et al 2009; Torralba et al 2006), these faces are too small to discern the detailed manner in which the facial features are scanned and whether this scanning is sensitive to viewing instructions. Our study allows us to consider whether face scanning is sensitive to viewing instruction, but for a stimulus in which the face is not the only component of the image.

Figure 9 shows the distributions of fixations on the face region of the portrait for all seven instruction conditions. The plots are constructed in the same way as was done for figure 6 above. Note that there are clear global differences between the seven instructions. Most notably, tasks 4 (what had the person been doing?) and 5 (remember the clothes) resulted in a lower proportion of looks to the face region being directed to the eyes.

Figure 9.

Fixation distributions on the face of the portrait for each of the seven instruction conditions. Note that the scaling for the colours varies between plots, with the bars to the right-hand side of each plot showing the viewing times denoted by the intensities of the colours. Viewing times shown in the bars to the right of each plot are in milliseconds and are cumulative across all participants.

From figure 9 it is also clear (as shown in the bars to the right-hand side of each plot) that the amount of time spent looking at the faces varies considerably between tasks. The data suggest that less time was spent looking at the face for tasks 5 (remember the clothes) and 6 (remember everything) than the other tasks. To consider this in more detail, we now turn to the question of the manner in which the entire portrait was viewed. To address this—and for comparison with Yarbus's original data—we first show eye movement records for a single randomly selected participant from the present study, viewing the portrait of Yarbus with seven different instructions (figure 10). As can be seen in both the original data collected by Yarbus and in our own data, the instructions given to the participant had a clear influence on the overall inspection behaviour of this observer during repeated viewings of Yarbus's portrait. It is interesting to note the similarities between the patterns of eye movements made under the seven instruction conditions between our data and those collected by Yarbus. For example, when asked to estimate age(s) of people (instruction 3), observers restricted their eye movements almost exclusively to facial and nearby regions. When asked to remember as much as possible about everything in the image (instruction 6), eye movements explored the full extent of the picture.

Figure 10.

One participant viewing the same image seven times, each with a different set of instructions. (a) Yarbus's original data (from figure 109 in Eye Movements and Vision). (b) Data from the present study. Eye movements from the full 50 s viewing period are shown for each condition. (1) Free examination of the picture. (2) Estimate the material circumstances of the person. (3) Give the age of the person in the picture. (4) Estimate what the person had been doing immediately before the photograph was taken. (5) Remember the clothes worn by the person. (6) Remember the positions and details of everything in the picture. (7) Estimate how long the person had been away from home.

The data in figures 9 and 10 both point to differences in the allocation of gaze across the seven instruction conditions. However, it is important to consider whether these results generalise across participants or are parochial to the selected observer. For The Unexpected Visitor, the question of generalisability of Yarbus's results has been addressed by DeAngelus and Pelz (2009); these authors showed that the same inter-instruction differences can be found in a group of observers. For our data, we address this question by considering gaze allocation across all participants in the present study. Figure 11 shows the distribution of viewing time combined for all participants across the entire portrait for each instruction condition. These distributions highlight clear differences in gaze allocation among instruction conditions and confirm that the differences found in the single participant plotted in figure 10 generalise across a group of participants.

Figure 11.

Gaze distributions for each of the seven instruction conditions. Data are accumulated over all participants. The bars to the right of each plot show the gaze time (in ms) that is indicated by each colour.

Figure 11 clearly shows that there were differences in the spatial allocation of attention to the portrait when viewing it under the seven different instructions. In order to explore these spatial differences quantitatively, we divided the image into nine regions of interest (figure 12) and considered how long was spent fixating on each of these regions of interest.

Figure 12.

Regions of interest.

When considering the allocation of fixations across the nine regions of interest, we can consider the proportion of fixation time in each of the regions (figure 13a). However, the regions of interest vary considerably in their relative sizes (figure 12). We therefore normalised the proportion of viewing time by the area of each region of interest in the following analysis. A two-way ANOVA [instruction condition (7 levels) x region of interest (9 levels)] revealed a main effect of the region of interest, F(8, 136) = 105.07, p < 0.001, partial η² = 0.861. Because fixation times are reported as a (normalised) proportion of the viewing period, there was no main effect of instruction condition. However, there was an interaction between instruction condition and region of interest, F(48, 816) = 3.22, p < 0.001, partial η² = 0.159 (figure 13b).

Figure 13.

(a) Proportion of viewing time in each ROI for each of the seven instruction conditions. (b) Normalised viewing time in each ROI for each of the seven instruction conditions. Regions of interest: 1 = hat, 2 = eyes, 3 = nose, 4 = mouth, 5 = rest of face, 6 = arms, 7 = buttons, 8 = rest of torso, 9 = background.

In general, across all conditions, apart perhaps from when asked to remember the clothes worn, people looked more to Yarbus's facial regions than to other regions of the portrait. Whether normalised or not, viewing time was very low on the arms, buttons, and background of the portrait (figure 13). The torso was looked at quite a lot (figure 13a); however, the torso occupied a large proportion of the image and so would be expected to receive a reasonable amount of viewing time whether or not it was of particular interest to the viewer. Normalising viewing time on the torso by the area of this region showed that it received a small relative fraction of the fixation time (figure 13b).

These data extend Yarbus's original findings concerning the influence of instructions on subsequent viewing behaviour. Here we show that, for a much simpler scene in which only one individual is present, the allocation of gaze changes depending on the instructions given prior to viewing. Moreover, we have shown that the manner in which a face is inspected depends on the instructions given. That is, the typical tendency to view the eyes and mouth, and to cycle between features, is not universal and can be diminished or absent depending on the viewer's task. Since faces are rarely seen in isolation under normal viewing conditions, some caution may be necessary in extending the interpretations from photographed faces to natural behaviour.

6. The legacy of Yarbus

The rather brief final chapter of Yarbus's Eye Movements and Vision has made a profound contribution to the subsequent literature on eye movement control during complex-scene perception. The legacy has been twofold: concerning how people look at faces and how cognitive factors such as instructions given prior to viewing can influence inspection behaviour.

In many respects it is the latter of these two contributions that Yarbus's book has become most well known for. Interestingly, Buswell (1935) had already shown that instructions influenced fixation behaviour. But Yarbus's visualisations of such clear differences in each of his seven records of a single observer viewing a single scene have remained an emblematic illustration of this point. The question of to what extent cognitive factors influence where observers look in complex scenes rose to the forefront of eye movement research in the early 1990s. Yarbus's prominence in this era coincided with a number of distinguished researchers beginning to emphasise the importance of cognitive factors on eye movement control. Two influential publications at the start of the 1990s placed cognitive factors firmly at the forefront of eye movement research: Eileen Kowler's (1990) chapter on cognitive influences on gaze and Keith Rayner's (1992) edited volume on Eye Movements and Visual Cognition. Three papers in the latter discuss the importance of Yarbus's work on cognitive control of fixation selection (Carroll et al 1992; Henderson 1992; Klein et al 1992). The 1990s also saw the emergence of the Active Vision approach to eye movements (Ballard 1991; Ballard et al 1992; Findlay and Gilchrist 2003), and it is at this time that Dana Ballard, Mary Hayhoe, and Mike Land all began to study eye movements in the context of natural behaviour. All of these authors have emphasised the importance of Yarbus's work as a demonstration that where we look depends critically on our cognitive task. Yarbus's work has since become established as a popular, elegant demonstration of high-level influences on gaze control. The question of whether and to what extent low- and high-level factors have a role in selecting where humans direct their gaze when viewing complex scenes continues to be a prominent and controversial topic in eye movement research (see Land and Tatler 2009; Tatler 2009).

The prominence of Yarbus's work in the modern debate about eye guidance is evidenced by the total number and yearly frequency of citations that Eye Movements and Vision has received (figure 14). A noticeable change in the popularity of this work is evident in the mid-1990s. Indeed, studies of eye movements in natural behaviour have been inspired heavily by his work. The notion of task-dependent patterns of fixations has driven and underpinned many of the studies of eye movements in the context of natural behaviour (Land and Lee 1994; Land and Tatler 2009).

Figure 14.

Citations per year of the English edition(s) of Yarbus's book Eye Movements and Vision. At the time of this paper going to press, Yarbus's book had been cited 1612 times, including 47 so far in 2010. (Data collated using the ISI Web of Knowledge to search for citations of Yarbus's book in peer-reviewed journal articles.)

While the prominence of Yarbus's work within the contemporary study of eye movement control during scene perception is clear, it should be noted that the impact of the work is by no means restricted to this field of research. The widespread recognition of the importance of this elegant demonstration of cognitive control of inspection behaviour has had an influence in areas as diverse as neuroscience, artificial intelligence, computer science, and engineering. Indeed, the book has been cited in 501 different journals, covering a wide range of subject areas. Figure 15 summarises the frequency of citations of Eye Movements and Vision across the nine main disciplines in which the book is cited.

Figure 15.

Citations of Eye Movements and Vision across main disciplines.

7. Concluding remarks

It is unusual for so little to be known about a visual scientist whose research is so well known. This applies to Yarbus, particularly amongst the English-speaking scientific community. We hope that we have attached a little more personal flesh to his sturdy scientific skeleton. His book on Eye Movements and Vision is cited with increasing frequency, and its attraction has shifted from the early chapters (relating to image stabilisation) to the later ones (concerning observation of complex scenes).

One area of study that has expanded enormously since 1967 is face perception. The characteristic pattern of eye movements when viewing pictures of faces was established by Yarbus. It has since been replicated many times and has become an accepted canon in face research. We extend this influential finding by considering face scanning when more than the face alone is presented to the viewer. Here we found that while facial features, particularly the eyes and mouth, still receive the greatest fraction of gaze allocation, the cyclic scanning between eyes and mouth reported by Yarbus was less evident when the face is viewed in the context of a body and cannot be seen in all observers.

The effect of instructions on inspecting complex scenes, demonstrated in Yarbus's recordings of a single observer viewing the painting The Unexpected Visitor seven times, each time with different instructions prior to viewing, has inspired and underpinned a wealth of eye-tracking studies since the 1990s. DeAngelus and Pelz (2009) recently returned to Yarbus's study and were able to confirm that the findings described by Yarbus for a single viewer generalised to multiple viewers of The Unexpected Visitor. We extend Yarbus's work by considering the influence of instructions on viewing a portrait. This allows us not only to consider whether Yarbus's findings generalise to a more simple visual stimulus but also the influence of instructions on face viewing. We find similar sensitivity to task instructions for viewing a portrait as was originally reported in Eye movements and Vision. Moreover, the extent to which particular features (and indeed the whole face) are looked at depends on the instructions given prior to viewing.

Yarbus was far seeing in his analysis of vision. In addition to the then contemporary issues he addressed on image stabilisation, his more exploratory experiments on viewing complex scenes strike a chord with those extending the study of eye movements beyond the confines of the laboratory.

Biography

Ben Tatler read Natural Sciences at the University of Cambridge, where he worked with Professor Simon Laughlin FRS on fly vision, before moving to the University of Sussex for his PhD, where he worked with Professor Mike Land FRS. Under Land's supervision, he developed an interest in human eye movement behaviour during natural tasks. After spending three years as a postdoc in Sussex, Ben moved to the University of Dundee in 2004, where he is now a senior lecturer. His main research interest is in how we use our eyes to support our everyday activities, including issues of eye guidance, scene representation, and object memory. For more information visitwww.activevisionlab.org.

Nick Wade received his degree in psychology from the University of Edinburgh and his PhD from Monash University, Australia. This was followed by a postdoctoral fellowship from the Alexander von Humboldt Stiftung, at Max-Planck-Institute for Behavioural Physiology, Germany. His subsequent academic career has been at Dundee University, where he is now Emeritus Professor. His research interests are in the history of vision research, binocular and motion perception, and the interplay between visual science and art. For more information visitwww.dundee.ac.uk/psychology/people/academics/njwade/.

Hoi Kwan. Originally from Hong Kong, Hoi (Jolly) Kwan was educated at Gordonstoun in Scotland. She obtained her degree in psychology from the University of Dundee, during which time she carried out experiments on eye movements. She has since worked in administration at the university.

John Findlay commenced his eye movement research with R. W. Ditchburn in the Physics Department at Reading University. His academic career has been at the Department of Psychology, Durham University, where he is now Emeritus Professor. His research focuses on human visual attention and eye movements; he is particularly interested in search for visual targets in both artificial and natural situations and attentional control in patients with brain damage and when viewing virtual reality displays. For more information visitwww.dur.ac.uk/psychology/staff/?id=582.

Boris Velichkovsky studied psychology at Lomonosov Moscow State University and physics at Berlin Humboldt-University. He has held professorships at the University of Bielefeld, the University of Toronto, and Dresden University of Technology. His research interests are related to mechanisms of human visual cognition and to a combination of methods and technologies derived from different scientific domains such as information technology, biotechnology, and cognitive science. He is a member of the Russian Academy of Sciences and now the director of the Institute for Cognitive Studies at Kurchatov Research Centre in Moscow.