Visual short-term memory guides infants’ visual attention

Abstract

Adults’ visual attention is guided by the contents of visual short-term memory (VSTM). Here we asked whether 10-month-old infants’ (N = 41) visual attention is also guided by the information stored in VSTM. In two experiments, we modified the one-shot change detection task (Oakes, Baumgartner, Barrett, Messenger, & Luck, 2013) to create a simplified cued visual search task to ask how information stored in VSTM influences where infants look. A single sample item (e.g., a colored circle) was presented at fixation for 500 ms, followed by a brief (300 ms) retention interval and then a test array consisting of two items, one on each side of fixation. One item in the test array matched the sample stimulus and the other did not. Infants were more likely to look at the non-matching item than at the matching item, demonstrating that the information stored rapidly in VSTM guided subsequent looking behavior.

1. Introduction

Infants’ visual environment is complex and cluttered, containing many objects and people—both new and familiar—that come in and out of view. Effectively learning about this complex visual environment involves determining which objects are relevant, directing gaze to those objects, and maintaining gaze on those objects in the face of distractions. Infants’ ability to control where and how long they look develops considerably in the first postnatal year (Johnson, 1994). In addition, infants’ looking is determined by a number of different factors such as stimulus complexity (Cohen, 1972), novelty (e.g., Welch, 1974; Wetherford & Cohen, 1973), and top-down features such as social content (e.g., Gliga, Elsabbagh, Andravizou, & Johnson, 2009; Gluckman & Johnson, 2013). Other work has shown that where infants look is influenced by the physical features of the stimuli such as color (e.g., Dannemiller, 1998) and movement (Volkmann & Dobson, 1976).

Most research on infants’ looking has examined their looking behavior in habituation or familiarization procedures, which provide infants with tens of seconds or more to learn about an image and store it in memory. This work has shown that when given adequate time to form a memory representation of the stimulus or stimuli, infants look longer at a previously unseen item (i.e., a novel image) than at the now-familiar item (Oakes & Kovack-Lesh, 2007; Rose, 1981). When infants are given less time to form a representation of the stimuli, they tend to look longer at the familiar item than at a novel one (Hunter, Ross, & Ames, 1982; Rose, 1981). Both patterns of findings demonstrate that information stored in memory over several seconds of learning influences the duration of infants’ continued looking at those stimuli versus novel stimuli.

More recently, there has been an interest in moment-to-moment changes in infants’ looking. Several studies have examined infants’ looking in arrays like those used in studies of visual search in adults. For example, by 6 months infants look first at human faces or other social stimuli rather than at non-social stimuli when presented with arrays of complex, realistic objects (Gliga et al., 2009; Gluckman & Johnson, 2013; Kwon, Setoodehnia, Baek, Luck, & Oakes, 2016). These results show that by the second half of the first year of life, complex information contained in the visual scene can guide infants’ eye-movements.

This recent interest in moment-to-moment changes in infants’ gaze makes contact with the extensive literature on adults’ oculomotor behavior when viewing complex scenes. Several findings are relevant. When viewing complex scenes, adult viewers typically make multiple eye movements per second. Moreover, vision is suppressed during the saccades that separate periods of fixation (Henderson, 2003). As a result, viewers must rapidly create memory representations to span the gaps between fixations (Bays & Husain, 2012). Consider, for example, a viewer looking at a table set out with four cups, four plates, and four spoons. Imagine that the viewer’s attention is drawn to a cup, and that the viewer fixates this cup for several hundred milliseconds. If the viewer can use information acquired during this fixation to guide the next several eye movements, he or she can explore the scene in a flexible and systematic manner. In this way, information that is rapidly obtained in each fixation can be used to determine what information should be acquired next.

Many previous studies have shown that adults have this ability: they can rapidly form visual short-term memory (VSTM) representations and use them to guide their subsequent shifts of covert and overt attention (Downing, 2000; Downing & Dodds, 2004; Olivers, Meijer, & Theeuwes, 2006; Soto, Hodsoll, Rotshtein, & Humphreys, 2008; Woodman & Luck, 2007). For example, adults can store one or more colors in VSTM at the beginning of a trial and then use these colors to guide eye movements toward items of matching color when performing a visual search task later in the same trial (Beck, Hollingworth, & Luck, 2012; Hollingworth, Matsukura, & Luck, 2013; Soto, Heinke, Humphreys, & Blanco, 2005). Information being held in VSTM can also guide covert shifts of attention in the absence of overt eye movements (Kumar, Soto, & Humphreys, 2009). Importantly, VSTM representations are generated rapidly in adults (e.g., 20–50 ms per item, Vogel, Woodman, & Luck, 2006) and can be used to guide attention within 200 ms (Vickery, King, & Jiang, 2005; Vogel et al., 2006). Thus, adults can rapidly store information in VSTM and then use it to guide their moment-to-moment allocation of both covert and overt attention. This is a key element of the architecture of visual cognition, and it allows the visual system to use the currently fixated information within a scene to guide the further acquisition of information from that scene.

Although significant progress has been made in understanding the ability of infants to rapidly create VSTM representations (Oakes et al., 2013; Ross-Sheehy, Oakes, & Luck, 2003), it is not yet known whether infants’ cognitive architecture is sufficiently sophisticated to use the perceptual information gained during one brief period of fixation to control oculomotor behavior moments later. This kind of rapid loop between perception and eye movements would be very useful as infants interact with and learn about the visual environment, but it would also require very sophisticated interactions between perceptual and oculomotor systems. Given the results of previous research, it is possible that infants can guide their visual search of a scene only by means of memory representations built up from information acquired over tens of seconds; if true, however, this would severely limit their ability to flexibly explore new scenes. Thus, although we know from previous research that infants can use information stored in memory to guide their eye movements, we do not yet know whether they can take information acquired in one brief period of fixation and use it to guide their allocation of attention moments later. The primary goal of the present investigation was to determine whether infant’s cognitive architecture can support the rapid looping between perception and gaze control needed for this ability. We focused on 10-month-old infants, because these infants are clearly old enough to store information robustly in VSTM (Oakes et al., 2013; Ross-Sheehy et al., 2003) and yet young enough that they might not yet have the ability to use this information to rapidly control attention.

Examining this question using controlled laboratory methods will provide insights into how infants acquire information from the complex, dynamic scenes they encounter in their everyday experience. Prioritizing visual input based on its relevance to current goals relies, in part, on the ability to recognize which items in the environment have been previously explored, and are thereby familiar, and which items are novel and require further exploration. Indeed, this is the underlying assumption of inhibition of return (IOR, Klein, 2000), or the idea that viewers will inhibit attending to a location that has previously been attended. Even infants show IOR (e.g., Hood & Atkinson, 1993; Valenza, Simion, & Umiltà, 1994), suggesting that moment-to-moment familiarity—at least of an attended location—influences their looking. However, IOR operates on the locations that have been attended and not on the information contained at a given location. Consequently, IOR findings do not show that information about the identity of the currently fixated location is rapidly stored in VSTM and guides subsequent shifts of attention. In the present work, we ask whether infants’ attention is guided in this manner.

To address this question, we created a simplified version of the cued visual search task that has been used to study the control of attention by VSTM in adults (Beck et al., 2012; Wolfe, Horowitz, Kenner, Hyle, & Vasan, 2004). The procedure is illustrated in Figure 1. During an initial sample period, we presented a single item in isolation at fixation (i.e., at the center of the display) for 500 ms, which was followed by a brief (300 ms) retention interval with a blank screen. Finally, we presented a test array of two items (one matching the sample and one different from the sample) at new locations, equidistant from the central fixation point, for 2000 ms (see https://osf.io/5cusz/?view_only=fe9962def5b848289c86bbcbcb6ac1c7 for an example of the sequence of events in in Experiment 2). The question was whether infants would systematically prefer one of the two items during the test period, indicating that (a) they formed a representation of the initial sample item, and (b) the representation guided their subsequent looking behavior. We required infants to look at the sample item at the beginning of each trial, so they were fixed at the center of the display when the test array appeared and had to make an eye movement away from this location to look at either test item. If infants’ looking is unrelated to the information stored during the sample period, their looking at the novel and familiar items during test will be random. However, if their looking reflects the information stored from the sample array, their looking at the novel and familiar items during test will differ, and we will have evidence that infants use information stored in VSTM to guide their visual attention.

Figure 1.

Schematic illustration of an experimental trial sequence in Experiment 1. In each trial, a sample array of a colored cue circle was presented for 500 ms followed by a brief retention period (300 ms). After the retention period, a visual search array (2000 ms) containing the original cue circle and a circle of a different color was presented with the two items appearing 4.41° from the original cue circle, one on each side of fixation.

Previous research using a related paradigm has already demonstrated that infants as young as 8 months can store color information in VSTM. For example, Oakes et al. (2013) presented infants with two items side-by-side during a 500 ms sample period, and after a 300 ms retention delay, they presented infants with a test period in which two items were presented in the same locations as were the items during the sample. During this test, one of the items was a different color than it was during the sample, and 8-month-old infants reliably preferred the changed item as evidenced by longer looking to that item. Thus, infants’ looking during the test period was systematic and reflected information they rapidly acquired during the sample. Importantly, although this procedure revealed that infants could use information stored in VSTM to detect a change, it did not show that infants could use information stored in VSTM to guide subsequent visual search behavior. The infants often moved their eyes to one of the two items in the initial sample array, and if they had merely been more likely to continue fixating that location when the item at that location changed color, this could have explained the greater looking duration at the location of a changed item compared to the location of an unchanged item. This pattern of fixation durations required the use of VSTM to determine whether the item at the currently fixated location matched or mismatched the color that was at the same location a few hundred milliseconds earlier. However, this pattern does not indicate that infants can use these VSTM representations to control the targeting of subsequent eye movements. That is, the previous results may have reflected a relatively simple mechanism for maintaining fixation and not a cognitive architecture in which VSTM can guide visual search behavior. The modified procedure used in the present investigation, in contrast, requires the infants to make an eye movement toward either the matching or nonmatching item during the test period. Thus, although the method used in the present study is quite similar to the method of the previous research, the present method makes it possible to draw broader conclusions about infants’ cognitive architecture.

In the present study, a consistent preference for either the matching item or the nonmatching item would indicate that VSTM was controlling gaze, and we did not know a priori which of the two items would be preferred. Several studies have shown that adults’ attention is directed toward items that match the contents of VSTM (Beck, Luck, & Hollingworth, 2018; Hollingworth & Luck, 2009; Olivers et al., 2006), which suggests that infants would preferentially move their eyes in the direction of the matching item. However, because infants’ looking is strongly influenced by novelty (Welch, 1974; Wetherford & Cohen, 1973), infants may prefer the novel or non-matching item. Indeed, Oakes et al. (2013) observed that infants preferred the changed, or non-matching, item during test. Under some conditions, adults may also inhibit items that match the contents of VSTM, effectively creating a bias toward non-matching items (Sawaki & Luck, 2011). Thus, there is previous evidence to suggest that infants might look at either the matching or the non-matching item if they effectively encode the sample item in VSTM; however, any systematic preference for the matching or non-matching item in this task necessarily reflects an effect of memory for the sample on infants’ subsequent looking behavior.

2. Experiment 1

2. 1. Method

2.1.1. Participants

We tested 32 healthy, typically-developing, full-term 10-month-old infants with no known vision problems and no familial history of colorblindness (i.e., we excluded boys whose mother or maternal relatives were colorblind). Our final sample included 20 infants (M_age = 306.65 days, SD = 12.33; 10 boys). Oakes et al. (2013) observed a large effect size (d = .95) in a similar task. A power analysis confirmed that 20 infants would be sufficient to detect even a modest effect size (e.g., d = .60). Twelve of the infants we tested were excluded due to failure to calibrate (n = 5), failure to contribute the minimum number of trials to the data analysis (see Data Processing section below; n = 5), or fussiness (n = 2). In the final sample, 16 infants were reported as White, 1 infant was reported as Asian, and 3 infants were reported as mixed race. Regardless of race, 3 infants were reported as Hispanic. All mothers had completed high school, and 11 mothers had earned at least a Bachelor’s degree.

We obtained from the state Department of Public Health the names and addresses of all infants born in the three counties closest to our lab. All parents located within a 20-mile radius of the lab were mailed information about our research, and parents indicated an interest in participating by phone, e-mail, online questionnaire, or return post-card. We contacted parents about scheduling an appointment when their infant reached the appropriate age. Infants received a small toy, t-shirt, or bib and a certificate of appreciation for participating.

2.1.2. Apparatus

We recorded eye gaze at 120 Hz using a Sensomotoric Instruments (SMI) RED eye-tracker attached to the bottom of a 22″ Dell LCD monitor (1680 × 1050 pixel resolution); the monitor was affixed to an adjustable arm which allowed us to flexibly adjust the location of the screen relative to each infant’s eyes to get a stable track. The eye tracker emitted an infrared light and determined the point of gaze (POG) from the reflection of this light source off the cornea and pupil from both eyes. We used iView and Experiment Center, software developed by SMI for this eye-tracker, to record eye-gaze information from the eye-tracker and to present the stimuli. The software was run on an Acer computer, received input from the eye-tracker, and provided output to both the Dell stimulus presentation monitor and a second Acer monitor that allowed the experimenter to monitor and control the experiment. A Logitech Carl Zeiss Tessar 2.0/3.7 2MP Autofocus web camera mounted on the top of the stimulus display screen allowed the experimenter to monitor the infant’s behavior.

2.1.3. Stimuli

A schematic of a single trial is presented in Figure 1. The primary experimental stimuli were circles, 2.5 cm tall by 2.5 cm wide (2.38° tall × 2.38° wide at a nominal viewing distance of 60 cm), in 12 easily distinguishable colors (see Table 1 for a list of the RGB values and CIE coordinates). We used these circles to create the sample displays, in which a single circle was presented in the center of the monitor, and the test displays, in which two different colored circles (a matching item that matched the sample in color and a non-matching item that was a different color) were presented away from the center. The test items were presented on an invisible circle, centered on the middle of the screen, with a radius of 4.41° (e.g., from the distance of the test item to the center of the circle, or the distance required to move one’s eyes from the center fixation to the middle of the target). We used three possible configurations of the two items (at 0° and 180°, at 60° and 240°, or at 120° and 300°). For each infant, the target appeared in each of the six possible locations twice, once in the first block and once in the second block of six trials. The test arrays were created using six color pairs; the colors in each pair were chosen to be maximally different from one another (180° apart from one another on the color wheel).

Table 1.

RGB Values and CIE coordinates for the items in each of the 6 pairs of colors used in Experiment 1.

		RGB Values			CIE coordinates
Pair	approximate color	R	G	B	x	y	luminance
One	Red	251	31	23	0.60	0.35	36.3
	Sea Blue	19	134	165	0.21	0.28	50.7
Two	Chocolate	211	107	36	0.48	0.43	54.7
	Cornflower Blue	78	128	253	0.20	0.19	53.9
Three	Olive	134	143	8	0.39	0.53	58.2
	Medium Purple	147	93	251	0.24	0.17	47.5
Four	Medium Orchid	168	81	245	0.25	0.16	45.4
	Kelly Green	86	159	27	0.34	0.56	65.2
Five	Forest Green	23	172	20	0.30	0.60	68.5
	Deep Pink	255	0	222	0.32	0.15	39
Six	Teal	51	163	115	0.26	0.42	63.2
	Violet-red	247	66	133	0.41	0.25	47.2

We also used several types of attention getters between trials to refocus infants’ attention to the center of the monitor. At the start of each trial, infants were presented with a black fixation cross (3.34° h × 2.86° w) that blinked on and off, accompanied by a ringing sound. In addition, brief animated clips (e.g., short clips from Sesame Street or Teletubbies) were used periodically to reengage infants’ general attention to the task.

2.1.4. Procedure

Each infant sat on a parent’s lap approximately 60 cm away from the stimulus presentation monitor. Parents were seated in a stationary chair and wore a pair of opaque glasses. A curtain separated the infant from the experimenter and the equipment. At the start of the experimental session, infants’ POG was calibrated to the eye-tracking system using Experiment Center’s 2-point calibration procedure. A looming circle was presented first at a point above and to the left of the center of the monitor and then at a point below and to the right of the center of the monitor. The eye-tracking system then calculated POG using the corneal and pupil information captured by the eye camera when the infant looked at each of these two points on the screen. Immediately following calibration, a verification procedure was initiated in which infants’ POG was checked by presenting an attention grabbing, animated yellow duck that shook and made noise in the top left corner, top right corner, bottom left corner, and bottom right corner of the screen. If this verification revealed poor calibration, the experimenter reinitiated the procedure.

The experimental phase began immediately after calibration. Each experimental trial was divided into a 500 ms presentation of the sample stimulus (the sample phase), a 300 ms blank period (the retention interval), and a 2000 ms presentation of the test display (the test phase). Infants received a maximum of 12 trials, arranged in 2 blocks of 6 trials. The blocks were created with the following constraints: 1) each color pair was presented only once per block, 2) for each color pair, one color was used as the sample stimulus in the first block and the other color was used as the sample stimulus in the second block, 3) the matching item (i.e., that item in the test array that matched the sample item) was presented in each of the six locations on one trial in each block. Infants were randomly assigned to one of three orders that varied in the particular item that was the target in each pair, the location of the items during test for each pair in the first and second block, and the order of the pairs of items within each block. To maintain infants’ interest level, each experimental trial was paired with a randomly selected instrumental music clip (classical covers of popular musicians). Although the trials were less than 3 s in duration, the music clips sounded natural—2 to 3 s is sufficiently long for a complete phrase or two of music. This procedure has been successfully used in several previous studies (Brandone, Horwitz, Aslin, & Wellman, 2014; Chong, Richmond, Wong, Qiu, & Rifkin-Graboi, 2015; Kwon et al., 2016).

Before each trial, the fixation cross blinked on and off in the center of the screen so that the infant was fixating the location of the sample stimulus when the trial began. Trials were initiated automatically using a “trigger” area of interest (AOI). Specifically, an experimental trial was initiated when the eye-tracking system detected a 200 ms fixation to an AOI surrounding the fixation cross. Trials continued in this fashion until 12 trials were presented or until infants became too fussy to continue.

One might be concerned that the relatively short delay between the sample and test stimuli might lead to the perception of apparent motion, with the sample stimulus appearing to move to the location of the test item with the same color. However, color and shape have relatively little impact on resolving ambiguities in apparent motion (e.g., determining whether the sample item moved to the location of the same-color or different-color test item) (Burt & Sperling, 1981; Navon, 1976, 1983), especially with the kind of long temporal gap between stimuli used in the present study (Kolers & Pomerantz, 1971). Although Green (1989) found that color correspondence can have some impact when the stimuli are isoluminant with the background and the stimulus cycles back and forth many times, color had little or no impact when the background was darker than the objects and a single cycle was presented (as in the present study). Thus, the existing literature indicates that the present stimuli should lead to little or no apparent motion.

2.1.5. Data Processing

Using SMI’s data analysis software, BeGaze, we created AOIs surrounding each item (i.e., the one sample item and the two test items), and around fixation during the test phase (i.e., where the sample item had previously appeared). All AOIs were approximately 4.5 cm tall by 4.5 cm wide (approximately 4.3° h by 4.3° w) thus, a bit bigger than our stimuli to adjust for calibration inaccuracy. From BeGaze, we extracted two measures from the gaze data stream. First, we filtered the data in BeGaze into distinct fixations. Fixations were detected using the dispersion-based algorithm supplied by the BeGaze software and BeGaze’s default fixation detection settings, which were 80 ms of looking within a 100 pixel dispersion (.27° h × .27° w) (Sensomotoric Instruments BeGaze Manual, 2011). This algorithm for defining fixations is within the standard in the field and should provide a relatively robust estimate of fixations (Shic, Scassellati, & Chawarska, 2008). Because we are not comparing infants of different ages, our data are not influenced by differences in quality by different groups (Wass, Smith, & Johnson, 2013); thus adopting this fixation criterion should yield clear and interpretable eye movement data. BeGaze generated a list of fixations for each phase of each trial, including where each fixation fell (e.g., in one of the AOIs, or “outside” an AOI).

We also extracted the summed duration of all samples that fell within each AOI during the sample and the test phases of each trial (i.e., without applying a fixation filter). We used these total durations to determine how long infants looked at the matching and non-matching items during the test array, as well as how long they looked at the sample item.

We used the following criteria for determining which trials (and infants) were included in the final analyses. First, to ensure that our analyses were based only on trials in which the infants looked during both the sample and test phases, we included only trials in which the duration of looking to the AOIs was more than 100 ms during each phase of the trial (sample phase and test phase). These conservative criteria are similar to that used by Oakes et al. (2013). Of the 240 trials completed, we excluded 52 trials that did not meet these criteria. Second, to ensure that we were evaluating infants’ shifts of gaze from the center of the display (i.e., from the sample) to the test items, for the first fixation analyses we included only trials in which infants were looking at the center of the screen for at least the first 75 ms of the test array. In addition, we included only fixation latencies greater than 200 ms, based on findings from previous research that adults’ eye movements in a similar paradigm require approximately 200 ms (Hyun, Woodman, Vogel, Hollingworth, & Luck, 2009). We excluded an additional 21 trials from the analyses of fixations for not meeting these criteria. Any infant who did not have at least 4 trials that met all these criteria were excluded from the final analyses—as indicated in the Participants section, six infants were excluded for this reason. Because our inclusion analyses were more conservative for first looks than for duration of looking, infants contributed slightly fewer trials (M = 8.35 trials, SD = 2.30) to the analyses of first looks than to analyses of looking time (M = 9.40 trials, SD = 2.28). We included trials that included looks to both items during test as well as trials that included looks to only one item during test. The processed data that were used for all the analyses reported here can be found at https://osf.io/5cusz/?view_only=fe9962def5b848289c86bbcbcb6ac1c7.

2.2. Results and Discussion

2.2.1. General Characteristics of Infants’ Looking

We begin by describing infants’ attentiveness in general by calculating their total looking time (i.e., the sum of all looking to the relevant AOIs on each trial). For each infant, we calculated the mean duration of looking during the sample and retention interval combined and during the test phase. Infants looked at the location of the sample item nearly the entire 800 ms period of the sample phase and retention interval, M = 699.90 ms (SD = 103.07 ms), indicating that they had sufficient time to encode the information presented in the sample array. Infants were similarly attentive to the items during test, accumulating 1112.52 ms (SD = 220.33 ms) of looking during this 2000 ms phase. Across the trials, during the test period infants made an average of 2.69 (SD = .49) individual fixations, with an average fixation duration of 444.71 ms (SD = 105.61).

2.2.2. Infants’ Looking During the Test Phase

Our primary question was whether infants’ eye movements during the test phase systematically reflected information that was encoded in VSTM during the brief sample phase. We calculated three measures to examine infants’ systematic preference for either of the two test items. First, we determined whether the first eye movement toward the test array (the “first look”) was directed toward the item that matched the sample or the non-matching item. We calculated a first-look preference score by dividing the number of trials in which the first look was directed to the non-matching item by the total number of trials with first looks to either test item (i.e., excluding trials in which the first look went to some other location, such as the edge of the monitor). Our second measure reflected whether infants spent more time fixating the matching or non-matching item. We computed a duration preference score by dividing the duration of looking to the non-matching item by the total amount of looking to both items combined during the test period. Finally, we computed for each infant the average duration of individual fixations to the match and non-match items on each trial.

Figure 2 shows the first-look preference score for each infant (open shapes), as well as the group mean (height of the bars). Because the test array contained two items, random responding would result in a preference score at the chance level of .50. However, on average .60 (SD = .13) of infants first looks were directed at the non-matching item, with the remaining first looks directed to the matching item. We compared the proportion of looks to the non-matching item to chance. Infants fixated this item more than expected by chance, t(19) = 3.38, p = .003, d = .76. These results provide strong evidence that (1) the infants stored the sample color in memory, and (2) this memory representation guided the infants’ allocation of attention to the search array.

Figure 2.

Proportion of first looks directed at the non-matching item during the test phase by Experiment. Mean proportions are indicated by bar height, and 95% confidence intervals are indicated by the error bars. Each open shape represents an individual infants’ proportion. Means that differed significantly from chance (.50) are indicated by **, p < .01.

Figure 3 shows the proportion of time spent looking at the non-matching item in each infant, along with the group mean. A mean proportion of .56 (SD = .07) of looking time during the test period was spent looking at the non-matching item; the remaining time was spent looking at the matching item. The proportion of time spent looking toward the non-matching item was significantly greater than expected from the chance level of .50, t(19) = 3.52, p = .002, d = .78. Thus, infants not only directed more of their first looks to the non-matching item, but they also spent more time looking at it. Note however that this effect might reflect the fact that infants looked first at the non-matching item more often—where infants looked first constrains how long they can look in total at the two items during our 2 s test period. To provide a purer measure of infants’ interest in the matching and non-matching item, we examined infants’ individual looks to each item items. On average, infants had 1.43 (SD = .35) individual fixation to the non-matching item and 1.26 (SD = .27) individual looks to the matching item per trial. Moreover, infants’ individual fixations to the non-matching item (M = 379.78 ms, SD = 114.12) were significantly longer than their individual fixations to the matching item (M = 284.79 ms, SD = 72.79), t (19) = 3.96, p = .001, d = .89 (see Figure 4). Thus, when infants fixated the non-matching items, they maintained those individual fixations longer than when they fixated the matching items, confirming that the non-matching item more effectively held their attention.

Figure 3.

Proportion of look durations (averaged across trials) directed at the novel (non-matching) item during the test phase by Experiment. Mean looking time is represented by the height of the bar, and the error bars represent 95% confidence intervals. Mean preference scores that significantly differ from chance are indicated by **, p ≤ .01. Each open shape represents an individual infant’s proportion.

Figure 4.

Average duration of individual fixations to the non-matching and matching items in Experiment 1 (left) and Experiment 2 (right). Mean looking time (averaged across infants) is represented by the height of the bar, and the error bars represent 95% confidence intervals; the average for each individual infant is represented by an open shape. For both experiments, individual fixations to the non-matching item were significantly longer than individual fixations to the matching item, ** p ≤ .01, * p ≤ .05.

3. Experiment 2

Experiment 2 was a conceptual replication and extension of Experiment 1. We conducted a replication because Experiment 1 provides the first evidence that infants can use information stored in VSTM to guide their visual search behavior, and the reproducibility of many psychological results has been called into question (Open Science Collaboration, 2015). It is therefore important to confirm that these results are robust and replicable. In addition, in Experiment 2 we used different visual stimuli (photographic images of common objects instead of colored circles) to show that the results of Experiment 1 are generalizable and do not rely on a specific stimulus set.

3.1. Method

3.1.1. Participants

Infants were recruited as in Experiment 1. Our goal was a final sample of 20 full-term infants who were free of vision problems or risk of colorblindness, and who had not participated in Experiment 1. We tested 26 infants and excluded 5 infants from our final sample due to fussiness (n = 1) or failing to complete the minimum number of trials (n = 4). The final sample included 21 10-month-old infants (M_age = 308.86 days, SD = 6.60 days; 13 boys). Sixteen infants were White, 2 infants were Asian, 2 infants were mixed race, and race was not reported for 1 infant. Seven infants were reported as Hispanic. All mothers had completed high school and 15 had earned at least a Bachelor’s degree. As in Experiment 1, all infants received a small gift as well as a certificate of appreciation for their participation.

3.1.2. Apparatus

All aspects of the apparatus were the same as Experiment 1 except that eye-gaze was collected by a SMI RED-m eye-tracker, recording at 120 Hz, and the software was run on a Dell laptop computer.

3.1.3. Stimuli

The stimuli were the same as those in Experiment 1, except as follows. We used 6 pairs of photographs of common, everyday objects (see Figure 5). To identify pairs of items for which infants had no a priori preferences, we collected preference data from a separate group of 13 10-month-old infants using the procedure and apparatus from Experiment 1. For these infants, the experimental trials involved the presentation of a single pair of items, located directly to the left and right of midline, for 2 s (so there was no sample phase). Infants’ looking to each item of each pair was recorded (on a total of 566 trials across the whole sample; an average of 43.54 different pairs of items per infant). On the basis of infants’ biases to look at one item versus the other, we selected six pairs of items for which infants had no strong preference for one item over the other (M = .55, SD = .03, range .50 to .59; all preferences were computed relative to the more preferred item in a pair for ease of comparison). The six item pairs were airplane and parakeet, apple and shoe, bowl and scissors, keys and wooden blocks, telephone and teakettle, clock and sunflower. The objects were on an average 4.5 cm tall by 4.5 cm wide (4.3° h × 4.3° w at a viewing distance of 60 cm).

Figure 5.

Pairs of stimuli and schematic illustration of an experimental trial sequence in Experiment 2. The timing was identical to the trials in Experiment 1.

We used these 6 pairs of items to create stimulus displays like those used in Experiment 1 (see Figure 5). As in Experiment 1, one item in each pair served as the sample stimulus in the first block, and the other item served as the sample stimulus in the second block. In addition, to increase the visibility of the detail in the objects, the objects were slightly bigger than the circles in Experiment 1. As a result, the distance between the center of each object and the center of the display was increased from 4.3° to 5.3°.

3.1.4. Procedure

The experiment began with SMI’s 5-point calibration procedure. The trials had the same sequence and timing as in Experiment 1. We constructed eight orders of 12 trials using the same constraints as Experiment 1. Within the blocks, the order of the individual trials was randomly determined for each infant and instrumental music clips accompanied each trial (e.g., Bach, Vivaldi). Data were processed in the same way as Experiment 1.

3.2. Results and Discussion

Using the same inclusion criteria as in Experiment 1 we excluded 19 of 252 trials that did not meet the inclusion criteria for minimum duration of looking in each phase of the trial (100 ms of looking in both sample phase and test phase), and we excluded 50 trials from the first looks analyses. Infants contributed an average of 8.71 trials (SD = 2.17) to the analyses of first looks, and 10.83 trials (SD = 1.22) to the novelty preference analyses. Examination of total looking revealed that infants were generally attentive during both the sample phase (M = 715.00 ms, SD = 70.08 ms) and the test phase (M = 1374.86 ms, SD = 200.46 ms). During the test phase, infants made an average of 3.26 (SD = .44) fixations, and individual fixation durations were 410.22 ms (SD = 107.30).

As in Experiment 1, our primary analysis examined the proportion of infants’ first looks that were directed to the non-matching item in the search display. As shown in Figure 2, the results were remarkably similar to those of Experiment 1. The proportion of infants’ first looks directed to the non-matching item was .60 (SD = 11.24), which was significantly greater than chance, t(20) = 4.00, p < .001, d = .87. Thus, as we observed in Experiment 1, infants directed significantly more of their first looks to an item that did not match the contents of VSTM than to an item that matched the contents of VSTM.

The proportion of time spent fixating the non-matching item is shown in the right half of Figure 3. On average, the proportion of time infants spent looking at the non-matching item was .55 (SD = .08). As in Experiment 1, this proportion was significantly greater than chance, t(20) = 2.77, p = .01, d = .60. Once again, not only did infants direct their first looks toward the novel item more often than toward the familiar item, they also spent more time looking at the novel item than at the familiar item. As in Experiment 1, the fact that the non-matching item better held infants’ attention was confirmed by the duration of infants’ individual fixations to each item (see Figure 4). Infants made on average 1.73 (SD = .37) fixations to the non-matching item and 1.54 (SD = .30) fixations to the matching items. The mean fixation duration for the non-matching item (361.33 ms, SD = 127.79 ms) was significantly longer than the mean fixation duration for the matching item (M = 302.46 ms, SD = 105.11 ms), t (21) = 2.34, p = .015, d = .51.

4. General Discussion

In both Experiment 1 and Experiment 2, infants saw a foveal sample stimulus for only 500 ms, and yet they rapidly encoded this stimulus into memory and used this information to control their looking behavior during a subsequent test array that was presented after a 300-ms delay. These data provide the first conclusive evidence that infants can use information briefly presented at one moment in time to control visual search behavior a few moments later. That is, not only did infants stored information rapidly in VSTM, they were able to use this VSTM representation to guide the targeting of the very next eye movement. Moreover, they were able to do this with both simple colored squares (Experiment 1) and realistic images of real-world objects (Experiment 2). This demonstrates that, by 10 months, the infant cognitive architecture is sufficiently sophisticated that it can perceive an object, store this object in VSTM, and then immediately use this information as an input to the oculomotor system.

Although prior research has conclusively established that extensive exposure to a stimulus will lead to a memory representation that can influence the amount of time infants spend looking at an object (Hunter, Ames, & Koopman, 1983; Hunter et al., 1982; Oakes & Kovack-Lesh, 2007; Roder, Bushnell, & Sasseville, 2000; Rose, 1981), the present study appears to be the first demonstration that infants can rapidly store information in memory and then immediately use this information to control the selection of the next saccade target. This ability to rapidly store and then utilize visual information requires far greater cognitive sophistication than would be required to explain previous findings of shorter looking times for objects that have been viewed for tens of seconds relative to novel objects. That is, the present result requires a cognitive architecture in which perceptual information can be rapidly “latched” into a persisting representation and then transferred into the saccade targeting systems of the posterior parietal cortex, frontal eye fields, and/or superior colliculus.

In an infant’s daily life, this sophisticated cognitive architecture would allow a single fixation in a scene to create a VSTM representation that could control the direction and duration of the very next fixation in that scene. This would presumably allow infants to guide their attention much more flexibly and dynamically as they explore scenes, with the information obtained in each brief fixation impacting their subsequent exploration of the scene. If it took ten or more seconds to form a memory representation or use this representation as a bias signal within the oculomotor control system, this could lead to very inefficient exploration of the environment.

Our results make two additional contributions to the literature. First, these data show that infants inhibit attending to a previously attended object identity, even when the object appeared in a new location. Many studies of attention in adults have shown how inhibition of irrelevant information is key to attentional selection (Olivers & Watson, 2006; Pylyshyn, Haladjian, King, & Reilly, 2008; Wühr & Frings, 2008). Moreover, Watson and Humphreys (Watson & Humphreys, 1997; Watson, Humphreys, & Olivers, 2003) argued that the visual system prioritizes new information over old information, in part, by inhibiting responding to the old information. In adults, there is some evidence that new objects capture attention and draw eye gaze (Brockmole & Henderson, 2005; Theeuwes, Kramer, Hahn, & Irwin, 1998).

Our finding that infants directed their visual attention away from the item that matched the information stored in VSTM, and instead directed their gaze to the non-matching information, is consistent with the argument that the attention system prioritizes new information. In addition, other studies with infants have revealed a similar type of preference (i.e., a preference for an item that differed from that stored in VSTM, Oakes et al., 2013). Work with human adults and non-human primates, however, have yielded conflicting findings. On the one hand, work with adult humans and adult monkeys have revealed that viewers are biased to attend to items matching the contents of VSTM in a subsequent visual search task (Chelazzi, Miller, Duncan, & Desimone, 1993; Hollingworth & Luck, 2009). But whether adults are drawn to or repelled from items that match the information being stored in VSTM depends on the context (Al-Aidroos, Emrich, Ferber, & Pratt, 2012; Sawaki & Luck, 2011; Woodman & Luck, 2007). For example, Sawaki and Luck (2011) observed that adults suppressed attention to items that were being held in VSTM as they performed a subsequent task. Sawaki and Luck proposed that this suppression could be a function of task difficulty, and that suppression under some circumstances may be beneficial in preventing the degradation of the representation of the task-relevant information in working memory. Given that we were unable to give infants instructions, it is unlikely that their systematic preference for novel items was driven by an internal strategy to suppress memory-matching items in order to prevent their representations of the sample items from degrading. Instead, it may reflect a general bias of infants toward maximizing the amount of information they acquire from the world.

Note that the inhibition for the previously seen item reported here is not the same as the inhibition observed in inhibition-of-return (IOR) tasks. In IOR tasks, the observer inhibits locations that have been previously attended (Klein, 2000). In the classic case, a location is cued and then a target appears in that location or in a different location. If the delay between the cue and the target is short, the response to the target is enhanced at the cued location, but if the delay is long (and participants have no incentive to maintain attention at the cued location), the response to the target is impaired at the cued location. This has been interpreted as a mechanism that prevents attention from becoming stuck on a single location and encourages attention to shift toward new sources of information. IOR has also been observed in infants. For example, Johnson and Tucker (1996) observed that 4- and 6-month-old infants were more likely to orient toward a target at a cued location than a target at an uncued location when the target appeared 200 ms after the cue, but this reversed when the target appeared 700 ms after the cue. Thus, both the present results and IOR are examples of a preference for novel sources of information.

However, although both our results and IOR reflect inhibition of old information, IOR involves inhibition of a familiar location and our task involved infants inhibiting an object that matched the identity of a previously encoded object. Because the sample item and the matching test item were presented at different spatial locations in the present paradigm, the bias to look at the novel item cannot reflect inhibition of attention to a particular location. Although there is evidence of color-based IOR in adults, the inhibition effect for color is considerably smaller than the inhibition effect for spatial locations (Law, Pratt, & Abrams, 1995). Additionally, IOR for color in adults could only be induced under very specific conditions and involved masking the spatial location of the cue before presenting the target. Because our task did not include a mask, it seems unlikely that our effects reflect IOR for color.

The second major contribution of the present study is that it demonstrates that infants’ VSTM can be used to control other cognitive operations, in this case eye movements. One issue that has been raised about VSTM is whether it is a working memory system. According to one view, a memory system is a working memory if the stored information can be used in the service of other cognitive functions (Baddeley, 1986; Luck & Vogel, 2013). Because infants used the information stored in VSTM to guide eye-movements or attention, the present study suggests that VSTM in infancy may be a working memory system. This is just a first step, and additional research is needed to determine if infant VSTM is used as a buffer that is used to control the operation of other cognitive processes.

In summary, these results further our understanding of the VSTM system in infancy in two key ways. First, they indicate that information about both simple and complex objects that is stored in VSTM at one moment in time can have an impact on where and when an infant’s eyes move just a few moments later. Second, they indicate that infant VSTM is a working memory system, in which information is stored rapidly in memory and impacts the operation of other cognitive operations. Previous research demonstrates that VSTM is present during infancy beginning as early as 4 months (Ross-Sheehy et al., 2003), and it will be important for future studies to determine the developmental time course of the use of VSTM to control gaze and operate as a working memory system.