Cues to individuation facilitate 6-month-old infants’ visual short-term memory

Abstract

Infants’ ability to perform visual short-term memory (VSTM) tasks develops rapidly between 6 and 8 months. Here we tested the hypothesis that infants’ VSTM performance is influenced by their ability to individuate simultaneously presented objects. We used a one-shot change detection task to ask whether 6-month-old infants (N = 47) would detect a change in the color of one item in a two-item array when the stimulus context facilitated individuation of the items. In Experiment 1, the two items in the display differed in shape and color, and in Experiment 2 the onset and offset times of the two items differed. In both experiments, 6-month-old infants detected a change, contrasting with previous results. Thus, young infants’ encoding of information about individual items in multiple-item arrays is related to their ability to individuate those items.

Visual short-term memory (VSTM) provides a mechanism for rapidly forming representations of visual inputs and is used to maintain continuity of perception across eye-movements and blinks (Hollingworth & Henderson, 2002; Irwin, 1991; Luck & Vogel, 1997). Without VSTM, the visual world would appear as a disconnected series of snapshots. Young infants can encode and retain information about a single item presented in isolation for hundreds of milliseconds (Catherwood, 1994; Catherwood et al., 1996), whereas their VSTM for the individual items in multiple-item arrays is severely limited (Ross-Sheehy et al., 2003). Specifically, in change detection tasks 4- and 6-month-old infants show evidence of being able to store in VSTM a single item presented in isolation, but not the individual items in arrays of two or three items (Oakes et al., 2013; Ross-Sheehy et al., 2003). In fact, in a change detection task 6-month-old infants failed to notice when every item in a 3-item array changed color (e.g. an array of red, green, and yellow to an array of blue, green and yellow; Oakes et al., 2009), although they did detect changes in 3-item arrays if all the items were the same color (e.g., three red items followed by three green items; Oakes et al., 2006). Thus, the observed failures do not reflect an inability to encode anything at all. Rather, these results taken together suggest that young infants do not encode and remember features of the individual items in these multiple-object arrays.

The implications of this limitation are profound. Consider how difficult it would be to see complex real-world scenes as continuous across saccades and eye blinks if one was unable to form separate representations of the individual items in those scenes. Although global characteristics such as spatial configuration or average hue may be remembered, it would be impossible to keep in mind the features that differentiate objects and people in those scenes. However, this characterization of infants’ VSTM conflicts with other evidence that in the first postnatal weeks and months infants are adept at some abilities that may require VSTM, such as planning and executing eye movements (Aslin & Salapatek, 1975; Hainline et al., 1984; Hood et al., 1992; Richards & Hunter, 1997), tracking objects as they move in and out of occlusion (e.g., Nelson, 1971; van der Meer et al., 1994), and remembering the number of items in a scene (Feigenson, Carey, & Spelke, 2002; Wynn, 1992). How do we reconcile infants’ apparently limited VSTM (as measured in change detection tasks, which are widely used to measure VSTM in adults) with the fact that they are adept at many other tasks that (although they do not test VSTM directly) seem to require VSTM?

One key aspect of vision is the individuation of visual objects. Individuation is the creation of a set of discrete entities from the spatially distributed pattern of light and color that strikes the eyes (Cavanagh, 2011). For example, as you are reading this page, your visual system is breaking up the spatially continuous visual input into a discrete set of letters and words. The ability to separate each item in a scene from the other items is critical for higher-level processes that operate on individuated objects, such as enumeration, object recognition, and visual short-term memory (Y. Xu & Chun, 2009). The typical laboratory VSTM tasks used with adults requires this type of individuation. In these tasks there is a brief (e.g., 100–500 ms) display of an array of items, followed by a short retention period, and finally the reappearance of the array with the properties of one or more items changed (e.g., Alvarez & Cavanagh, 2004; Awh et al., 2007; Luck & Vogel, 1997). The logic is that if participants detect the change, they must have stored the visual information and retained it in memory over the brief retention period. Often the arrays consist of multiple simple objects that vary in only one dimension (e.g., color). Encoding and retaining information about the individual items in such array requires that observers first must rapidly individuate those items (Y. Xu & Chun, 2009). If, for example, a yellow item and a red item were not individuated, this might lead to memory for an orange item. Although it may be trivial for the adult visual system to decompose each display into multiple discrete individual items (e.g., a yellow square, a red square, and a brown square), the infant visual system may have more difficulty parsing such displays into separate objects. Thus, failures of young infants to detect changes in VSTM tasks may be a result of their inability to individuate the items in the scenes rather than a lack of VSTM storage capacity per se (or a combination of reduced capacity and inability to individuate).

Individuation may be especially difficult for the kinds of stimuli that have typically been used in laboratory studies of VSTM—arrays in which the items vary along only one dimension (e.g., color). Although in these arrays spatiotemporal information could indicate individual items (e.g., the individual items are in separate locations), Feigenson (2005) demonstrated infants have difficulty simultaneously individuating items in such arrays. Further, in real-world scenes, objects typically vary along multiple dimensions, not just one (e.g., color, shape, size, texture, orientation) and they move along independent trajectories, appearing or disappearing from a visual scene at different times; such variation may provide the visual system with multiple cues to individuate the items within a scene. By eliminating inter-item variation and temporal cues in an attempt to simplify and control the stimuli in VSTM experiments, we may make it more difficult for infants to rapidly individuate the items, and as a result to attend to, process, and encode the information about individual items.

Infants’ ability to individuate items in dynamic visual displays and events has been extensively studied (e.g., Carey & Xu, 2001; Leslie et al., 1998; Wilcox & Baillargeon, 1998; Wilcox & Schweinle, 2002; F. Xu & Carey, 1996). The individuation in these studies differs in important ways from the rapid multiple simultaneous individuation described in the previous paragraph. Specifically, the work on infants’ individuation has used tasks in which objects are shown one at a time as they move behind and are revealed from occluders. The events last tens of seconds, and each object is visible—typically one at a time—for many seconds. In these experiments, after watching these dynamic events, infants’ memory for the number or kind of objects is tested. This work broadly indicates that infants can individuate items when they move independently of one another in time and space (Kaldy & Blaser, 2013; Kibbe et al., 2011; F. Xu & Carey, 1996), and that individuation develops prior to the ability to encode individual object properties (Kibbe et al., 2011; Leslie et al., 1998). Little work has been conducted on infants’ parallel individuation of multiple simultaneously presented items, however. In one of the few reported studies, Krojgaard (2007) observed that young infants have difficulty individuating highly similar items presented simultaneously.

In the change detection tasks that assess VSTM, individuation refers to the rapid, parallel recognition of multiple independent entities. This kind of individuation of simultaneously presented items plays a role in adults’ VSTM (Y. Xu, 2009; Y. Xu & Chun, 2009). For example, evidence from adults’ enumeration of items in a visual array—an ability that is closely related to VSTM (Piazza et al., 2002; Y. Xu & Chun, 2009)—suggests that limits on visual processing reflect a fixed limit on the number of discrete items that can be apprehended simultaneously (Ester et al., 2012). This type of individuation is required for infants’ detection of the number of items in an array, especially for small numbers. Some have argued that the ability to track objects as individuals—and the limits on that ability—underlies infants’ sensitivity to small numbers (Carey & Xu, 2001; Feigenson, Carey, & Hauser, 2002; Feigenson & Carey, 2003). To test this, Feigenson (2005) showed that 6-month-old infants attended to number in arrays of fully visible items only if the items could be individuated on multiple features; when the items were identical, infants did not attend to number.

The present study tested the hypothesis that the poor performance of young infants in VSTM tasks involving multiple objects is a result, at least in part, of limitations in their ability to individuate the items in these multiple-item displays. If their performance is limited by their ability to individuate, then we should be able to improve their performance by making the individuation process easier. Thus, we tested whether we can facilitate 6-month-old infants’ visual short-term memory of multiple-object scenes by providing additional cues to individuation.

Previous evidence indirectly supports this possibility. First, 6-month-old infants fail to prefer changing stimulus streams in which the colors of all the items in an array of three differently colored items changed each time it reappeared (e.g., red, green, black, then changes to blue, orange, purple, then changes to red, brown, yellow) (Oakes et al., 2009). This failure is striking because all the items changed; thus, if infants had attended to, encoded, and remembered even one of the items, they should have detected that a change has occurred. However, this would have required first individuating the three items. Second, Ross-Sheehy, Oakes, and Luck (2011) presented 5.5-month-old infants. Specifically they tested infants’ detection of changes in streams involving 3-item arrays in which on each reappearance one item changed and in each array one of the items continuously rotated throughout the trial. Infants looked longer at streams in which the rotating item changed color on each cycle than at streams in which one of the nonrotating item changed color (another, non-changing item, rotated in these arrays). Presumably, the rotation of the item helped infants individuate that item (and selectively attend to it, store it in VSTM, etc.). Thus, they were able to detect changes in the color of the rotating item.

Finally, at this age infants do detect changes in multiple-item arrays when the need to individuate is minimized. For example, 6-month-old infants show a preference for changing streams over nonchanging streams when all the items in a given display are the same color and all items change to a new color on each cycle (e.g., 3 green objects then 3 red objects then 3 brown objects; Oakes et al., 2006). In addition, infants at this age respond to changes in the spatial configuration of 3 items in an array, which involves linking rather than individuating the items (Oakes et al., 2011). Thus, young infants can succeed at change detection with multiple-item arrays when they need not individuate the items to do so.

The fact that young infants succeed in change detection when one item is made salient (Ross-Sheehy et al., 2011) may help explain why infants are able to function in the real world—a world in which important items are often salient and multiple cues are typically available to individuate different objects. In particular, the objects in real-world scenes often differ from each other along multiple dimensions (e.g., color, shape, size, movement trajectories), and these differences should make them easier to individuate than the highly similar objects used in most change detection experiments. Similarly, real-world objects often differ from each other temporally, providing further support for recognizing them as separate entities. We asked here how 6-month-old infants performed in VSTM change detection tasks when the items within each array differed from each other on multiple dimensions, rather than just one (Experiment 1), or if the onset and offset times of the objects differed from each other to minimize temporal grouping (Experiment 2).

Experiment 1

In Experiment 1, we asked whether 6-month-old infants’ detection of changes in multiple-item arrays would be facilitated by increasing the distinctiveness between the items within each array. Previous research shows that infants are sensitive to this type of manipulation. For example, Feigenson (2005) found that 7-month-old infants could simultaneously individuate items that differed in multiple features more easily than highly similar items. In the present experiment, we tested 6-month-old infants’ VSTM for color in multiple-item arrays in which the items differed on color and shape, rather than color alone. We varied item shape (rather than size, for example) because multiple studies have shown infants are sensitivity to shape—particularly in the context of individuation—before other visual features (see Krojgaard, 2004, for a review).

Method

The method and procedure used here were approved by the Institutional Review Board of the University of California, Davis under the protocol “Understanding cognitive development in infancy: Attention and visual short-term memory,” (protocol number 220219–49).

Participants

We recruited healthy, full-term, typically developing 6-month-old infants for both experiments reported here. Infants were not eligible if they were born preterm (e.g., more than 21 days before the due date), had any known neurological or vision problems, or had a family history that put them at significant risk for colorblindness (male infants with maternal familial colorblindness, and female infants both with maternal familial colorblindness and whose father was colorblind). Each infant participated in only one of the two experiments. In each experiment, our target sample size was 24 infants. This target was selected using rule-of-thumb, and was based on the sample sizes used by Oakes et al. (2013). However, post-hoc power analyses using G*Power (Faul et al., 2007) indicated that a sample of 24 infants was sufficient to provide over 90% power to detect above-chance change-detection performance with the effect sizes observed here (in which above-chance performance was observed). In other words, if change detection performance is not significant with this sample size, and we therefore accept the null hypothesis, there is only a 10% chance that this is a Type II error (assuming that the true effect size is comparable to those we observed). Although post-hoc power analyses are potentially problematic, our target sample size also provides sufficient power to detect the effect sizes previously observed with 6-month-old infants in this task (Oakes et al., 2013).

To achieve our target sample size in Experiment 1, we tested 25 infants who met our inclusion criteria (described earlier). Our final sample included 24 infants (Mean age = 181 days, Range = 170 days – 196 days; 15 male infants); one of the tested infants became fussy and did not finish the session. Of the 24 infants in the final sample, 16 were Caucasian, 3 were Asian, 1 was Black or African American, 2 were mixed race, and 2 infants’ race was not reported. Five infants were reported to be Hispanic (2 Caucasian infant, 1 Black or African American infant, 1 mixed race infant, and 1 infant whose race was not reported). One mother had completed 8^th grade; the other 23 mothers had completed some college and 18 of those mothers had completed at least a Bachelor’s degree.

We obtained recorded births from the State Department of Health and sent informational packets to families who lived within 30 miles of our lab. Families indicated they were interested in participating in studies in our lab via email, phone, or by returning a letter; when infants approached the appropriate age for this study, we contacted families inviting them to participate. For participating, infants received a book, toy, or shirt and a certificate of appreciation.

Apparatus

In both experiments, eye movement data were collected using an Applied Sciences Laboratory (ASL) pan/tilt D6 eye tracker fitted with a magnetic head tracker (Ascension Flock of Birds) and controlled by a Dell computer. The eye camera for the tracker was positioned in front of and just below a 37” Westinghouse LCD monitor (16:9 aspect ratio) on which stimuli were presented during the experimental session. The ASL eye camera was focused on the infant’s right eye, and a separate wide-angle camera directed at the infant allowed the experimenters to monitor the infant’s general behavior. The ASL software determined the point of gaze (POG) using the recorded pupil and corneal reflections obtained from an infrared light source. The magnetic head tracker was positioned just above the infant’s right eye, affixed to a soft cotton infant-sized headband. The eye tracking system used information about the head sensor’s location (and thus the infant’s right eye) to quickly redirect the eye camera if the infant moved outside the eye camera’s range. A separate Dell computer was used to present stimuli on the Westinghouse monitor and to send stimulus information to the computer recording the eye gaze data.

Procedure

The procedure was similar to that of Oakes et al (2013). Infants were seated in a caregiver’s lap approximately 100 cm from the screen and approximately 75 cm from the eye camera and infrared light source. Once seated, infants were fitted with the soft headband with the head sensor attached, and an experimenter adjusted the eye camera to obtain a stable image of the infant’s right eye. Once the headband was adjusted, the experimenter moved behind a curtained wall. The sessions were controlled by two experimenters seated out of sight behind the curtained wall: an eye tracking experimenter, who interacted with the ASL software that controlled the eye tracker and recorded the data, and a stimulus-presentation experimenter, who interacted with the software that controlled stimulus presentation on a second computer. The stimuli were presented using custom programs created in Adobe Director, which both presented the stimulus and sent event codes to the eye tracking computer to be stored with the eye gaze data.

First, each infant’s POG was calibrated with a five-point calibration routine (left upper, right upper, center, left lower, and right lower). The stimulus-presentation experimenter presented a looming dot that changed colors and was accompanied by sounds (e.g. beeping sounds or ringing sounds) at each of 5 points on the screen (~8.2° above and ~8.2° to the left of the center of the screen and ~8.2° above and 8.2° to the right, one in the center of the center of the screen and then two at the bottom—8.2° below and ~8.2° to the left and right). The eye tracking experimenter monitored the stability of the pupil and corneal reflections during this phase and indicated to the ASL eye tracking system (via a keypress) when the infant was fixating the calibration stimulus. The quality of calibration was visually verified by presenting stimuli at the calibration locations and confirming that the cross-hairs (indicating the infants’ POG) was centered on the dots. If the calibration was judged to be poor (e.g., the center of the cross hairs was consistently above the verification stimuli), the calibration procedure was repeated.

Immediately after calibration, the stimulus-presentation experimenter initiated the experimental trials. Between trials, an attention-getting stimulus was presented in the center of the screen. These stimuli were pictures of toys (e.g., a stuffed horse, a rattle) that loomed from 0° high by 0° wide to 16° high by 16° wide (at a viewing distance of 60 cm) and were accompanied by sounds (e.g., beeps, rings). When the stimulus-presentation experimenter determined the infant was fixating that attention getter (e.g., the crosshair indicating POG was centered on that item), he or she initiated an experimental trial by pressing a key on the stimulus-presentation computer. Each experimental trial was a one-shot change detection trial (see Oakes et al., 2013) and consisted of three phases: a sample phase in which two items were displayed for 500 ms, a retention phase in which the items disappeared and only a blank screen was visible for 300 ms, and a post-change phase in which the items reappeared in their original locations on the screen for 2000 ms (see Figure 1; examples of sessions and all the data for both experiments reported here can be found at https://osf.io/t8x4z/?view_only=202a63c8be4446cdb5497d91000e418b). These parameters and stimuli were identical to those used previously by Oakes et al. (2013) with the exception that in the previous study the post-change phase was 3000 ms. We shortened this phase because Oakes et al. found that infants rarely looked at the items for longer than 2000 ms (and their primary analyses were based on infants’ looking in those first 2000 ms); moreover, the shorter trial durations allowed us to shorten the session and/or present more trials per infant.

Figure 1.

Depiction of a single trial in Experiment 1 (left) and observed Change Preference Scores (right). Each circle in the right panel represents the median preference score from a single infant; the individual circles are scaled to the number of trials completed and values are scattered horizontally solely to make the individual circles easier to see. The black line within the cloud of circles indicates the mean of the preference scores, with the 95% confidence interval shown by error bars. * = p >. 05 for one-sample t test against chance.

On each trial, our custom program selected two colors at random from a pool of 8 colors: red, teal, black, yellow, orange, green, purple, and brown. In addition, our program selected two shapes at random from a pool of 8 shapes: square, circle, diamond, star, cross, triangle, crescent and hexagon (all subtending 6.4° high by 7.9° wide). For each array, the two items were centered 11.4° from the center of the screen to the left and right of the midline such that items were 22.6° from one another (center to center of the items). All items were presented on a grey background. As is common in these procedures, music accompanied each trial to maintain interest (Hurley & Oakes, 2015; Oakes et al., 2017).

During the sample phase, two items appeared (e.g., the black circle and white diamond in the sample phase depicted in Figure 1). During the post-change phase, the color of one item changed relative to the original sample display; this item was the changed item. The color of the other item remained the same as the sample and was the unchanged item (e.g., the black circle the post-change phase in Figure 1). Across trials, the side on which the changed item occurred was randomized so that changes occurred approximately equally often on the right and left. Trials continued until 64 had been completed or until the infant was too fussy or uninterested to continue (e.g., the infant was crying, refused to look at the monitor, turned to look at the parent).

If necessary, the stimulus presentation experimenter could present clips from child-oriented animated shows (e.g., Blues Clues, Mary Poppins) between trials to reorient infants to the screen when they looked away. These videos provided the infant a break from the experiment or allowed them to refocus towards the screen.

Data processing

The ASL system recorded the X and Y coordinates of the infants’ POG at a rate of 60 Hz, along with event codes sent via the presentation computer’s parallel port to the eye tracker control unit to indicate display information. We used an on-line filter, such that the X and Y coordinates for each recorded sample reflected an average of the current sample and the three previous samples. A blink filter was implemented that assumed that pupil loss of 12 or fewer samples was a blink. There was a slight lag between when the stimulus appeared on the Westinghouse monitor and when the event code was registered and recorded. As part of our data processing, these two signals were synchronized. As a result, we had eye gaze data recorded for 1917 ms after the onset of the change, rather than for the full 2000 ms.

We used a custom Matlab toolbox to aggregate looking samples that occurred within three areas of interest (AOIs): two AOIs (14.25° by 19.2°) for the individual items in the arrays and a third AOI (5.7° by 8.5°) for the central region where the attention-getter was presented. Spatial accuracy in eye tracking with infants and young children is highly variable (Dalrymple et al., 2018; Frank et al., 2012; Morgante et al., 2012), and our large AOIs help deal with both spatial imprecision in the eye tracker and calibration drift. Our inclusion criteria were the same as in Oakes et al. (2013): we included individual trials if the infant looked at least (1) 100 ms during the 800 ms combined sample and retention phases (to all three AOIs combined), and (2) 200 ms total during the post-change analysis window to the two AOIs. Our post-change analysis window began 200 ms after the onset of the changed stimulus and ended at 1917 ms after the change in Experiment 1 (see below for how this analysis window was determined). Using these criteria, we excluded 143 trials, and 1165 trials were included in the final analyses. We included the data from individual infants if they had at least 4 trials that met these criteria. All infants contributed at least 4 trials except for the one infant whose session was terminated for fussiness.

Analysis Windows

There were two analysis windows, which can be seen in Figure 2: 1) a preference score analysis window that was used to investigate the proportion of looking to the changed item during the post-change phase, and 2) a time-course analysis window that was created to investigate infants’ moment-to-moment looking at each of the two items over time starting from before the change (during the sample period) and running through the post-change phase. Oakes et al. (2013), observed that infants typically did not look for the entire 3-s post-change phase. Thus, for both experiments, following Oakes, et al., the change preference score analysis window only included the portion of the post-change phase in which at least 50% of all trials (across all infants) contributed data (i.e., infants were looking at one or the other item-based AOIs). We used this criterion to ensure that our results reflect responding by most infants on most trials; samples based on less than 50% of trials may be biased toward a subset of infants who looked longer on each trial. In this experiment, as illustrated in Figure 2, more than 50% of the trials across all infants had recorded data for the full trial length (our data recording ended at 1917 ms due to the adjustments for differences in the timing of event codes and stimulus presentation described earlier). Thus, our analysis window in this experiment extended until the end of the trial. Because planning a saccade typically takes at least 200 ms in tasks that require participants to make a match between a stimulus and memory (Hyun et al., 2009), we used 200 ms after the onset of the post-change phase as the starting point of our change preference analysis window. We thus set our change preference window to include data beginning at 200 ms and running until 1917 ms after post-change onset.

Figure 2.

Proportion of trials across all infants in Experiment 1 which gaze was directed to one of the two objects at each sample point (and therefore contributed data that could be used to assess the preference for the changed object). The horizontal light dotted line indicates the 50% cut-off threshold for trials. The vertical light dotted line indicates the point in the trial where the stimulus presentation changes (post-change phase). The solid dark line reflects the proportion of trials with observed data. The thick broken lines indicate the first and last points at which at least 50% of all trials had fixations recorded to one or the other item (the other trials had fixations directed to other locations on the screen or off the screen altogether). These lines also indicate the analysis window for the Monte Carlo time-course analysis (see text). The analysis window for the preference score is shown in the shaded area, which begins 200 ms after the post-change onset and ends at the end of the analysis window (1917 ms).

The second analysis window—the time-course analysis window—was used to evaluate infants’ looking during the period beginning during the sample phase when at least 50% of all trials contributed to the data, through the retention and post-change periods, ending at 1917 ms. This window can be seen in Figure 2. Inspection of this figure showed that this window started at −383.33 ms (before the onset of the post-change phase array) and lasted until 1917 ms after the post-change array onset.

Results

Infants completed 48 trials on average (range 11 to 64 trials). Infants looked at one of the three AOIs (left item, right item, or center) on average during the sample and retention phases combined for 700.68 ms of the possible 800 ms (SD = 45.42); thus, infants were interested in the sample array and looked sufficiently long enough to encode information about color (Catherwood et al., 1996). Infants looked on average at one of the two AOIs during the post-change phase for 1175.31 ms (SD = 238.6) out of the possible 1917 ms.

Our primary analyses examined whether infants preferred to look at the changed item during the post-change phase. For each trial, we calculated a change preference score during the change preference score analysis window by dividing the total time spent looking at the AOI for the changed item by the total duration spent looking at the AOIs for the changed and unchanged items combined. A change preference score greater than .50 would indicate that infants looked longer at the changed than at the unchanged item. We determined the median change preference score across trials for each infant; the median is less influenced by extreme values, thus minimizing the effect of trials in which infants looked at only one item or the other. Figure 1 shows these individual median scores (indicated by circles) and the overall group mean of these median scores (indicated by the solid black line). The mean of the median scores across infants was .60 (SD = .15), which was significantly different from chance, t(23) = 3.24, p = .004, d = .66. Further, 20 of the 24 infants had a median score above .50, and this number was significantly different from the number that would be expected by chance (binomial test, p < .05). In Figure 1, the symbol showing the value for each infant is scaled by the number of trials completed (larger circles indicate more trials), and it is clear that there was no systematic relation between infants’ median change preference score and the number of trials completed, r = .16, n = 24. Moreover, the results are the same whether or not we use the full set of scores for each infant, or if we only analyzed the first N trials (20, 30, or 40). Thus, our results are not biased by some infants becoming more sensitive to the change over trials.

We next evaluated how infants’ preference for the changed item varied over time within the trial (i.e., during the post-change period). Infants may be initially attracted to the changed item, but the preference is short-lived and therefore an overall preference is not observed when aggregating looking across the entire trial (i.e., infants may typically shift first to the changed item but only stay there for a few hundred milliseconds before returning to no preference). To investigate this possibility, we conducted in a time-course analysis. Specifically, we coded each infant’s looking during each 16.67 time sample in the period from 383.33 ms before the change occurred (as determined by the 50% cutoff described earlier and seen in Figure 2) to the end of the post-change analysis window; the sample was coded as a 1 if during that sample the infant’s gaze was focused on the changed item and as a 0 if during that sample the infant’s gaze was focused on the unchanged item. No score was given for looking at other regions of the screen. This was done for each trial for each infant.

For each infant, we then calculated an average preference score for each sample period to obtain the single-infant time course of the change preference score. These single-infant preference scores were then averaged across infants at each time point. Figure 3 shows the resulting infant-weighted mean preference time course. Time points in which the line falls above .5 indicate that more looking was directed at the changed item at that point, whereas time points in which the line is below .5 indicate more looking was directed at the unchanged item at that point. Error bars indicate 95% confidence intervals.

Figure 3.

Preference for the changed item at each time point as a function of time relative to the onset of post-change phase in Experiment 1. The solid graphed line represents the change preference at each time point, and the vertical bars represent the 95% confidence interval at each of those time points. Each time point that was significantly greater than chance (without correction for multiple comparisons) is indicated by an asterisk. The set of asterisks from 566.7 to 1066.7 ms formed a cluster that was statistically significant after correction for multiple comparisons as determined by our permutation analyses (see text for details).

As illustrated in Figure 3, infants showed a preference for the changed item from 566.7 to 1066.7 ms after the test phase onset. To address whether this preference was above chance, we conducted a cluster length analysis using a permutation-based control for multiple comparisons (Oakes et al., 2013, and similar approaches in Bullmore et al., 1999; Hayasaka & Nichols, 2004; Nichols & Holmes, 2001). In this approach, single-sample t-tests are first used to find individual times points with a significant preference for the changed item without correction for multiple comparisons. Because eye position tends to remain constant for a period of time before moving, the next step was to find clusters of individually significant t values and determine the duration of each such cluster. A nonparametric permutation test is then used to determine whether any of the runs are significantly longer than would be expected by chance.

The permutation test uses random permutations of the data to estimate the distribution of scores (cluster lengths) that would be obtained if the null hypothesis were true. To simulate the null hypothesis being true, one item in the 2-item array is randomly assigned as the changed item on each iteration of the resampling (thus, by chance, half of the time the changed item will be assigned as changed and half of the time it will be assigned as unchanged). These permutations are each analyzed in the same way as the actual data, and the t-tests comparisons to chance are calculated at every time sample. This process was repeated 1000 times, and we generated a distribution of runs of consecutive samples with significant t-tests. The distribution of the longest length of runs of successive significant samples for each resampling can be seen in Figure 4. If the run length obtained in the actual data falls in the top 5% of this distribution, it is considered significant. Analyzing the group average at each sample period yielded 32 samples that were significantly above chance, 28 of which were consecutive— much greater than the 16 samples calculated as the 95% cutoff. Thus, the preference scores for the 28 time points from 566.7 to 1066.7 ms can be considered to be significantly greater than expected by chance after correction for multiple comparisons.

Figure 4.

Distribution of run lengths obtained from 1000 permutations of the data in Experiment 1. This is the distribution of values that would be expected by chance (the null distribution). The dotted line indicates the 95% cutoff (i.e., 95% of the permutation iterations had run lengths shorter than this). We considered any observed run length from the actual data in Experiment 1 above this cutoff as statistically significant.

Discussion

In summary, the change preference score—whether aggregated over the post-change analysis window or analyzed at each time sample—provided robust evidence that infants detected the change in the array of two items. This finding is critical to our understanding of the development of VSTM as it is the first to show a context in which 6-month-old infants unambiguously detect changes to multiple-item arrays presented rapidly in a change detection task. These data suggest that increasing the differences between items helps young infants attend to the individual items, process features of individual items, and encode in VSTM the features of at least one item in the multi-item array. Similar results were observed by Feigenson (2005) for infants’ attention to the numerosity of small-set arrays. She observed that when 7-month-old infants are shown arrays of 1 or 2 objects that varied in color, pattern, and texture, infants responded to changes in numerosity and not continuous extent. This contrasted with previous findings in which infants responded to continuous extent but not numerosity when shown arrays of 1 to 3 identical items (e.g., black dots or squares Clearfield & Mix, 1999, 2001; Feigenson, Carey, & Spelke, 2002)

Feigenson (2005) argued that heterogeneity of object properties in such displays may promote the creation of object representations and/or the formation of memory representations for individual objects. We suggest 6-month-old infants in the present experiment successfully detected a change in one of the items in the two-item arrays of Experiment 1 because the items differed both in color and shape, thus facilitating young infants’ rapid individuation of the two items in the arrays.

Experiment 2

Experiment 2 had two goals. First, we sought to conduct a conceptual replication of Experiment 1. Because most previous studies report that 6-month-old infants fail to detect a change when arrays contain two or more items (Oakes et al., 2006, 2009, 2013; Ross-Sheehy et al., 2003), it is important to be certain that the success reported here is a generalizable result. Second, we sought to examine the effectiveness of temporal cues to individuation. When objects move separately in space and time, the cognitive system uses this information to individuate the objects (Flombaum & Scholl, 2006). For example, Sekuler and Bennett (2001) found that elements of an image that followed the same luminance time course tended to be grouped together, whereas elements that had different time courses tended to be segregated/individuated. Therefore, we asked in Experiment 2 whether giving the two objects in a display different time courses (i.e., slightly different onset and offset times) would increase infants’ ability to individuate the objects and detect changes in object color between the sample and test displays.

Infants were presented with a task similar to Experiment 1, except that the items in these arrays were the same shape and we facilitated individuation by having the onset and offset of one object delayed by 100 ms relative to the onset and offset of the other object. As a result, the two items had different time courses (as in the study by Sekuler & Bennett, 2001). We expect that any temporal cue or cues that indicate independent movement could support individuation; however, onset time was chosen as a first step to addressing the role of temporal-based individuation in supporting VSTM.

Method

Participants

The final sample consisted of 23 infants (mean age = 183.6 days, range = 171–195 days; 12 boys). Thirteen infants were Caucasian, 4 infants were Asian or Pacific Islander, and 5 infants were of mixed race (the race of one infant was unreported). Six infants were Hispanic. All mothers had completed high school and 15 had obtained at least a Bachelor’s degree. All tested infants were included in the analyses.

Apparatus, Procedure, Stimuli, and Design

The experiment was identical to Experiment 1 with the following exceptions. As depicted in Figure 5, each array consisted of two different colored rectangles, and the two items in the sample array were presented with a 100 ms timing difference. In the sample phase, one item appeared alone and remained visible for 100 ms before the other item appeared; the two items remained visible together for 400 ms; then the first item disappeared and the second item was visible alone for 100 ms. The second item then disappeared, which defined the beginning of the 300 ms retention period during which the screen was blank. Each individual item remained on the screen for only 500 ms, but the timing difference of the stimuli meant that the sample phase lasted a total of 600 ms. The left- and right-side items were equally likely to onset first, and the changed item was equally likely to be the item that appeared first or the item that appeared second. The items appeared and disappeared simultaneously during the post-change phase. As in Experiments 1, trials continued until 64 trials had been completed or until the infant was too fussy or uninterested to continue. The data were processed as in Experiment 1. In this experiment, the adjustment for the lag between when the stimulus appeared and the recording of event codes meant we had recorded eye gaze data for 1950 ms after the onset of the change. We excluded 132 trials; the final sample included 670 trials across infants.

Figure 5.

Depiction of a single trial in Experiment 2 (left) and observed Change Preference Scores (right). Each circle in right panel represents the median preference score from a single infant; the individual values are scaled to reflect the number of trials completed and are scattered horizontally to make the individual circles easier to see. The black line within the cloud of circles indicates the mean of the preference scores, with the 95% confidence interval shown by error bars. * = p , >05 for one-sample t test against chance.

Results

Infants included in the analyses completed an average of 29 trials (range: 7–60 trials). During the sample phase, infants looked at one of the three AOIs on average for 780.28 ms (SD = 49.50) of the possible 900 ms. We calculated a window of analysis for the change preference score as well as the time-course analyses using the same procedure as in Experiment 1. Figure 6 shows the points at which less than 50% of trials contributed to the data (and thus were used as the cutoff points for the analysis windows). For Experiment 2, the point at which 50% of all trials contributed began at 416.67 ms before the onset of the post-change phase and ended 1650 ms after the onset of the post-change stimulus. On average, infants looked to one of the three AOIs during the analysis window of the post-change phase for 957.45 ms (SD = 195.36) of the possible 1650 ms.

Figure 6.

Proportion of trials across all infants in Experiment 2 which gaze was directed to one of the two objects at each sample point (and therefore contributed data that could be used to assess the preference for the changed object). The lighter horizontal dotted line indicate the 50% cut-off. The vertical dotted line indicates the point in the trial where the change occurs (post-change phase onset). The solid dark line reflects the proportion of trials with observed data. The thick broken vertical lines indicate the first and last points at which at least 50% of all trials had fixations recorded to one or the other item (the other trials had fixations directed to other locations on the screen or off the screen altogether). These lines also indicate the analysis window for the Monte Carlo time-course analysis (see text). The analysis window for the preference score is shown in the shaded area, which begins 200 ms after the post-change onset and ends at the end of the analysis window.

The mean across infants of the median change preference score was .61 (SD =.13), which was significantly different from chance, t(22) = 4.12, p < .001, d = .85. Eighteen of the 23 infants had a median score greater than .50, which was greater than would be expected by chance (binomial test, p < .05). As observed in Experiment 1, there was no relation between the number of trials completed and median change preference, r = .04, n = 23. This is illustrated by the size of the circles—which is scaled by the number of trials completed—and the median preference score in Figure 5. In addition, the results were the same when we analyzed only the first N (20, 30, 40, or 50) trials, indicating that the effects did not change as the session progressed.

We next evaluated infants’ preference for the changed item sample-by-sample using the procedure described in Experiment 1. Figure 7 shows the infant-weighted mean preference time course for this experiment; infants showed a clear preference for the changed item from approximately 283 to 933.3 ms after the onset of the post-change display. Analyzing the group average at each sample period yielded 51 samples that were significantly above chance, 40 of which were consecutive. This cluster length is longer than the 16 samples calculated as the 95% cutoff (calculated using the same sampling techniques as in Experiment 1; see Figure 8 for distribution of clusters). In other words, infants exhibited a significant preference for the changed item from 283.3 ms to 933.3 ms after the onset of the test array (corrected for multiple comparisons).

Figure 7.

Preference for the changed item as a function of time relative to the onset of post-change phase in Experiment 2. Each time point that was significantly greater than chance (without correction for multiple comparisons) is indicated by an asterisk. The set of asterisks from 283.33 – 933.33 ms formed a cluster that was statistically significant after correction for multiple comparisons.

Figure 8.

Distribution of run lengths obtained from 1000 permutations of the data in Experiment 2. This is the distribution of values that would be expected by chance (the null distribution). The dotted line indicates the 95% cutoff (i.e., 95% of the permutation iterations had run lengths shorter than this). We considered any observed run length from the actual data in Experiment 2 above this cutoff as statistically significant.

The design of this experiment allowed us to examine whether infants’ responding varied as a function of which item changed color from sample to test. Specifically, the jittered presentation may have helped infants individuate one of the items, reducing the set size to 1 item, much like the rotating object in Ross-Sheehy et al.’s (2011) study. During the sample period, infants did look longer at the item that appeared second than at the item that appeared first. We evaluated this by dividing the amount of time infants looked at the item that appeared second by their total looking at the two items. Infants’ mean proportion of looking to this second item was M =.79 (SD =.33), significantly more than the expected chance level of .50, t(22) = 4.24,p < .001, d = .88. This raises the possibility that infants in this experiment encoded only the features of that second item, and that they detected a change only when the change occurred in the item that onset second.

We tested this possibility first by calculating separate change preference scores for trials in which the first-appearing item changed and trials in which the second-appearing item changed. Infants’ proportion of looking to the changed item was significantly above chance only when the second-appearing item changed, M = .67, (SD = .18), t(22) = 4.39,p < .001, d = .94. When the first-appearing item changed, the preference score was at chance, M = .50, (SD =.23), t(22) = .01, p = .99, d < .001. This pattern is consistent with the conclusion that infants encoded only the second item that appeared, and only detected a change when that second item changed.

The time-course analyses for each type of trial, however, provide additional insight (see Figure 9). During the sample period, infants’ looking was usually directed at the item that appeared second during the sample—the changed item in the top graph and the non-changed item in the bottom graph. They then maintained their looking to that item if it changed (top panel of Figure 9). Because infants looked more at the second-appearing item during the sample period, they tended to be looking at the unchanged item at the start of the test period when the first-appearing object was the one that changed, leading to the appearance of a bias away from the changed item early in the trial (bottom panel of Figure 9). However, they did not maintain this bias for long once the test array appeared. This is clear by comparing the proportion of time infants looked at the “to-be-changed” target item (the first-appearing item) during the sample phase (from −900 through 0 ms, M = .23) with the proportion of time they looked at that same item during the post-change phase (200 ms – 1650 ms, M = .50). They significantly increased the proportion of their looking devoted to this item from pre to post change, t(22) = 2.78, p = .011, d = .62. That is, infants shifted their looking away from the second-onset item and redirected their gaze to the changed item.

Figure 9.

Separate time course of change preference (left) and median of mean change preference scores collapsed over the entire preference score window (right) in Experiment 2 for trials in which the second-appearing item changed (top) and trials in which the first-appearing item changed (bottom). In the time course plots, samples in which the preferences differs significantly from chance are indicated with an asterisk. Note in the bottom graph, infants begin to prefer the second onset item before the change, and maintain a preference for that item after the change, despite the fact it is the non-changing item. In the collapsed data, each circle represents the preference score from a single infant. The black line within the cloud of circles indicates the mean of the median preference scores, with the 95% confidence interval shown by the gray error bars.

This pattern may reflect one of two influences. First, infants may detect that the item they are fixating is unchanged, and it therefore fails to maintain their interest and they shift toward looking at the other item. Alternatively, infants may detect the change in the non-fixated item and shift their attention toward that changed item. Regardless of which interpretation is correct, the fact that infants maintain their looking at a changed item and do not maintain their looking at an unchanged item indicates that they have encoded in VSTM the color of at least the initially fixated item (i.e., the item that onset second), and that their looking behavior during the sample period reflects their recognition of the discrepancy or lack of discrepancy between that stored information and the currently available information.

Clearly, this study demonstrated that infants detected the change in the array of two items, providing findings that converge with those of Experiment 1. Together, these experiments show that young infants can detect changes in multiple-item arrays when the stimuli are modified to make it easier to individuate the items.

Exploratory Examination of the Data

Note that although Figure 9 suggests that infants encode and remember only the item they actually look at, the data were not filtered to take into account which items were fixated during the sample period. Although a full exploration is beyond the scope of the present paper, we conducted an exploratory examination of infants’ responses in both Experiments 1 and 2 as a function of which item they looked at more during the pre-change period. For both experiments we divided each infants’ data into trials in which they looked more at the to-be-changed item and trials in which they looked more at the non-changing item during the pre-change period. In Experiment 1, there were 553 trials in which infants looked more at the to-be-changed item and 556 trials in which infants looked more at the item that did not change during the pre-change phase; in Experiment 2, there were 286 trials in which infants looked more during the pre-change period at the to-be-changed item and 305 trials in which infants looked more at the item that did not change.

The time course figures and the plots of the median change preferences in the analyses window are presented in Figure 10. We did not conduct formal analyses of the data given the exploratory nature of the question being addressed, but several patterns are clear from the data. First, infants tended to remain fixated on the item they had been fixating when the change occurred. The top lines in each graph represents trials in which infants preferred the to-be-changed item during the prechange period. In both experiments, infants continued to prefer that item well into the change period. The bottom lines in each timecourse represent trials in which infants were initially fixating the item that does not change; in these trials infants initially prefer the unchanging item (which they were looking at before the change occurred), but over time their responding approaches chance. Altogether, the main analyses and these exploratory examinations of the data suggest that the modifications in this task facilitated infants’ selection, processing, and encoding of one of the two items, and their detection that that item did or did not change color.

Figure 10.

Separate time course graphs (left) for Experiments 1 and 2 depicting trials in which infants looked longer at the to-be-changed item or the non-changing item during the prechange phase and graphs for each kind of trial (right) depicting the median of mean change preference scores collapsed over the entire preference score window. In the collapsed data, each circle represents the preference score from a single infant. The black line within the cloud of circles indicates the mean of the median preference scores, with the 95% confidence interval shown by the gray error bars.

Note that infants in previously reported change detection tasks may have also remembered the item they fixated, particularly in the one-shot change detection task. But, in contrast to the results reported here, 6-month-old infants did not show an overall preference for the changed item when the two items were identical in shape and onset at the same time (Oakes et al., 2013). Moreover, the fact that infants of this age failed to detect changes in the simultaneous streams task even when every (identically shaped) item changed colors (Oakes et al., 2006, 2009) suggests that in this context infants did not seem to select a single item and detect when it changed.

General Discussion

Whereas previous studies have repeatedly failed to find evidence that young infants can detect changes in multiple-item arrays (Oakes et al., 2009, 2013), the present study has shown that 6-month-old infants can succeed in this task when cues are added that make it easier to rapidly individuate the items. For example, Oakes et al. (2013) used a procedure that was nearly identical to that used in the present study, except that the objects within a given array were identical in terms of both shape and timing. Six-month-old infants exhibited no evidence of being able to detect color changes under those conditions. The present study found that when the shapes in the arrays varied (Experiment 1) or had different onset/offset times (Experiment 2), 6-month-old infants did detect changes in color. This pattern mimics that observed by Feigenson (2005) for 7-month-old infants’ sensitivity to the number of items in arrays containing 1 to 3 items. Infants recognized differences in number when the items differed along a number of dimensions (Feigenson, 2005), but not when they were similar (Clearfield & Mix, 2001; Feigenson, Carey, & Spelke, 2002). Thus, infants in this age range appear to rely on object features to attend to, process, and encode individual items when multiple items are simultaneously visible).

Our first study of VSTM in infants (Ross-Sheehy et al., 2003) concluded that the ability of older but not younger infants to detect changes in multiple-element arrays was best explained by an increase in VSTM storage capacity. However, the present results (as well as the results of Oakes et al., 2009 and; Ross-Sheehy et al., 2011) indicate that the changes in VSTM performance between younger and older infants are not entirely a result of increased capacity. Taken together, the pattern of results across many experiments suggests that infants’ VSTM performance is influenced by their developing ability to individuate similar objects.

However, storage capacity may also develop over this period, and capacity at 6 months may be limited to a single item. This conclusion would be consistent with results reported by Kaldy and Leslie (2005) suggesting that 6.5-month-old infants could store identifying information about only 1 item in working memory. What is unknown is whether older infants’ apparent greater VSTM capacity reflects their processing and storing multiple items in VSTM, or their ability to rapidly simultaneous individuate a single item in VSTM. Future research will need to address the role of individuation and how many items are stored in older infants’ VSTM. But, one important implication of the work reported here is that given that objects in the natural environment often differ along multiple dimensions, infants may be able to use VSTM more successfully in their everyday lives than one might have expected on the basis of previous studies. Thus, these results do not simply demonstrate that infants of this age can use VSTM to detect changes in multiple-item arrays (potentially addressing a competence-performance distinction), they also begin to reveal how the development of individuation processes contributes to infants’ performance of VSTM tasks.

This development of the ability to rapidly individuate objects may have significant implications for infants’ visual object cognition in general (see Y. Xu & Chun, 2009). As described earlier, Feigenson (2005) suggested this connection with respect to infants’ attention to number. Other work also suggests developmental changes related to the ability to individuate. For example, Ross-Sheehy et al. (2016) found that infants’ ability to segregate figure from ground develops in the middle of the first postnatal year. Figure-ground segregation is an important step in identifying multiple individual objects in a visual array. Kaldy and Blaser (2013) also observed that young infants could use color or movement to individuate simultaneously presented items that were the same shape; in this case the displays were visible for several seconds, giving infants more time to process the visual information in the arrays. Taken together, the results of these studies suggest that infants have difficulty individuating items in multiple-item arrays, but may succeed when the items vary along multiple dimensions (Experiment 1), have separate onsets (Experiment 2), move independently (Kaldy & Blaser, 2013; Ross-Sheehy et al., 2011), or are visible for a relatively long time (Kaldy & Blaser, 2013). Infants’ use of spatiotemporal versus featural information to individuate items (presented one at a time) also develops over the first year (Wilcox & Schweinle, 2003; F. Xu & Carey, 1996). Thus, multiple skills that develop across the first year are important for tasks such as change detection, and it is possible that differences in younger and older infants’ use of VSTM reflects developmental changes in these skills.

Our focus on individuation may seem at odds with previous conclusions that what develops during the first year is the ability to bind object features to locations (Kaldy & Leslie, 2005; Kâldy & Leslie, 2003; Oakes et al., 2006, 2009). However, it is difficult to separate these explanations, and binding has been described as part of the individuation process (Wutz et al., 2012) or as a process subsequent to individuation (Wilson et al., 2012). Clearly, the two processes are intertwined. Here we chose to focus on the concept of individuation because our results do not require that infants’ bind object features to a particular location. However, infant responding in this task may reflect, at least in part, the ability to bind objects to locations (see Oakes et al., 2009).

In summary, the results reported here extend our understanding of the factors that contribute to infants’ limited VSTM capacity. Specifically, infants’ difficulty detecting changes in multi-item arrays does not solely reflect the absolute number of items that they can hold in VSTM. Rather, their performance in VSTM tasks is also influenced by processes that allow them to attend to, process, and encode individual items in the array. Research like that reported here will provide a further understanding of how visual cognitive abilities such as individuation, attention, feature binding, and VSTM operate together as the infants view the world.