2-year-olds succeed at MIT: Multiple Identity Tracking in 20- and 25-month-old infants

Abstract

Infants’ ability to remember objects and their locations emerges during the first year of life (Kaldy & Leslie, 2005; Richardson & Kirkham, 2004; Ross-Sheehy, Oakes, & Luck, 2003). However, not much is known about infants’ ability to track objects’ identities in a dynamic environment. Here we tailored the Delayed Match Retrieval eye-tracking paradigm (Kaldy, Guillory, & Blaser, 2016) to study infants’ ability to track two object identities during occlusion - an infant version of Multiple Identity Tracking (MIT) (Oksama & Hyönä, 2004). Delayed Match Retrieval uses virtual ‘cards’ as stimuli that are first shown face-up, exposing to-be-remembered information, then turned face-down, occluding it. Here, cards were subject to movement during the face-down occlusion period. We used complex, non- nameable objects as card faces to discourage verbal rehearsal. In three experiments (N = 110), we compared infants’ ability to track object identities when two, previously exposed, cards were static (Experiment 1), were moved into new positions along the same trajectory (Experiment 2), or were moved along different trajectories (Experiment 3), while face down. We found that 20-month-olds could remember two object identities when static, however, it was not until 25 months of age that infants could track when movement was introduced. Our results show that the ability to track multiple identities in visual working memory is present by 25 months of age.

1. Introduction

1.1. Multiple Identity Tracking

Keeping track of individual objects and agents - teammates on a field, passing cars on the road - is fundamental to maintaining an up-to-date representation of a dynamic visual world. Multiple Object Tracking (MOT) was introduced 30 years ago to study this skill experimentally (Pylyshyn & Storm, 1988). In this paradigm, the observer’s task is to track a cued, target subset of identical-looking, independently-moving objects (classically, circles) for a fixed amount of time. When the time is up, the objects stop, and the observer has to select the items that belonged to the target subset. This paradigm led to a number of insights into the mechanisms underlying visual attention and Visual Working Memory (VWM) (Cavanagh & Alvarez, 2005; Meyerhoff, Papenmeier, & Huff, 2017; Pylyshyn, 2001; Yantis, 1992).

But how well can one track the identities of moving objects? To answer this question, variants of the MOT paradigm were developed. Pylyshyn (2004) placed numerals on the target subset before they started moving, and found that while observers were able to subsequently pick out the members of the subset, they showed poor performance when they were asked to recall their numbers. Oksama and Hyönä(2004, 2008) introduced the term Multiple Identity Tracking (MIT) for their variant of this paradigm, where instead of circles, they used line drawings of everyday objects. In their version, object identities were visible throughout the movement phase, disappearing just before responses were required. Horowitz and colleagues (2007) used cartoon animals in their MIT task with simpler, translational movements. In all cases, there was a consistent “content deficit”: fewer items could be tracked when participants needed to report the location of targets with a particular identity (where did the rabbit go?), rather than just the locations of all targets in the set (where did the four animals go?).

While we situate the present study in an MIT framework, it is also related to the literature on the development of visual working memory (VWM, for a recent review, see Fitch et al., 2016). VWM has been identified as a fundamental (though not sufficient) process underlying MOT (Drew, Horowitz, Wolfe, & Vogel, 2011; Fougnie & Marois, 2006; Meyerhoff et al., 2017) and, in MIT, memory demands are even higher, as participants need to encode and dynamically maintain identity-location bindings. Robust inter-individual correlations have been found between MIT performance and standardized measures of VWM (Oksama & Hyönä, 2004). In addition, Makovski and Jiang (2009) demonstrated that tracking performance in a MOT task was better when objects were all unique in color. (This at first seems to be at odds with the “content deficit” effect. However, in this paradigm, participants did not need to report the identities of targets, only their locations after the tracking period. Indeed, when participants had to hold a set of colors in their VWM in a dual-task design, this eliminated the advantage of tracking uniquely colored objects.)

While MIT has an established literature in adults, little is known about its developmental trajectory. In the next section, we review what we do know about infants’ emerging skills for remembering object identities (in static occlusion tasks) and keeping track of objects while hidden¹.

1.2. The development of identity tracking

Object permanence emerges very early in development (Baillargeon, 1987; Baillargeon, Spelke, & Wasserman, 1985). By 5 months, infants can encode a hidden object’s location in VWM, and will look longer when it is retrieved from a different location. However, evidence for the ability to track an object’s identity under 6 months is mixed (Newcombe, Huttenlocher, & Learmonth, 1999; Stavans & Baillargeon, 2018). At 6 months, infants can successfully encode two objects and will notice if one is missing when occluders are removed, but do not notice if one of them has changed identity (Kibbe & Leslie, 2011). By 9 months of age, infants can remember not only the location of a hidden object, but also its identity, i.e. what was where, and will look longer if two objects were not in their original locations after taking away the occluders (Kaldy & Leslie, 2005). Later, at 12 months, infants can succeed at encoding three objects and their respective hiding locations (Kibbe & Leslie, 2013). Taken together, starting early in the first year, infants show a gradual increase in the ability to remember the identities of occluded objects.

Piaget’s invisible displacement task was the first to test infants’ ability to track an object changing location while occluded (Piaget, 1954). In this task, infants were presented with multiple hiding locations, a target was placed in one of them, then transferred (while occluded) to another location, after which the infant was allowed to search for it. Follow-up studies found that 18-month-olds could track an object’s change in location (when experimenters moved the object while hidden in their hands), while 12- and 15-month-old infants could not (Corrigan, 1981; Somerville & Haake, 1985). In addition, 20-month-olds demonstrated fewer perseverative errors (searching for the object in the original hidden place) than 15-month-olds (Sophian & Sage, 1983; Wiebe, Lukowski, & Bauer, 2010). Similarly, in a location-updating task, perseverative errors occurred when 23-month-olds encoded the initial location of an object through direct observation, testimony, or both, suggesting that information about the previously encoded location may have interfered with the information about the new location. It was only at 30 months of age that they were able to resolve the conflicting information (Ganea & Harris, 2013). Taken together, evidence in invisible displacement tasks and verbal updating tasks suggests that the ability to track an occluded object gradually emerges between 18 and 30 months of age.

These tasks, however, only tested infants’ ability to track one object, without testing the ability to concomitantly track its identity. Despite its importance in everyday tasks, there are only a handful of studies on infants’ ability to track identity. Richardson and Kirkham (2004) were the first to test infants’ ability to track objects’ identities while hidden under moving occluders. In this influential study, they showed that after 6-month-old infants learned an association between two sounds and two location placeholders, they were able to successfully track the placeholders as they changed locations, looking to the correct updated location upon hearing the associated sound. Follow-up studies (Kirkham, Richardson, Wu, & Johnson, 2012) described the developmental shift of using multiple cues across perceptual domains to track objects and locations from 3 to 10 months of age. It is important to note, that in this paradigm, infants first learned the (fixed) sound-object associations over 48 seconds of training, therefore, during test trials they could rely on long-term (recognition) memory for the initial identity-location bindings. Subsequently, VWM was only required for tracking the location of placeholders as they moved. In our task, however, even the specific identity-location association had to be encoded and maintained in VWM on each trial (“this time, the swirl is on the face of the left card, and the watermelon-slices are on the face of the right card”), based on a brief (1-2 s) encoding interval, and these links had to be tracked during occlusion and recalled during test. Trial-by-trial, infants needed to clear the contents of VWM in order to encode new bindings of identity and location (for a similar argument, see Reznick, 2008, and Kaldy & Leslie, 2005).

1.3. Overview of the present study

The current study aims to examine infants’ ability to track the identities of moving objects while occluded. To test this, we adapted the Delayed Match Retrieval (DMR) eye-tracking paradigm (Kaldy et al., 2016). In test trials, three face-down virtual ‘cards’ were presented. Two of them (the Match and the Non-Match) flipped over sequentially to expose the faces of the cards, then flipped back face-down. Next, the third card - the Sample - was flipped to reveal a match to one of the previously exposed, now face-down, cards (i.e., the Match). Infants were to make an anticipatory saccade to the (face-down) Match card. Previously, we showed that both 10- and 13-month-olds could succeed in this task with both simple geometric shapes (Kaldy et al., 2016) and familiar objects (Cheng, Kaldy, & Blaser, 2019). In the current study, we introduced movement conditions to the face-down Match and Non-Match cards before the Sample was revealed. To be successful here then, infants needed to track the identities of the cards as they shifted positions.

According to Piaget’s classic description and recent studies, infants over 18 months of age can typically pass the invisible displacement task with one object (Diamond, Prevor, Callender, & Druin, 1997; Piaget, 1954; Somerville & Haake, 1985; Sophian & Sage, 1983; Wiebe et al., 2010). Since our task requires tracking the identity of multiple objects, infants may not be able to succeed until a later age, so we tested 20- and 25-month-old infants. By this age, infants are capable of silently labeling simple, familiar objects (Mani & Plunkett, 2010), thus may attempt verbal rehearsal. In order to minimize this, similarly to Oksama and Hyönä (2008), we used complex, non-nameable stimuli (see Figure 1a).

Figure 1. Stimuli and sequence of test events in a typical trial.

(a) Virtual cards, shown face-up, exposing to-be-tracked objects. In each trial, two of the four cards were chosen randomly to be the Match and the Non-Match. (b) Sequence of test events in Experiment 1, (c) Experiment 2, (d) and Experiment 3. Note that in Experiment 3, during the delay, one of the two target cards moved to a location that had been previously occupied (“old” location), the other card moved to a location that had not been previously occupied (“new” location). The Match card could be either one of the two cards. In half of the trials, the Match ended up in an “old” location (as in Fig. 1b), while in the other half, in a “new” location. Anticipatory gaze responses were analyzed during the 2-second response phase (see red frame), where performance was measured as the percent of trials where the infants’ first look was to the Match card. Areas of Interest (AOIs) for the Match, Non-Match, and Sample cards, are shown overlaid on the frame in the upper right corner.

In Experiment 1, we tested 20-month-olds in a standard, ‘no movement’ DMR task (Figure 1b). Based on our previous findings, we hypothesized that by this age, infants could remember two static identity-location bindings even with abstract, non-nameable stimuli. Then, in Experiment 2, we introduced a relatively simple, ‘translational’ movement of the cards during a retention phase (Figure 1c). In Experiment 3, we introduced a more complex ‘shuffle’ movement, where the two cards simultaneously moved on orthogonal paths, with one of them crossing the midline (one of the face-down cards moved to a new location, while the other card took the position that the first had occupied) (Figure 1d). We hypothesized that identity tracking emerges during the end of the second year of age, producing an age-related increase in performance between 20- and 25-months. Since we hypothesized that MIT is still developing over this age range, we further expected that participants would perform better with the relatively simple translational motion of Experiment 2 as compared to the complex shuffle movement of Experiment 3 (this experiment further tested the possibility that moving an object into a previously occupied location may interfere with encoding, as compared to an object moved to an entirely new location).

2. Experiment 1: Delayed Match Retrieval (DMR), no movement

2.1. Method

2.1.1. Participants

Twenty-two healthy, full-term infants (11 girls) were recruited from the Greater Boston area, and tested at University of Massachusetts Boston (M = 19.3 months, SD = 1.42, age range: 17.70 - 22.76 months). This sample size (N = 22 infants per group) was based on a power analysis (Faul, Erdfelder, Lang, & Buchner, 2007) conducted for a one-sample t-test, to be able to detect a minimum d = 0.65 effect size (80% power, alpha = 0.05); and matched the sample size used in a previous DMR study from our lab (Cheng et al., 2019) To be included in the analyses, each participant had to complete at least 3 trials, where they fixated each of the two to-be-tracked cards during the encoding phase, and one of the (face-down) cards during the response phase. We used a minimum of 3 trials as an inclusion criterion to err on the side of inclusivity, and to be consistent with the criteria used in our previous study (Cheng et al., 2019). An additional five infants were tested, but did not meet this criterion and were excluded. Caregivers received a small gift and $20 compensation for their time and travel expenses. All caregivers gave informed consent before the experiment.

2.1.2. Apparatus and Stimuli

Caregivers sat in a chair holding their infants in their laps in front of a Tobii T120 eye-tracker (Tobii Technology, Stockholm, Sweden) in a dimly lit testing room. Caregivers were asked to wear a visor to cover their eyes and not to interact with their infants during testing. The standard Tobii 5-point infant calibration was used. As described above (1.3. Overview) experimental stimuli were virtual cards that could be shown face up, revealing an unfamiliar, abstract object, or face-down, obscuring it. We used a total of four different objects as card faces (Figure 1a). Trial-by-trial, two different cards were chosen randomly as the Match and the Non-Match cards, while the Sample card had an identical image to the Match. Cards subtended 5×5 deg and were arranged symmetrically with their centers 5 deg from the center of the screen.

2.1.3. Design and Procedure

Infants were first presented with four brief familiarization sequences (10 s in total), during which two face-up, matching cards entered the screen, approached each other, jiggled, and then exited together. This sequence was repeated once for each of the four object types.

In test trials, three cards (the Match, Non-Match, and Sample) entered, face-down, from the side of the screen and formed a triangular arrangement near the center. The Match and Non-Match cards were always at the bottom corners, with the Sample on top. During the encoding phase, the Match and Non-Match were flipped face up (an animation that took approximately 500 ms), sequentially (first, one of the two cards, randomly chosen, was exposed, then, after 1 s, the other card was flipped face up). The two cards stayed face up for 1 s, and then both flipped face-down simultaneously. After that, the Sample card, which had an identical image to the Match, was flipped face-up. The reveal of the Sample marked the beginning of the 2 s response phase, during which anticipatory gaze responses were recorded. After this phase, the Match card was immediately flipped face-up, accompanied by a brief (~800 ms) reward animation (e.g., a colorful burst of fireworks) at its location. This reward was designed to both provide feedback about the location of the Match and to encourage participants to make an anticipatory saccade to the Match location so as not to miss the brief reward. The Match then moved next to the Sample, touched it, and then all three cards flew off screen (Figure 1b). To maintain infants’ engagement, we added unique sounds to each of the card movements (e.g., flipping, moving, and touching) and alternated between three different reward animations (fireworks, sparkles, flashbulbs). (Please see Supplementary_material_Exp1_demo for a demonstration of the test event sequence.)

Twelve test trials were presented. Card identity, the order of the cards being exposed, the side of the Match (left/right), and the reward animation type was counterbalanced across trials. The counterbalancing of the first three factors ensured that infants had to keep track of the object identities in each trial. Each test trial was followed by an ‘attention grabber’ sequence (a cartoon sun rotating in the center of the screen for 4 s, with a sound effect), to attract infants’ gaze towards the screen.

2.2. Data Analysis

Each card was bounded by a 7×7 deg Area of Interest (AOI) (the area was slightly larger than the cards, which subtended 5×5 deg, to account for minor calibration errors). During the response phase, participants should be motivated to look at the Match card as quickly and accurately as possible - and to linger there - in anticipation of the reward animation. We calculated VWM performance as the percent of correct responses based on the dependent variable of their first looks² , that is, which of the two face-down cards, Match or Non-Match was fixated first during the response phase (i.e., after the Sample was revealed). We also calculated the percent of correct responses based on longer looks, that is, which of the two face-down cards accumulated longer looking time during the response phase. Since the response phase was only 2 seconds, on most trials the card that participants fixated first was also the one fixated longer, so these two variables are highly correlated. In analyzing both of these measures, we followed previously established practices (e.g. Addyman and Mareschal, 2010, Hochmann, Mody, and Carey, 2016, and Kaldy et al., 2016). If the infant did not look at either of the two cards during the response phase, the trial could not be coded, and was excluded from further analysis. Analyses of gaze data were done using custom MATLAB scripts.

2.3. Results

2.3.1. Overall Performance

Participants contributed an average of 8.9 valid trials (SD = 2.5) out of 12. Participants’ average performance was 56% correct (SD = 0.12), which was significantly above chance (50%) according to a one-sample t-test (t(21) = 2.50; p = 0.021, CI = [0.51, 0.62], d = 1.09, t-tests were two-tailed throughout all analyses) (Figure 2). Infants’ average performance based on which of the two cards (Match or Non-Match card) garnered a longer look showed similar results (M = 56%, SD = 0.12, t(21) = 2.41; p = 0.025, CI = [0.51, 0.62], d = 1.05). We tested the effects of the side of the Match card (left/right), whether the Match was presented first or second using χ² tests (treating each trial as an independent observation). Neither of these were significant (all χ² < 1.17, p > 0.30).

Figure 2. Results of Experiment 1 (DMR, No movement).

Individual and group average performance (percent correct responses based on first looks). The size of the circle corresponds to the number of valid trials the infant contributed. Error bars reflect SEM.

2.3.2. Learning effect: Analysis of trial-by-trial performance

It is also important to note that besides tracking two object identities, successful performance in our task requires encoding the matching rule: anticipatory looking toward the Match card during the response phase will result in being able to catch the short reward animation. While this does not seem to be a performance-limiting factor in infants (Kaldy et al., 2016), it is possible that it may interact with the increased demands of the memory task itself. Therefore, in each of our experiments, we also analyzed how performance changed over the block of trials. We performed a linear regression between average task performance in each trial, and trial number (1 to 12). We did not find a significant effect of learning over trials (R² = 0.04, p = 0.54 based on first looks; R² = 0.01, p = 0.85 based on longer looks), suggesting that 20- month-olds’ performance did not change systematically over the block of trials.

3. Experiment 2: DMR with Translational movement

Experiment 1 showed above-chance performance in 20-month-old infants, suggesting that they could keep track of two (static) object identities during occlusion, even when the identities were abstract unfamiliar shapes. In the next two experiments, we explored whether infants of the same age and older (at 25 months) could keep track of identity while the objects moved during occlusion. Here, during the retention phase, to-be-tracked cards underwent translational movement, from the top to the bottom of the screen, preserving their relative left/right positions.

3.1. Method

3.1.1. Participants

Forty-four full-term, healthy infants (17 girls) were recruited from the Greater Boston area, and tested at University of Massachusetts Boston. The infants were recruited in the same way as in Experiment 1. We tested two age groups with 22 infants in each: 20-month-olds (M = 19.8 months, SD = 1.06, age range: 18.13 - 21.73 months) and 25-month-olds (M = 24.8 months, SD = 1.36, age range: 22.2 - 26.9 months). An additional six infants (four in the 20-month-old group and two in the 25-month-old group) were tested, but were excluded due to an insufficient number of valid trials (i.e., fewer than three, with trial inclusion criteria as defined in Experiment 1).

3.1.2. Apparatus and Stimuli

The same apparatus and stimuli were used as in Experiment 1.

3.1.3. Design and Procedure

Infants were first presented with the same familiarization trials as in Experiment 1. For test trials, the only substantive difference from Experiment 1 was that (face-down) cards moved during the retention phase. In each trial, three cards (the Match, Non-Match, and Sample cards) entered from the side of the screen, face-down, and formed a triangular arrangement near the upper side of the screen. The Match and Non-Match were at the top corners of the triangle, and the Sample was always in the center of the screen. During the encoding phase of the trial, the Match and Non-Match were flipped face-up, sequentially. The two cards stayed face-up for 1 s, and then both flipped face-down at the same time. After that, the two face-down cards moved down to the bottom of the screen (1.2 s), and the Sample was exposed. The response phase (2 s) and the subsequent reward animations were the same as in Experiment 1. (Please see Figure 1c and Supplementary_material_Exp2_demo.)

3.2. Data Analysis

Data analysis was the same as in Experiment 1.

3.3. Results

3.3.1. Overall Performance

Participants on average contributed 7.2 valid trials (SD = 2.8) in the 20-month-old group and 6.6 trials (SD = 3.0) in the 25-month-old group (out of 12 trials). There was no significant difference in the number of valid trials between the two age groups (two-sample t-test: t(42) = 0.68; p = 0.50).

20-month-old participants’ average performance based on first looks was 50% correct (SD = 0.16), which was not different from chance (one-sample t-test: t(21) = -0.11; p = 0.92, CI = [0.42, 0.57], d = 0.05) (see Figure 3). The performance of 25-month-olds, however, was significantly above chance at 61% correct (SD = 0.18) based on first looks to the Match card (t(21) = 2.78; p = 0.011, CI = [0.53,0.69], d = 1.21). The same results were found based on longer looks to the Match card, with 20-month-olds again at chance (M = 49%, SD = 0.17, t(21) = −0.21; p = 0.84, CI = [0.42, 0.57], d = 0.09) and 25-month-olds showing significantly above-chance performance (M = 59%, SD = 0.19, (t(21) = 2.25; p = 0.035, CI = [0.51, 0.67], d = 1.21).

Figure 3. Results of Experiment 2 (Translational movement).

Individual and group average performance (percent correct responses based on first looks). The size of the circle corresponds to the number of valid trials the infant contributed. Error bars reflect SEM.

We tested the effects of the side of the Match card (left/right), and whether the Match was presented first or second during encoding. Infants responded similarly when the Match card was on the left or on the right in both age groups (all χ² < 0.15, p > 0.69). We did not find any differences in performance when the Match card was first vs. second shown in either of the two age groups (all χ² < 0.26, p > 0.61).

3.3.2. Learning effect: Analysis of trial-by-trial performance

To examine the potential effect of learning in 20-month-olds, we compared differences in task performance over trials. We performed a linear regression between average task performance (based on first looks) and trial number (1 to 12), and we found a moderate, marginally significant positive trend indicating a learning effect over trials: R² = 0.325, p = 0.052. Task performance based on longer looks did not show this trend over trials (R² = 0.13, p = 0.24). For 25-month-olds, the result of the linear regression analysis between trial number and percent of correct responses was not significant (R² = 0.16, p = 0.19 based on first looks; R² = 0.02, p = 0.69 based on longer looks).

4. Experiment 3: DMR with Shuffle movement

In this experiment, during the retention phase, to-be-tracked cards underwent a more complex ‘shuffle’ movement, shifting positions along orthogonal trajectories, with one of the two cards crossing the midline.

4.1. Method

4.1.1. Participants

Forty-four full-term, healthy infants (24 girls) were recruited from the Greater Boston area, and tested at University of Massachusetts Boston. They were assigned to two age groups with 22 infants in each: 20-month-olds (M = 19.4 months, SD = 1.24, age range: 17.83 - 21.77 months) and 25-month-olds (M = 25.0 months, SD = 1.80, age range: 22.4 - 27.97 months). An additional seven infants (three in the 20-month-old group and four in the 25-month-old group) were tested, but were excluded due to an insufficient number of valid trials (fewer than three, trial inclusion criteria as defined in Experiment 1). The infants were recruited in the same way as in Experiment 1.

4.1.2. Apparatus and Stimuli

The same apparatus and stimuli were used as in Experiment 1.

4.1.3. Design and Procedure

Infants were first presented with same familiarization trials as in Experiment 1. For test trials, the only substantive difference compared to the test events of Experiment 1 was that the cards moved during the retention phase. In each test trial, the Match, Non-Match, and Sample cards entered either from the left or right side of the screen, face-down, and formed a triangular arrangement either near the left or right edge of the screen. The Match and Non-Match were at the side corners of the triangle, and the Sample was always in the center of the screen. The timing of the encoding phase was identical to Experiment 1. The two cards stayed face up for 1 s, and then both flipped face-down at the same time. The Match and Non-Match then moved clockwise (when cards started on the right side), or counter-clockwise (when cards started on the left side), to the bottom of the screen to become horizontally aligned. In this way, the top card moved down to the bottom of the screen, while the bottom card moved sideways to the opposite side of the screen; two simultaneous, orthogonal movements. Movement unfolded over 1.2 s. (The timing of the events during retention was the same as in Experiment 2). After that the Sample card was flipped face up. Events in the following response and reward phases were identical to Experiment 1. (See Figure 1d and Supplementary_material_Exp3_demo)

4.2. Data Analysis

Data analysis was the same as in Experiment 1.

4.3. Results

4.3.1. Overall Performance

On average, 20-month-olds contributed 6.9 valid trials (SD = 2.2), and 25-month-olds 8.1 trials (SD = 2.9), out of 12 trials. There was no significant difference in the number of valid trials between the two age groups (two-sample t-test: t(42) = 1.56; p = 0.13).

20-month-olds’ average performance was not different from chance based on the percent of first looks (one-sample t-test: M = 48%, SD = 0.20, t(21) = −0.50, p = 0.62, CI = [0.39, 0.57], d = 0.21) or the longer looks (M = 47%, SD = 0.19, t(21) = −0.72, p = 0.48), CI = [0.39, 0.56], d = 0.31) to the Match. In 25-month-olds, the average performance over all 12 trials was 57% correct (SD = 0.20) based on first looks, which, while not significantly different from chance (t(21) = 1.59; p = 0.13, CI = [0.48, 0.65]), showed a medium effect size (d = 0.69) (See Figure 4). Infants’ average performance based on which of the cards (Match or Non-Match) garnered the longer look was significantly better than chance (M = 58%, SD = 0.19, t(21) = 2.114; p = 0.046, CI = [0.50, 0.67], d = 0.92).

Figure 4. Results of Experiment 3 (Shuffle movement).

Individual and group average performance (percent correct responses based on first looks). The size of the circle corresponds to the number of valid trials the infant contributed. Error bars reflect SEM.

4.3.2. Learning effect: Trial-by-trial analysis

As in the previous two experiments, we conducted a linear regression between performance based on first looks in a particular trial and the trial number. We did not see a significant relationship in either of the two age groups in Experiment 3 (in 20-month-olds: R² = 0.09, p = 0.35 based on first looks; R² = 0.13, p = 0.25 based on longer looks; in 25-month-olds: R² = 0.17, p = 0.19 based on first looks; R² = 0.06, p = 0.46 based on longer looks). In sum, infants in Experiment 3 did not show a significant learning effect over trials.

4.3.3. Interference when updating information to a new location vs. an old location

Updating remembered information at a previously encoded location may cause interference, a failure to update representations based on new information (Ganea & Harris, 2013). To explore this, we compared infants’ average performance when the Match card moved to a ‘new’ (previously unoccupied) location to when the Match card moved to an ‘old’ (previously occupied) location. We did not find a significant difference in performance based on first looks in either of the two age groups (20-month-olds: M_{Old_Location} = 49%, M_{New_Location} = 50%, t(20) = 0.08, p = 0.94 (paired t-test); 25-month-olds: M_{Old_Location} = 59%, M_{New_Location} = 55%, t(21) = 0.49, p = 0.63). Performance based on longer looks showed the same pattern (20-month-olds: M_{Old_Location} = 51%, M_{New_Location} = 47%, t(20) = 0.61, p = 0.55 (paired t-test); 25-month-olds: M_{Old_Location} = 60%, M_{New_Location} = 57%, t(21) = 0.45, p = 0.66).

We also tested the effects of the side of the Match card (left/right), and whether the Match was presented first or second. Neither of the effects were significant in either of the two age groups (all χ² < 3.28, p > 0.07).

4.3.4. The development of location updating performance

Given the similarities between the location updating tasks in Experiments 2 and 3, we tested the effect of task difficulty and age on performance (based on first looks). We performed a univariate 2×2 ANOVA with Age Group (20-month-olds, 25-month-olds) and Experiment (Experiment 2: Translational movement, Experiment 3: Shuffle movement). This analysis showed no significant effect of Experiment (F(1,86) = 0.293, p = 0.59, η² = 0.004), but a significant main effect of Age Group (F(1,86) = 5.337, p = 0.023, η² = 0.06). The interaction between Age Group and Experiment was not significant (F(1,83) = 0.243, p = 0.623, η² = 0.003).

6. General Discussion

6.1. Summary of results

The current study examined infants’ multiple identity tracking (MIT) abilities. We used a modified Delayed Match Retrieval (DMR) task, where virtual cards were shown face-up with to-be-tracked objects on their faces, then turned face down. After a retention phase, a ‘sample’ card was revealed that matched one of the two previously seen cards, and infants were expected to find (i.e., make an anticipatory saccade to) the matching card. In Experiment 1, we replicated previous results using DMR (Cheng et al., 2019; Kaldy et al., 2016), showing that infants were able to remember the identities of two objects, without movement. We then examined infants’ MIT ability as cards moved during the retention phase. Cards either had translational movement (Experiment 2), when perceptual grouping could help tracking, or, more challengingly, ‘shuffled’ to new locations (Experiment 3), which involved orthogonal trajectories.

Overall, 20-month-olds showed at-chance performance at the task when movement was involved, while 25-month-olds performed significantly better than chance even with translational or shuffle movement³. While the overall ANOVA did not show a significant effect of movement type, the effect size in the 25-month-olds was larger with translational movement (Experiment 2: d = 1.21, 1.21; for results based on first looks and longer looks, respectively) than with shuffle movement (Experiment 3: d = 0.69, 0.92). We also analyzed how infants’ performance changed over trials. We did not expect a learning effect in the conditions that showed an above-chance performance overall (20-month-olds in Experiment 1, 25-month-olds in Experiment 2 and 3). However, with the 20-month-olds in Experiment 2, there was some indication that success was not entirely out of reach with the simpler translational movement, as these younger infants showed a learning trend (R² = 0.325) over the block of 12 trials⁴.

6.2. The cognitive demands of multiple identity tracking

Keeping the identities of two (static) objects in VWM is within the abilities of infants by 20 months of age. Beyond the current findings in Experiment 1, this has been shown in infants under 1 year of age in paradigms with widely different task demands by Feigenson and Carey (2003), Kaldy and Leslie (2003), and Ross-Sheehy et al. (2003). It is clear, though, that tracking those identities once movement is introduced increases cognitive demands. Several adult studies have measured how much more challenging MIT tasks are compared to MOT tasks, i.e., the “content deficit” (Horowitz et al., 2007). Hollingworth and Rasmussen (2010) presented a surprising finding with adults in a paradigm that was similar to the present study. They contrasted VWM performance in two conditions when two colored squares swapped positions during a delay. For example, participants saw a red square in a 5 o’clock position and a green square in an 11 o’clock position. Then the colors disappeared and the square outlines moved along a circular path such that they ended up taking up the other square’s position. Participants were then asked to identify the colors in the updated versus the original positions. They found that, contrary to the predictions of theories of MOT, binding of color to the original locations of the objects was stronger (adults reported the objects’ color faster and more accurately) than binding to the updated locations, despite clear visual evidence that the objects had moved. This shows that encoding of identity-location bindings at the original position was more robust, and location updating reduced the robustness of memory for identity in adults. This effect may contribute to the lower performance of 20- versus 25-month-olds in Experiments 2 and 3.

In terms of the complexity of the required location updating, we contrasted two types of movements. Infants viewing the translational motion of Experiment 2 could potentially exploit perceptual grouping to reduce cognitive demands, just as adults (Woodman, Vecera, & Luck, 2003; Yantis, 1992). Previous studies have also demonstrated that infants during the second year of life can use cues (e.g. perceptual/conceptual similarity) to help them track more items (Feigenson & Halberda, 2008; Rosenberg & Feigenson, 2013). In contrast, the shuffle movement of Experiment 3 likely required infants to track two trajectories separately. Adult MOT studies found that more changes in target trajectories impaired tracking performance (Ericson & Beck, 2013). Though we did not see a main effect of movement type in the ANOVA analysis, there were hints that tracking of the shuffle movement was less robust. Results in that experiment were mixed, with above-chance performance found only in the total look duration measure, but not with the first look measure. Further work is required to investigate this finding, but we speculate that occasionally, participants who were less certain about the location of updated bindings made an initial, likely random, first look (thereby diminishing first-look performance), followed by a corrective look to the other location.

6.3. The brain mechanisms underlying identity tracking

Neuroimaging studies with children in an MIT task have not yet been conducted. However, several neuroimaging studies have investigated the cortical mechanisms underlying MIT performance in adults. Despite differences in the behavioral paradigms (in one, object identities were constantly changing during tracking, in the other, objects rotated behind occluders), two independent fMRI studies concluded that activation in frontal regions (in the inferior precentral sulcus) and posterior parietal areas (intraparietal sulcus and superior parietal lobule) were responsible for MIT (Lyu, Hu, Wei, Zhang, & Talhelm, 2015; Takahama, Miyauchi, & Saiki, 2010). Another recent study directly contrasted brain activity patterns underlying MOT and MIT tasks (Nummenmaa, Oksama, Glerean, & Hyönä, 2017). There, while both tasks activated the same extended frontoparietal circuits identified above, in the MIT tasks there also was an additional load-dependent activity increase in the lateral prefrontal cortex and ventral temporal areas known to subserve object recognition and VWM. Given this, we would hypothesize that it is the maturation of the network described by Nummenmaa and colleagues, over the course of the second year of life, that accounts for older infants’ better performance in our task.

6.4. Limitations and future directions

Our study is the first, to our knowledge, to study MIT in infants. In that context, it is important to note limitations of our study, and places for follow-up work. First, by adding the movement trajectory in Experiments 2 and 3, we also lengthened the retention interval relative to Experiment 1 (from 1 to 2.2 s). A few prior studies have looked at the effect of the length of delay on VWM performance in infants and children, although not with 1.5-2-year-old infants. Kaldy and Leslie (2005) found that 6-month-old infants’ VWM performance was not affected when the delay was increased from 4 to 7 seconds (with one object) or, at the same age, from 3 to 5 seconds (with two locations, O’Gilmore & Johnson (1995)), and similarly, 7-year-old children’s VSTM performance was not affected by a change from 1.5 to 2.3 seconds (with up to 4 objects) (Shimi & Scerif, 2017). Taken together, while longer delays may ultimately have a negative impact on performance, it is unlikely that the amount of the difference here (1 to 2.2 s) had a substantial impact on performance. Furthermore, no results have suggested an interaction effect with age, so any negative effect of an increased delay is unlikely to have created the relatively poor performance of the 20- versus 25-month-olds in Experiments 2 and 3.

Secondly, we evaluated the development of MIT by contrasting tasks where tracking was required (the translational and shuffle movements of Experiments 2 and 3, respectively) to one where objects remained static (Experiment 1). Here, we interpreted older infants’ facility with MIT, and younger infants relatively poor performance, as evidence that the ability to track identity is developing in this age range. However, it is possible that younger infants’ lower performance in Experiments 2 and 3 was not due to identity-tracking challenges, per se, but instead some distraction caused by the movement itself. There is no prior work that we know of that directly addresses this question, but, in general, infants’ representations of unfamiliar objects can be fragile (Horst & Samuelson, 2008). Future work should investigate this possibility by testing a condition where the cards move, inconsequentially (e.g. jiggling in place), or move and return to their original locations.

Lastly, it is possible that infants may use a ‘process of elimination’ cognitive strategy to reduce the need to remember both objects. Through disjunctive reasoning (the principle of mutual exclusivity), instead of remembering both to-be-remembered objects (A and B), infants could just remember one (say, A) and then if the Sample is B, use a ‘find NOT-A’ strategy (Halberda, 2003; Markman, Wasow, & Hansen, 2003) to identify the correct, matching card. A recent study suggests that infants as young as 12 months of age are able to reason this way (Cesana-Arlotti et al., 2018), however, they did not spontaneously do it in a DMR task at 14 months (Hochmann et al., 2016). Our task was not designed to isolate the use of this specific strategy, and future studies (for instance, with more to-be-tracked objects) should test at what age can children spontaneously apply this strategy in this type of VWM task.

6.5. Closing remarks

In addition to the empirical contributions discussed above, our study also makes some important methodological contributions. As demonstrated here, the DMR task can be tailored to different ages in early development (8- and 10-month-olds (Kaldy et al., 2016); 13-month-olds (Cheng et al., 2019); and 20- and 25-month-olds in the current study), making it possible to study visual attention and VWM for objects across a wide age range from infancy to early childhood. The task does not require receptive language skills, making it ideal for preverbal populations, or children with language delay or deficit (e.g., children with Autism Spectrum Disorder). Furthermore, this task can be easily modified to parametrically study the multiple cognitive components of VWM (i.e. processing speed, sustained attention, inhibitory control) by manipulating task parameters (e.g., encoding time, number of objects, retention duration).

Figure 5. Summary of average VWM performance across three experiments.

Percent correct performance based on first looks (darker bars) and percent correct performance based on longer looks (lighter bars) in Experiment 1, 2, and 3. Error bars indicate SEM, and whiskers indicate 95% confidence interval. (* p < 0.05; n.s. - not significant.)

Table 1.

Summary of results across three experiments

Experiment	N	Age in months (SD)	Valid trials, out of 12 (SD)	Performance based on first looks [CI]	Performance based on longer looks [CI]
Exp. 1: No movement (20-m-olds)	22	19.3 (1.42)	8.9 (2.5)	56% (*) [0.51, 0.62]	56% (*) [0.51, 0.62]
Exp. 2: Translational (20-m-olds)	22	19.8 (1.06)	7.2 (2.8)	50% (n.s.) [0.42, 0.57]	49% (n.s.) [0.42, 0.57]
Exp. 2: Translational (25-m-olds)	22	24.8 (1.36)	6.6 (3.0)	61% (*) [0.53, 0.69]	59%(*) [0.51, 0.67]
Exp. 3: Shuffle (20-m-olds)	22	19.4 (1.24)	6.9 (2.2)	48% (n.s.) [0.39, 0.56]	47% (n.s.) [0.39, 0.57]
Exp. 3: Shuffle (25-m-olds)	22	25.0 (1.80)	8.1 (2.9)	57% (n.s.) [0.48, 0.65]	58% (*) [0.50, 0.67]

Highlights.

• We tested infants’ (20- and 25-month-olds) ability to track the identities of two moving objects during occlusion - an infant version of Multiple Identity Tracking (MIT).

• 25-month-olds successfully tracked two objects both in a simple translational movement condition, and a more complex ‘shuffle’ movement condition.

• 20-month-olds only succeeded when no movement was involved during retention.

• Our results show that the ability to track multiple identities in visual working memory is present by 25 months of age.