Abstract
The relationship between object files and visual working memory (VWM) was investigated in a new paradigm combining features of traditional VWM experiments (color change detection) and object-file experiments (memory for the properties of moving objects). Object-file theory was found to account for a key component of object-position binding in VWM: With motion, color memory came to be associated with the new locations of objects. However, robust binding to the original locations was found despite clear evidence that the objects had moved. This latter binding appears to constitute a scene-based component in VWM, which codes object location relative to the abstract spatial configuration of the display and is largely insensitive to the dynamic properties of objects.
Visual working memory (VWM) is a component of the human working memory system that supports the maintenance of perceptual information from a small subset of objects in a visual scene (for a review, see Luck, 2008).1 VWM has been demonstrated to play an important role in change detection and perceptual comparison (Hyun, Woodman, Vogel, Hollingworth, & Luck, in press; Luck & Vogel, 1997), eye movement control (Chelazzi, Miller, Duncan, & Desimone, 1993; Hollingworth, Richard, & Luck, 2008), visual memory across saccades (Henderson & Anes, 1994; Henderson & Siefert, 2001; Irwin, 1992; Irwin & Andrews, 1996), visual search (Desimone & Duncan, 1995; Olivers, Meijer, & Theeuwes, 2006; Soto, Humphreys, & Heinke, 2006; Woodman, Luck, & Schall, 2007), and mental imagery (Hyun & Luck, 2007). It has become increasingly evident that a brief visual memory is required to perform many apparently simple perceptual operations, such as efficiently orienting the eyes to a particular object in the visual field (Hollingworth et al., 2008).
Much of the existing research on VWM has focused on the topic of feature binding. In particular, researchers have examined the means by which ventral stream features, such as color, shape, and orientation, are bound into a coherent object representation and maintained in VWM (Allen, Baddeley, & Hitch, 2006; Gajewski & Brockmole, 2006; Irwin & Andrews, 1996; Johnson, Hollingworth, & Luck, 2008; Luck & Vogel, 1997; Olson & Jiang, 2002; Raffone & Wolters, 2001; Treisman & Zhang, 2006; Wheeler & Treisman, 2002; Y. Xu, 2002a, 2002b). However, an equally important form of binding in VWM is the association of object representations with particular locations, i.e., object-position binding. If one puts one’s cell phone down on a counter and then decides to pick it up a moment later, a memory representation that binds together the perceptual features of the cell phone (silver, small, rectangular) might play a role in guiding attention to the location of objects that match the remembered perceptual properties of the cell phone (Duncan & Humphreys, 1989; Olivers et al., 2006; Soto, Heinke, Humphreys, & Blanco, 2005). However, a memory representation that also maintains the binding of the cell phone to a particular scene location would allow attention, the eyes, and ultimately the hand to be oriented directly and efficiently to the appropriate location, without necessarily requiring visual search for the cell phone at all (Hollingworth, 2009). Although visual memory for perceptual features and memory for locations are often studied separately, VWM will be of the greatest practical utility if one remembers both the visual form and the location of a task-relevant object. Understanding the binding of objects to locations in VWM is therefore central to understanding how VWM is used to support intelligent real-world behavior.
The study of object-position binding also has the potential to illuminate how an object’s perceptual features come to be bound in the first place. In Treisman’s theory of visual binding (Treisman, 1988), position serves as the nexus for associating the different features of an object, providing a point of correspondence among separate representations of object features such as color, size, and shape. In addition, spatial position uniquely specifies a particular object, allowing the features of one object to be segregated from those belonging to other objects (Kahneman, Treisman, & Gibbs, 1992); although there might be multiple objects in a scene with the property “green”, there is typically only one object present at any given location. In this manner, position can establish “object-hood”, with features of one object, and only the features of that object, bound into a coherent representation through shared location. Kahneman et al. have referred to this type of spatially defined object representation as an “object file”.
Furthermore, the binding of objects to locations in memory plays an important role in establishing perceptual continuity within the dynamic environments of everyday experience. Object motion, observer motion, occlusion, saccades, and blinks disrupt perceptual input almost constantly, creating gaps in the availability of visual information. A brief visual memory is required to bridge these gaps and to establish the mapping of objects visible before and after the disruption. Memory for the binding of objects to locations (i.e., object files) is one means by which the visual system establishes object continuity (Hollingworth & Franconeri, in press), with an object treated as continuous across disruption if its position is consistent with the interpretation of a single, persisting entity (Kahneman et al., 1992).
In sum, the ability to bind object properties to locations in memory plays an important role in multiple aspects of visual behavior, including the formation and maintenance of object representations, the mapping of objects across visual disruptions and change, and the guidance of attention and the eyes to the remembered locations of task-relevant items. Historically, object-position binding in brief forms of visual memory has been studied in two separate lines of inquiry: research probing the spatial structure of VWM and research examining the nature of object-file representations. In the next sections, we review these two literatures. Then, we report a series of experiments designed to integrate them.
Binding objects to locations in VWM
Within the literature on VWM, initial studies of spatial binding sought to estimate memory capacity for object-position pairings. Irwin (1992; Irwin & Andrews, 1996) found that participants could remember the locations of 3–4 letters across a brief delay. A similar estimate was obtained for the binding of real-world objects to scene locations (Zelinsky & Loschky, 2005). Subsequent studies have examined the spatial structure serving to organize VWM representations of objects into a larger representation of a scene (Hollingworth, 2007; Jiang, Olson, & Chun, 2000; Olson & Marshuetz, 2005). Jiang et al. presented a set of four color patches, followed by a test array of colors. The task was color change detection. At the test, the locations of the color patches either remained that same as at study or were scrambled. Memory for color was significantly impaired by scrambling, indicating that color was not stored independently of location in VWM. Not all spatial changes impaired memory for perceptual features, however. In a different experiment, Jiang et al., systematically expanded the array from study to test. Test objects were presented at previously unoccupied screen positions, but the relative inter-object positions (i.e., the configuration of the array) were preserved. Unlike scrambling, expansion of the array did not significantly impair change detection performance, suggesting that object position was coded relative to the abstract configuration of objects in the scene. Consistent with these findings, Hollingworth (2007; see also Olson & Marshuetz, 2005) translated an array of objects from study to test in a change detection paradigm, and object position was likewise found to be coded in an array-relative reference frame. Participants viewed a study and test array of natural, real-world objects in a change detection task requiring memory for object orientation. Change detection performance was reliably higher when the entire array was translated between study and test (which changed the target’s absolute position but preserved its relative position within the array) than when all array objects except the target were translated (which preserved the target’s absolute position but changed its position relative to the array). Taken together, these results suggest that objects in VWM are bound to locations, that object position is coded in array-relative coordinates, and that the larger contextual representation is fairly abstract, representing the spatial configuration of the array but not necessarily the precise metric relationships among objects.
Object Files
The second line of evidence relevant to object-position binding in brief forms of visual memory comes from the literature on object files. Kahneman et al. (1992) proposed that when a visible object is attended, it recruits a spatial index, which marks its location. The spatial index itself has no content, but serves simply as a pointing device, similar to the spatial indexes proposed by Pylyshyn and Storm (1988). When one observes perceptual features of an object, such as its color, shape, or identity, those features are retained briefly in memory and are bound to the spatial index marking that object, forming the object file. When object position changes, and there is spatiotemporal information to link the initial and changed locations (e.g., when an object moves smoothly to a new location), the spatial index is automatically updated so that it marks the new position of the object, and correspondence is established between the two states of the object. Because the properties of the object are bound directly to the spatial index, memory for those properties automatically comes to be associated with the object’s new location, a process of object updating. Such a mechanism would enable the visual system to maintain a continuous representation of the features of an object despite dynamic changes in object position.
The primary empirical support for the object-file framework has come from the object reviewing paradigm introduced by Kahneman et al. (1992). In this procedure, two boxes are displayed. The boxes are briefly filled by two preview letters. The letters are removed, and the boxes move to new locations. One test letter is then displayed, and the participant’s task is to name the letter. When the test letter is the same as one of the preview letters, it can appear within the same box as in the preview (same object condition) or in the other box (different object condition). The central finding is faster naming latency in the same object condition, a same-object benefit. The same-object benefit is taken as evidence that the position index was updated with motion so as to mark the new object location, and remembered properties of the original object came to be associated with the new location, facilitating perceptual processing when they matched the features of the target letter appearing at that location.
The object-file framework has played an important role in subsequent research across much of visual cognition. Extensions of the original object file studies have addressed the nature of the content of the object file (Gordon, 2004; Gordon & Irwin, 1996; Henderson, 1994; Henderson & Anes, 1994; Henderson & Siefert, 2001; Mitroff, Scholl, & Noles, 2007), the factors that control the correspondence and updating processes (Hollingworth & Franconeri, 2009; Hollingworth et al., 2008; Mitroff & Alvarez, 2007; Mitroff, Scholl, & Wynn, 2004; Moore & Enns, 2004; Moore, Mordkoff, & Enns, 2007; Noles, Scholl, & Mitroff, 2005; Richard, Luck, & Hollingworth, 2008; Scholl & Pylyshyn, 1999; Scholl, Pylyshyn, & Feldman, 2001), and the application of the object-file framework to related domains, such as memory across saccades (Henderson, 1994; Irwin, 1992), scene perception (Hollingworth, 2007; Hollingworth & Henderson, 2002; Zelinsky & Loschky, 2005), and infant object processing (Feigenson & Carey, 2005; F. Xu, Carey, & Welch, 1999).
Do VWM and Object Files Constitute the Same System?
To what extent can the large literature on object files be used to draw inferences about the nature of object-position binding in VWM? Is there a unitary VWM system that handles all forms of object memory and position binding? Are object files and VWM simply two terms describing the same system? A good deal of circumstantial evidence suggests that they might be. First, the Kahneman et al. (1992) object reviewing paradigm depends on memory, because the letters are not visible during the motion of the boxes. Given the relatively short ISI between presentation of the preview letters and the test letter, it is natural to suppose that VWM supports letter memory in this paradigm, although it is possible that the letter stimuli are encoded verbally and do not draw upon VWM. Second, position-consistency effects similar to those observed by Kahneman et al. are also observed across eye movements (Henderson & Anes, 1994; Henderson & Siefert, 2001), and memory across eye movements has be demonstrated, independently, to depend on VWM (Hollingworth et al., 2008). Finally, the binding of perceptual features in VWM could be accomplished by associating each of those features with a common spatial position, consistent with the original framework of Treisman and colleagues (Treisman, 1988). In this view, object files and VWM constitute two ways of describing the same system: Object representations in VWM are simply those objects for which attention has bound separate features to a common spatial position (Treisman & Zhang, 2006; Wheeler & Treisman, 2002).
Perhaps the strongest evidence that VWM and object files constitute the same system comes from a recent study by Flombaum and Scholl (2006). Flombaum and Scholl generated a paradigm in which colored objects moved back and forth on the computer screen, periodically disappearing and emerging from behind a set of occuluders. As an object emerged, its color could potentially have changed, and participants monitored for such changes. It is likely that this change detection depended on VWM (Luck & Vogel, 1997). Flombaum and Scholl manipulated spatiotemporal parameters of the moving objects. Factors that led participants to perceive a single object moving behind the occluder also facilitated the detection of color changes, suggesting dependence on a common, object-based representation for the computation of correspondence across occlusion and for the maintenance of object color in VWM.
Despite the evidence reviewed above, there are reasons to be cautious about equating object files and VWM. First, the original Kahneman et al. (1992) proposal was that object files are relatively early visual representations that directly support the conscious perception of object persistence. VWM, on the other hand, maintains higher-level visual object representations, abstracted away from precise sensory representation in early vision (Phillips, 1974). And VWM does not support conscious perception in any direct way. Whereas early forms of visual sensory memory generate conscious experience of visible persistence (Averbach & Coriell, 1961; Sperling, 1960), VWM does not. In a standard change detection task with a 1000 ms ISI between study and test arrays, one does not experience the objects as visibly persisting during the delay. Because VWM does not directly support conscious visual perception, it would seem unlikely that objects files (at least as they were originally conceived by Kahneman et al.) and VWM constitute the same system.
However, it is important to consider that the original object reviewing paradigm of Kahneman et al. (1992) did not probe conscious perception; all that was measured was priming. Thus, claims regarding the updating of memory with motion need not be inextricably bound with claims regarding conscious perception. In fact, the updating of object memory with motion and the conscious perception of object continuity appear to be dissociable. Mitroff, Scholl, and Wynn (2005) presented ambiguous motion displays in which two moving objects could be perceived either as bouncing off each other or as passing by each other. When participants reported perceiving that the objects passed by, the same-object benefit was observed not at the appropriate locations given this experience but rather at the locations consistent with an interpretation of bouncing; the experience of object continuity and the binding of observed features to object locations diverged. Thus, it is presently unclear whether the Kahneman et al. reviewing paradigm provides a valid method for understanding the perceptual mechanisms generating conscious perception of object persistence. For the present purposes, we will consider this paradigm only as a means to measure the binding of object features to locations and the updating of that binding with motion, the focus of our study.
Second, and most importantly for the present study, the object file framework does not provide a natural explanation for the results of key studies examining object-position binding in VWM. In Jiang et al. (2000), color change detection was unimpaired by systematic spatial expansion of the object array. In Hollingworth (2007) and Olson and Marshuetz (2005), array translation had minimal or no effect on object memory. In these experiments, there was no continuity between the object locations before and after the test. When the object arrays were either expanded or translated, the test objects did not occupy their original absolute locations, there was no object motion that could have linked the original positions to the new positions, and the ISIs between study and test arrays were too long to generate apparent motion. Thus, there was no direct spatiotemporal information that would have caused the remembered object properties to become associated with the new test locations. Yet, change detection when the array was systematically expanded or translated was significantly higher than performance when the objects were shifted, randomly, to new locations. Such results indicate a form of spatial representation in VWM more abstract than that necessary to support precise spatial tracking and updating of objects as they move, the standard domain of the object file system.
The Present Study
The purpose of the present study was to examine whether the object-file framework can account for object-position binding in VWM. We tested whether a key representational feature of the object-file framework—the updating of property memory to a new location with object motion—is found for visual memory under conditions that are known to engage the VWM system. To this end, we developed a hybrid paradigm that bridged the VWM and object file literatures. We combined the reviewing paradigm of Kahneman et al. (1992) with a color memory task that is known to depend on VWM (Luck & Vogel, 1997). If the object representations in VWM are object files, then we should observe the standard Kahneman et al. updating effect for objects that are being held in VWM.
The basic paradigm is illustrated in Figure 1. Participants first saw a set of four randomly chosen digits (not shown in Figure 1) that they repeated aloud throughout the trial so as to suppress verbal recoding of the colors. Next, a random array of four boxes was displayed. The boxes were filled briefly by four different colors (represented as different fill patterns in Figure 1). The colors were removed, and the boxes moved over 500 ms of animation so as to trade locations. Then, a set of test colors appeared. All colors were the same as in the initial set, or one color was replaced by a new color. Participants reported whether all of the test colors were “same” or whether one test color was “new”. Note that the basic task (brief memory for a small set of colors) and the memory retention interval (a total of 1000 ms) are the same as those used in studies examining VWM (Luck & Vogel, 1997). In addition, the recognition memory instructions are equivalent to asking participants whether one of the colors changed to a new color. Thus, the present task was equivalent to a standard, VWM change-detection task.
Figure 1.
Overview of the events in a trial of the hybrid VWM/object-file paradigm. In this figure and in all subsequent figures, the colors that appeared in the boxes are represented as different fill patterns.
To probe the updating of color memory with object motion, we manipulated the spatial correspondence between the initial colors and test colors in three conditions (Figure 2). In the updated condition, the colors in the test array appeared in the locations that were consistent with the motion of the objects. Consider the sample trial illustrated in Figure 2. In the initial array, the color represented by the checkerboard pattern appears in top-left box. That box then moves to the bottom position. In the test array, the checkerboard color appears in the appropriate bottom position (i.e., in the same object in which it appeared originally). The same is true of all four colors in the array. This condition was analogous to the “same object” condition in Kahneman et al. (1992). In the original condition, the test colors appeared in the same locations in which they appeared in the initial array (i.e., as if there had been no object motion). This condition allowed us to test whether color memory had been fully updated, so that memory for the colors became associated with the new locations and was no longer associated with the original locations. Finally, in the no correspondence condition, the test colors appeared in locations that did not correspond either to the updated positions or to the original positions. This condition was analogous to the “different object” condition of Kahneman et al. It served as a baseline against which to compare performance in the updated and original conditions.
Figure 2.
Top panels. Illustration of the method and correspondence conditions in Experiment 1. Bottom panels. Accuracy and RT data as a function of correspondence condition for same trials alone (left) and all trials (right). Error bars in this and in subsequent figures are 95% confidence intervals based on the error term of the correspondence effect.
If the object representations in VWM are object files, then performance in the updated condition should be higher (i.e., faster RT and/or higher accuracy) than performance in the no correspondence condition. This follows from similar logic as informed the original Kahneman et al. (1992) study. If color memory is updated with motion so that each color comes to be associated with the new location of the box in which it appeared, this should have at least two effects at test. First, comparison operations should be relatively efficient in the updated condition, because the remembered positions of the array colors would align with the positions of the test colors. Yellow in the test array would appear at the same location to which yellow is bound in memory, enabling participants to determine “same” for yellow with a single comparison. However, in the no correspondence condition, multiple comparisons would be required to determine “same” for a particular test color, because there would be no spatial correspondence between the remembered locations of the initial colors and the locations of the test colors. The result of these additional comparisons would be longer “same” RTs in the no correspondence condition than in the updated condition, and, if multiple comparisons generated a higher probability of error, lower accuracy. Second, interference generated by the test display should be minimized in the updated condition. If, in the updated condition, the remembered colors are updated with motion and align with the test colors, then there should be minimal interference generated by the appearance of the test display. This should benefit performance, particularly on “same” trials. In contrast, if the remembered locations and test locations of the colors are misaligned (as expected in the no correspondence condition), the test display colors are likely to interfere with memory for the initial colors (Makovski, Sussman, & Jiang, 2008), impairing performance. An advantage for the original condition over the no correspondence condition would replicate the original Kahneman et al. same-object benefit, but in a paradigm that we can be confident depends directly on VWM.
In addition to testing this main prediction, we examined the extent to which the updating of object property memory with motion is absolute. That is, does updating occur in the strong sense that the remembered object properties come to be associated with the new location of the object and are no longer associated with the original location of the object? Kahneman et al. (1992) did not make explicit claims about binding to the original location. This is likely due to that fact that their method (in which objects moved to previously unoccupied locations) did not allow them to test performance at the original location. However, evidence from inhibition of return (IOR) suggests that the properties of dynamic objects can be associated both with the new locations of the objects and with the original locations of the objects (Tipper, Driver, & Weaver, 1991; Tipper, Weaver, Jerreat, & Burak, 1994). This work implies that memory for the attentional properties of objects can have a component that is updated with object motion and a component that is not updated with motion. Whether this type of structure applies to memory for the perceptual attributes of objects has not been tested before. In the present study, objects moved so as to trade locations, enabling us to test the binding of memory for the perceptual features of an object to the original object location. If VWM representations remain bound, at least in part, to the original object location, then performance in the original condition should be higher than performance in the no correspondence condition. If, however, updating is complete, with no persistence of binding to the original location, then performance in the original condition should be equivalent to that in the no correspondence condition.
These predictions were tested in the first section of the study (Experiments 1 and 2). To preview the results, we replicated the original Kahneman et al. (1992) same-object benefit, with better performance in the updated condition than in the no correspondence condition. Object-file theory does appear to capture important properties of object-position binding in VWM. However, robust binding to the original location was also observed, and this binding was actually stronger than binding to the updated locations, with a consistent advantage for the original condition over both the no correspondence condition and the updated condition.
Having observed the phenomenon of robust binding to the original position, in the second section of the study (Experiments 3–5), we probed the relationship between original-position binding and previous studies that have examined spatial structure in VWM (Hollingworth, 2007; Jiang et al., 2000; Olson & Marshuetz, 2005). Consistent with theses earlier studies, binding to the original location was found to survive translation and array expansion. Colors were bound to the original positions relative to an abstract, configural representation of the array and not to the original absolute locations. Together, these data suggest that there are two mechanisms of object-position binding in VWM: One that is sensitive to online, dynamic object events and which updates color memory with motion (the object-file system) and a second that is relatively insensitive to motion and codes memory for the properties of objects within an abstract representation of the spatial configuration of the array.
Experiment 1
In Experiment 1, we implemented the basic VWM/object-file paradigm illustrated in Figures 1 and 2.
A discussion of several features of the general method is necessary. The original Kahneman et al. (1992) experiments used naming latency of a single test letter as the primary dependent measure. More recent work on object files has modified this paradigm (Kruschke & Fragassi, 1996; Mitroff et al., 2004). In the modified method, two letters are displayed in two boxes, the empty boxes move to new locations, and a single test letter appears that either was or was not one of the two letters presented earlier. Participants report “old” or “new”. In accordance with the original naming latency method, RT is faster when an old letter appears in the same object in which it appeared originally versus when it appears in the different object (when a new letter is presented, it doesn’t have any history of being associated with either object, and the data from these trials are discarded). The modified object-reviewing paradigm is ideal for the present study, because it explicitly requires memory for the features of objects.2
The present task is functionally equivalent to this modified object-file paradigm, except colors were used instead of letters and four test items were presented instead of one. This latter feature of the method was designed to maximize sensitivity to spatial correspondence effects and also to replicate the whole-array test conditions of Luck and Vogel (1997). When only a single test stimulus is presented in a standard object-file paradigm, participants need only compare one test item with memory. However, when the four test colors are presented in the “same” condition of the present method, all four must be compared with memory, potentially compounding the effects of spatial consistency (see also Hollingworth & Franconeri, 2009). Indeed, the consistency effects on RT in the present paradigm were considerably larger than those found in other object-file paradigms, and complementary effects on accuracy were observed as well.
Method
Participants
Forty participants from the University of Iowa community completed the experiment. They either received course credit or were paid. All participants reported normal or corrected-to-normal vision.
Stimuli
Each object array consisted of four boxes, outlined in black. The boxes subtended .68°, and the outline contours were .06° wide. The screen background was set to a uniform gray (RGB: 148,148,148). The objects could appear in a 7.32° × 7.32° square region at screen center. Object positions were chosen randomly with the following constraints. Each object was no closer than .58° from the center of the screen, and the centers of each box were at least 2.0° from each other. The colors that appeared in each box filled the box and subtended .58°. Colors were drawn randomly without replacement from a set of seven: red, blue, yellow, light blue, light green, violet, and white.
When the boxes moved to trade locations, the choice of which objects went to which locations was determined randomly, with the constraint that during motion, the centers of two objects could never be closer than .85°, which ensured that the objects never overlapped during their motion (this was true of all the experiments in the study). If a particular set of object locations would have generated overlap during motion, four new object locations were chosen randomly, as described above. This process was repeated until a stimulus array that met the non-overlap criterion was obtained.
The motion itself was divided into 50 frames, each presented for 10 ms, for a total of 500 ms of motion animation. The objects traveled in straight lines to their new locations. The distance between the initial object location and its new location was divided evenly into 50 units. Thus, objects arrived at their new locations at the same time. Object that had to travel a longer distance moved faster than objects that had to travel a shorter distance. The mean distance traveled was 3.9° (SD = 1.29°) at a mean rate of 7.8°/s.
In the test image for “same” trials, all four colors were the same as in the initial image. For “new” (i.e., changed) trials, one color was replaced by a new color drawn randomly from the remaining three colors that had not appeared in the initial image.
In the updated condition, the test colors appeared in the locations consistent with the motion of the array. In the original condition, the test colors appeared in their original locations. In the no correspondence condition, the test colors appeared in locations that were neither the appropriate updated locations nor the original locations.
With four objects, there were two possible no-correspondence locations. Twenty-four of the 40 participants completed Experiment 1a, which was a 3 (updated, original, no correspondence) × 2 (color: same, new). The no correspondence locations were chosen at random from the two possible sets of locations. The remaining participants completed Experiment 1b, a 4 (updated, original, no correspondence A, no correspondence B) × 2 (color: same, new) design, which included both sets of no correspondence locations. In this latter design, any given initial array color was equally likely to appear at any of the four test locations. There were no significant differences between performance in the two sub-experiments, and the data were combined.
Apparatus
The stimuli were displayed at a resolution of 800 by 600 pixels by 24-bit color on a 17-in. video monitor with a refresh rate of 100 Hz. The initiation of image presentation was synchronized to the monitor’s vertical refresh. Responses were collected using a serial button box. The presentation of stimuli and collection of responses was controlled by E-Prime software running on a PC-compatible computer. The room was dimly illuminated by a low-intensity light source. A forehead rest maintained a viewing distance of 80 cm.
Procedure
Participants were tested individually. Each participant was given a written description of the experiment along with a set of instructions. Participants were informed that they would view a series of object arrays made up of boxes. On each trial, a set of colors would appear briefly in the boxes, the boxes would move, and a second set of colors would appear, with all colors old or one color new. They were instructed that the positions of the colors were irrelevant to the task. Participants pressed one button if the colors were all same and a different button if one was new. They were instructed to respond as accurately and as quickly as possible.
On each trial, participants first saw a screen instructing them to press a button to start the trial. This screen also contained four randomly chosen digits. Participants began repeating the four digits out loud at a rate of at least two digits per second and continued repeating the digits throughout the trial (the experimenter monitored digit repetition to ensure that participants complied). When ready, participants pressed a button to initiate the trial. The following sequence of events occurred (Figure 1): Uniform gray field, 700 ms; initial array of empty boxes, 1000 ms; colors within each box, 250 ms; empty boxes, 300 ms; motion animation, 500 ms; empty boxes, 200 ms; test colors, until response. The total ISI between the initial array of colors and the test array was 1000 ms, as in most studies of VWM. Button response terminated the trial. There was a 1500 ms delay between trials.
After reviewing the instructions, participants were shown three demonstration trials, in which the trial events occurred at half speed. Participants were reminded that the positions of the colors in the test image were irrelevant to the task. Participants then completed a practice session of either 12 (Experiment 1a) or 16 (Experiment 1b) trials, two in each condition. This was followed by the experiment session. In Experiment 1a, participants completed 240 trials, 40 in each of the six conditions created by the 3 (updated, original, no correspondence) × 2 (same, new) design. In Experiment 1b, participants completed 360 trials, 45 in each of the eight conditions created by the 4 (updated, original, no correspondence A, no correspondence B) × 2 (same, new) design. Trial order was determined randomly.
Results
Data for Experiment 1 are reported in Figure 2. Performance was recorded for the “same” and “new” trials. On “new” trials, when a new color is displayed, the spatial correspondence between this color and the initial array is undefined, which complicates interpretation of these trials. Thus, the primary analyses were conducted over “same” trials alone, which is the standard approach in the literature (Kruschke & Fragassi, 1996; Mitroff et al., 2004, 2005). However, in the following experiments, there were often performance differences among correspondence conditions for “new” trials. In addition, three of the four colors on “new” trials did appear in the initial array, providing useful evidence about spatial correspondence. Thus, in addition to reporting data from “same trials”, we also report data for all trials.
Accuracy analyses over “same” trials were computed using percentage correct on the same-new task. For accuracy analyses over all trials, the hit and false alarm rates were used to calculate d′ as a measure of sensitivity to change. In the very few cases that a participant had a hit rate of 1.0 or a false alarm rate of 0.0 (at which d′ is undefined), performance was treated as ½ trial incorrect. Percentage correct and A′ data produced the same pattern of results as d′. The raw accuracy and RT data for Experiments 1–4 are reported in Appendix A.
RT analyses were conducted over correct trials. RTs beyond 2.5 standard deviations from the participant’s mean in each condition were eliminated as outliers. No more than 3.3% of the data was eliminated in any experiment. Outlier trimming did not alter the pattern of results in any experiment.
Same Trials
The effect of correspondence condition (updated, original, no correspondence) was reliable, both for accuracy, F(2, 78) = 43.2, p < .001, and for RT, F(2, 78) = 16.7, p < .001. Consistent with the hypothesis that VWM and object files constitute the same system, recognition memory (i.e., change detection) performance was reliably more accurate, F(1, 39) = 33.0, p < .001, and faster, F(1, 39) = 8.18, p = .007, in the updated condition than in the no correspondence condition. However, we also observed robust binding to the original locations. Performance was reliably more accurate, F(1, 39) = 33.0, p < .001, and faster, F(1, 39) = 38.9, p < .001, in the original condition compared with the no correspondence condition, an original-position advantage. Surprisingly, there was a robust advantage for the original condition over the updated condition, both for accuracy, F(1, 39) = 12.0, p = .001, and for RT F(1, 39) = 7.56, p = .009.
All Trials
The data for all trials (same and changed) produced a similar pattern of results. There was a reliable effect of correspondence condition, both on accuracy (d’), F(2, 78) = 13.2, p < .001, and RT, F(2, 78) = 5.57, p = .005. Performance was reliably more accurate, F(1, 39) = 17.1, p < .001, and faster, F(1, 39) = 9.28, p = .004, in the updated condition than in the no correspondence condition. Performance was reliably more accurate, F(1, 39) = 21.6, p < .001, and faster, F(1, 39) = 8.34, p = .006, in the original condition compared with the no correspondence condition. The original and updated conditions did not differ for accuracy, F < 1, or for RT, F < 1.
In summary, benefits were observed both on accuracy and RT for the updated and original conditions over the no correspondence condition. In addition, when considering “same” trials alone, there was a benefit for the original condition over the updated condition. However, this latter effect was reversed in the “new” trials (see Appendix A), generating no reliable differences between updated and original conditions when considering the full data set.3
Discussion
Performance in a hybrid VWM/object-file task that required memory for the colors of moving objects was more accurate and faster when the test colors appeared in positions consistent with their motion (updated condition) than when there was no correspondence between the positions of the initial and test colors (no correspondence condition). This result replicates the Kahneman et al. (1992) same-object benefit but in a paradigm known to depend on VWM. Thus, the object-file framework accounts for key features of object-position binding in VWM.4
If the present method had included only the updated and no-correspondence conditions, it would have been reasonable to conclude that the object-file framework provides a complete account of object-position binding in VWM and that object-files and VWM representations are identical. However, the results from the original condition showed robust binding to the original locations of the colors despite clear evidence of object motion (and in a paradigm that places minimal demands on spatial tracking), potentially indicating a component of VWM that is relatively insensitive to motion.
Experiment 2
Before we can be confident that binding to the original location reflects a general property of VWM, we sought to eliminate a plausible alternative explanation for the original-position advantage observed in Experiment 1. The four colors used in Experiment 1 might simply have exceeded participants’ capacity for updating color memory with motion. This possibility is consistent with evidence that participants have a limited capacity for keeping track of the surface feature properties of moving objects (Horowitz et al., 2007; Pylyshyn, 2004; Rasmussen & Hollingworth, 2008; Saiki, 2003; Scholl, Pylyshyn, & Franconeri, 1999). Thus, in Experiment 1, color memory might have been updated for a subset of the objects, and for these objects, there may have been no binding of color to the original locations. However, for the remainder of the objects, binding was not updated, leaving color memory bound to the original locations and thus generating the original-position advantage. In this view, if the number of objects to update does not exceed updating capacity, there should no longer be any advantage for presenting the colors at the original positions.
There is at least some existing evidence regarding the capacity for updating color memory with motion. Saiki and colleagues (e.g., Saiki, 2003) generated an estimate of approximately 2 objects in a paradigm in which objects were periodically occluded. Rasmussen and Hollingworth (2008) estimated updating capacity in a task quite similar to the present task. Participants saw four colors (always red, green, blue, and yellow) that appeared in boxes. One, two, three, or all four of the boxes were cued as “to-be-tracked”. The empty boxes moved to previously unoccupied locations. After motion, one of tracked boxes was probed, and participants reported which of the four colors originally appeared in that box. Using assumptions similar to those used to estimate object capacity in VWM (Cowan et al., 2005; Pashler, 1988), Rasmussen and Hollingworth found an updating capacity of approximately two objects, consistent with the Saiki results. In particular, updating performance when only one or two objects had to be tracked was near ceiling (98% correct and 96% correct, respectively). Thus, we can be confident that it is possible for people to update memory for the colors of at least two moving objects.
In Experiment 2, set size was reduced to two objects (see Figure 3) to ensure that the task did not exceed updating capacity. The two objects traveled on semi-circular paths to trade locations, and the correspondence conditions were limited to updated and original. (The no-correspondence condition was not possible with two objects.) The color memory demands of this experiment were well within standard VWM capacity estimates of 3–4 objects (Luck & Vogel, 1997). In addition, the tracking of object motion for two objects is well within estimates of spatial tracking capacity of 4–5 objects (Pylyshyn & Storm, 1988). If updating is complete, and the properties of an updated object are no longer associated with the original location, we should observe an advantage for the updated condition over the original condition.
Figure 3.
Top panels. Illustration of the method and correspondence conditions in Experiment 2. Bottom panels. Accuracy and RT data as a function of correspondence condition for same trials alone (left) and all trials (right).
Method
Participants
Twenty-four new participants from the University of Iowa community completed the experiment.
Stimuli
Each object array consisted of two boxes. The boxes were the same as in Experiment 1. Each array was constructed as follows. A virtual circle was defined around central fixation, with a randomly determined radius between 1.7° and 3.1°. The first object was placed at a random position on the circumference of this circle. The second box was placed on the circle 180° from the first position. A set of 10 colors was used. The colors were approximately evenly spaced around the color wheel (RGB: 255,242,0; 247,148,29; 237,28,36; 175,33,142; 86,46,145; 17,73,240; 0,173,239; 3,230,144; 0,128,0; 141,198,63). When a new color was added, it was a color adjacent to one of the initial colors. Thus, the change magnitude in Experiment 2 was smaller than that in Experiment 1, so as to counteract the performance benefit expected by the reduction in set size. Initial color arrays were chosen randomly with the constraint that each color must have at least one adjacent color value empty (so that each object could possibly be replaced by an adjacent color).
Motion was either clockwise or counterclockwise, randomly chosen. Unlike the straight-line motion of previous experiments, two objects required curved motion so that they never overlapped. Each object moved on a semicircular pathway, 180° around the circumference of the virtual circle. The motion sequence lasted 600 ms. The mean distance traveled for each object was 7.6° (SD = 1.3°) as a mean rate of 12.7°/s. All other motion parameters were the same as in Experiment 1.
Procedure
The sequence of events in a trial was the same as in previous experiments. Participants completed four demonstration trials, 12 practice trials (three in each of the four conditions), and an experiment session of 240 trials. In the experiment session, there were 60 trials in each of the four conditions created by the 2 (updated, original) × 2 (same, new) design. The entire session lasted approximately 40 min.
Results
Figure 3 shows percentage correct and RT results as a function of correspondence condition.
Same trials
There was a reliable accuracy advantage for the original condition over the updated condition, F(1, 23) = 20.0, p < .001, and a reliable RT advantage for the original condition over the updated condition, F(1, 23) = 5.46, p = .029.
All trials
There was a reliable d’ advantage for the original condition over the updated condition, F(1, 23) = 5.97, p = .023, and a reliable RT advantage for the original condition over the updated condition, F(1, 23) = 5.50, p = .028.5
Discussion
Despite the minimal color memory and tracking demands of a two-object task, and despite evidence suggesting a color-updating capacity of approximately two objects (Rasmussen & Hollingworth, 2008), memory performance was superior in the original condition compared with the updated condition. Binding to the original location was actually more pronounced that the updating of binding to the new location, even though the objects clearly traveled to new locations. Such a result is surprising when one considers that we often need to update the properties of objects when those properties are not currently visible. Consider the following example. One notices that Person A, standing near the fireplace, is wearing a red tie. That person turns around (so that the tie is no longer visible) and walks to the other side of the room. Simultaneously, Person B moves to occupy the original position near the fireplace. One certainly would expect that memory for the color of the tie would be updated so that it continued to be associated with Person A at the new location (and the benefit for the updated condition over the no correspondence condition in Experiment 1 confirms this expectation). However, it is surprising that memory for the color of the tie would be more robustly bound to the original location near the fireplace, especially when that location is now occupied by a different person.
Follow-up Experiments
Given this counterintuitive finding, we conducted four follow-up experiments designed to probe the robustness of binding to the original location. In each case, the basic paradigm was modified with the intention of improving updating performance and potentially eliminating the advantage at the original location. Because none of these manipulations produced a pattern of data qualitatively different from that of Experiment 1, we have condensed the report of these experiments. In each follow-up experiment (N = 24), the basic method was similar to Experiment 1, except three-object arrays were used (instead of four-object arrays). Three objects were chosen as the minimum number that allowed us to examine all three correspondence conditions: updated, original, and no correspondence. The boxes moved in straight lines so as to trade locations (as in Experiment 1).The colors were drawn from the same set used in Experiment 2, and color changes were always to an adjacent color. The data from the four follow-up experiments are reported in Appendix B.
In Follow-up Experiment 1, perceptual support for updating was provided by having the colors remain visible during the first third of the motion sequence. Despite clear evidence that the colors had moved away from the original locations, performance remained reliably faster and more accurate in the original condition than in the updated and no correspondence conditions.
In Follow-up Experiment 2, the proportions of trials in the three correspondence conditions were modified so that the test colors were four times more likely to appear in the updated locations than in either of the other two possible locations. This should have provided significant incentive for participants to update memory for the colors with object motion, potentially eliminating binding to the original locations. Yet, the pattern or results was unchanged, with a reliable advantage for the original condition over the updated and no correspondence conditions.
In Follow-up Experiment 3, participants completed a concurrent explicit updating task. On a subset of trials, one color appeared at test, and participants had to report whether it was in the appropriate location given the motion of the objects (i.e., at the appropriate updated location). These trials were intermixed randomly with the standard trials, providing incentive to update color memory on all trials. In addition, the colors remained visible during the first third of motion. There was some evidence that updating performance improved, as the advantage for the original condition over the updated condition was largely eliminated. However, the advantage for the original condition over the no correspondence condition remained robust, demonstrating persistent binding to the original locations.
Finally, in Follow-up Experiment 4, the method was essentially the same as in Experiment 1, except each test display contained only a single test color in one of the three boxes (instead of test colors in all of the boxes). The test color was either “same” or “new”. When it was the same, it could appear either in the updated, original, or no correspondence location. “New” trials were not analyzed, because the single test color did not correspond to any of the preview colors. The use of a single test color was intended to encourage participants to adopt a local, object-based comparison rather than a global comparison, potentially making the method more sensitive to object-based updating6. In addition, the single test color generated a paradigm similar to the original Kahneman et al. (1992) method, in which only a single test stimulus appeared. Nevertheless, a significant original-position advantage remained. There was no performance difference between the original and updated conditions, although the accuracy trend favored the original condition. In addition, there was no significant difference between the updated and no correspondence conditions, although the numerical trends favored the updated condition. Critically, however, there was a reliable accuracy advantage for the original condition over the no correspondence condition and a numerical trend in the same direction for RT. Thus, the use of a single test color did not generate any observable improvement in updating performance, and it did not eliminate the original-position advantage.
Follow-up Experiment 4 generated smaller effects compared with those in Experiment 1. The magnitude of the basic same-object advantage (updated minus no correspondence) was 20 ms, which is similar to other studies using a single test stimulus (e.g., Mitroff & Alvarez, 2007; Mitroff et al., 2004). In Experiment 1, however, there was a reliable, 30-ms same-object advantage, along with a large 7% effect on accuracy. The cause of this difference is likely to lie in the number of test stimuli. In Follow-up Experiment 4, there was only a single test stimulus and therefore only one comparison that could have been influenced by spatial correspondence. In contrast, four comparisons were necessary to determine “same” in Experiment 1, each of which could have been influenced by spatial correspondence, increasing the sensitivity of the paradigm. The benefits of using multiple test stimuli in this type of paradigm are discussed further in Hollingworth and Franconeri (2009).
In summary, binding to the original location is a highly robust phenomenon. It is not simply a consequence of limited capacity in the updating of object properties with motion (Horowitz et al., 2007; Pylyshyn, 2004; Rasmussen & Hollingworth, 2008; Saiki, 2003; Scholl et al., 1999), because the original-position advantage remained robust when set size was reduced to only two objects. In addition, it does not appear to be strongly influenced by strategic factors. These findings do not minimize evidence that color comes to be associated with the new locations; the advantage for the updated condition over the no correspondence condition in the present experiments clearly shows that color memory does come to be associated with the new locations of moving objects. However, in spite of such sensitivity to object motion, color memory remains robustly bound to the original locations. A component of object-position binding in VWM appears to relatively insensitive to motion.
This finding is similar to evidence showing that there are two components to IOR: one that is sensitive to motion and “travels” with an object, and one that remains associated with the original attended location (Tipper et al., 1991; Tipper et al., 1994). Such evidence has typically been interpreted as indicating an object-based component and a space-based component to IOR, with the latter reflecting binding to the original absolute location in manner that is independent of object or scene structure. We will argue that, in the present case, a simple dichotomy between object-based and space-based coding is insufficient to explain the binding of object features to the locations in VWM. In Experiments 3–5, we show that remembered features are not necessarily bound to the original absolute locations. Instead, remembered features are bound to the original location defined relative to an abstract representation of the spatial layout of the array. In this manner, binding to the original location is sensitive to scene structure and is best understood as a scene-based coding of object location, rather than a purely space-based coding.
Experiment 3
Having observed the phenomenon of original-position binding in the first section of this study, the second section (Experiments 3–5) was devoted to understanding more precisely the manner in which object properties are bound to the original object locations. Given the observed insensitivity to motion, a clear possibility is that original-position binding reflects the type of position coding observed in studies of spatial structure for non-moving arrays of objects (Hollingworth, 2007; Jiang et al., 2000; Olson & Marshuetz, 2005). In these experiments, array objects were shifted to new locations in a single step across an ISI of ~1000 ms. Spatial changes that disrupted object-to-object spatial relationships impaired VWM for an object’s perceptual features. However, changes in the absolute positions of objects that did not disrupt abstract object-to-object spatial relationships (systematic expansion, translation) had little or no effect on VWM performance. Object-file theory already has difficulty accounting for these effects, because there was no linking motion with which to establish correspondence between the initial positions and the test positions. Thus, binding to the original positions plausibly depends on a system of spatial representation that codes object position in an abstract, array-relative coordinate system and is relatively insensitive to object motion.
In Experiment 3, the basic paradigm was modified so that at the terminus of object motion, the entire array had been translated from its original position (Hollingworth, 2007; Olson & Marshuetz, 2005). That is, instead of moving so as to trade absolute locations, the objects moved so as to create the same configuration of locations but offset from the absolute locations of the original array. This is illustrated in Figure 4. The object-to-object spatial relationships were preserved from study to test, but none of the same absolute locations were occupied from study to test.
Figure 4.
Top panels. Illustration of the method and correspondence conditions in Experiment 3. Note that the test array preserves the same spatial configuration as the study array, but translated from the original array positions. Bottom panels. Accuracy and RT data as a function of correspondence condition for same trials alone (left) and all trials (right).
In the updated condition, the test colors occupied the appropriate locations given the motion. In the configuration condition, the test colors occupied the same array-relative locations as they had at study (the analog of the original condition in previous experiments). In the no correspondence condition, the test colors did not correspond either to the updated locations or to the original configuration positions.
If the robust advantage for the original condition in previous experiments reflects a visual memory system that binds object properties to positions coded relative to the abstract configuration of the array (Hollingworth, 2007; Jiang et al., 2000; Olson & Marshuetz, 2005), then performance in the configuration condition should be superior to that in the no correspondence condition; the advantage for the original position within the configuration should be preserved despite metric changes to the array. If, however, the original position advantage was generated by a mechanism that binds object properties to absolute locations (i.e. a purely space-based code), then there should be no benefit associated with the configuration condition, because the test locations were not the same absolute locations as those occupied at study.
Method
Participants
Sixteen participants completed Experiment 3.
Stimuli
Each object array consisted of three boxes. The boxes were the same as in Experiment 1, and the color set and procedure for introducing a change were the same as those used in Experiment 2.
To form the object arrays for Experiment 3, a virtual circle was defined around central fixation, with a radius of 2.8°. The first box was placed at a random position on this circle. The other boxes were evenly spaced around the circle, separated by 120°. Then, each object was jittered both horizontally and vertically. The distance of the jitter was randomly chosen between −1.0° and +1.0° for each spatial dimension. The array translation was implemented as follows. On each trial, one of the three object positions (position 1) was randomly chosen, and a second object position (position 2) was also randomly chosen. During motion, the positions of objects were translated so that the configuration of objects was offset from the study configuration. This offset was calculated so that position 1 in the test configuration (we will call it position 1') was midway between positions 1 and 2 of the study configuration. Using Figure 4 as an example, we shall label the three study array positions 1 (top left), 2 (middle right), and 3 (bottom). And we shall label the same relative positions in the test configuration as 1', 2', and 3'. Position 1' lay midway between positions 1 and 2, and positions 2' and 3' were offset the corresponding distance and direction. During motion in the Figure 4 example, the object at position 1 moved to position 3', the object at position 2 to position 1', and the object at position 3 to position 2'. Thus, objects traded positions within the configuration, and the entire configuration was offset ½ of the distance between positions 1 and 2. None of the absolute positions of the initial and test arrays overlapped. And, as in all previous experiments, the boxes never overlapped during their motion. The mean distance traveled by the objects was 5.0° (SD = 0.9°) at a mean rate of 10.0°/s. All other motion parameters were the same as in Experiment 1.
Procedure and Design
The sequence of events in a trial was the same as in previous experiments. Participants completed a practice block of 12 trials, evenly divided among the main conditions. In the experiment session, participants completed 360 trials, 60 in each of the 6 conditions created by the 3 (updated, configuration, no correspondence) × 2 (same, new) design.
Results and Discussion
Figure 4 shows mean accuracy and RT results for Experiment 3.
Same Trials
The effect of correspondence condition (updated, configuration, no correspondence) was reliable, both for accuracy, F(2, 30) = 18.8, p < .001, and RT, F(2, 30) = 7.03, p = .003. Again, performance was reliably more accurate, F(1, 15) = 6.32, p = .024, and faster, F(1, 15) = 6.53, p = .022, in the updated condition than in the no correspondence condition, consistent with the prediction of object file theory. Of central importance, however, there was a reliable advantage when the test colors appeared in the original positions within the configuration, despite the fact that these were not the original absolute locations. Performance was reliably more accurate, F(1, 15) = 24.5, p < .001, and faster, F(1, 15) = 9.04, p = .007, in the configuration condition compared with the no correspondence condition. Further, performance was reliably more accurate, F(1, 15) = 20.1, p < .001, and marginally faster, F(1, 15) = 3.24, p = .092, in the configuration condition compared with the updated condition.
All Trials
The effect of correspondence condition was reliable for accuracy (d’), F(2, 30) = 14.9, p < .001, but not for RT, F(2, 30) = 1.65, p = .21. There was a reliable advantage for the updated condition over the no correspondence condition for RT, F(1, 15) = 1.34, p = .027, but not for accuracy, F(1, 15) = 1.06, p = .319. There was a reliable accuracy advantage for the configuration condition over the no correspondence condition, F(1, 15) = 29.5, p < .001, and a marginal advantage on RT, F(1, 15) = 3.40, p = .085. Finally, there was a reliable accuracy advantage for the configuration condition over the updated condition, F(1, 15) = 16.7, p < .001, but no difference on RT, F < 1.
Despite the fact that the absolute locations of the initial and test arrays did not overlap, we again observed an advantage for the original position condition (within the configuration) over the other two conditions. The component of object-position binding in VWM that is relatively insensitive to motion appears to code object locations in an array-relative reference frame.
Experiment 4
As an even stronger test of abstract, array-relative coding, in Experiment 4 we added an array expansion manipulation. There were two motion conditions. In the translation condition, the motion of the objects generated a translation of the array, replicating Experiment 3. In the translatiom+expansion condition, the motion of the objects not only produced a translation of the entire array but also a systematic expansion of the array (Jiang et al., 2000). The translation+expansion condition is illustrated in Figure 5.
Figure 5.
Illustration of the method and correspondence conditions in the translation+expansion condition of Experiment 4.
Method
Participants
Thirty-two participants completed Experiment 4.
Stimuli
The arrays in Experiment 4 were constructed in a manner similar to that in Experiment 3. First, a random position was chosen within a 5.7° × 5.7° region at screen center. This position served as the center of the initial array. The first box position was chosen at a random angular offset (1–360°) and at a randomly chosen distance (between 1.7° and 4.0°) from the central position. Then, the second object was chosen by adding a random angular offset, between 90° and 150°, to the first, again with a randomly chosen distance, between 1.7° and 4.0°, from array center. The third box position was chosen in the same manner but relative to the second position. The translation was computed in the same manner as in Experiment 3. The mean distance traveled by the objects was 4.8° (SD = 2.5°) at a mean rate of 9.6°/s.
In the translation+expansion condition, the objects moved so that when they arrived at the test locations, the configuration was expanded, with each box 1.0° further from the array center than in the initial array, as shown in Figure 5. The mean distance traveled by the objects was 5.3° (SD = 2.6°) at a mean rate of 10.6°/s.
Procedure and Design
The sequence of events in a trial was the same as in previous experiments. Participants completed a practice block of 12 trials, evenly divided among the main conditions. In the experiment session, participants completed 360 trials, 30 in each of the 12 conditions created by the 2 (expansion, no expansion) × 3 (updated, configuration, no correspondence) × 2 (same, new) design.
Results
Translation only
The results from the translation condition of Experiment 4 are reported in Figure 6.
Figure 6.
Experiment 4. Accuracy and RT data as a function of correspondence condition for same trials alone (left) and all trials (right). The top row shows results from the translation only condition. The bottom row shows results from the translation+expansion condition.
Same Trials
The effect of correspondence condition (updated, configuration, no correspondence) was reliable, both for accuracy F(2, 62) = 20.9, p < .001, and RT, F(2, 62) = 14.6, p < .001. There was an accuracy advantage for the updated condition over the no correspondence condition, F(1, 31) = 11.9, p = .002, but no difference on RT, F(1, 31) = 1.79, p = .19. The advantage for the configuration condition over the other two conditions was again robust. Performance was reliably more accurate, F(1, 31) = 33.4, p < .001, and faster, F(1, 31) = 28.9, p < .001, in the configuration condition compared with the no correspondence condition. Further, performance was reliably more accurate, F(1, 31) = 12.1, p = .001, and faster, F(1, 31) = 15.9, p < .001, in the configuration condition compared with the updated condition.
All Trials
The effect of correspondence condition was reliable, both for accuracy (d’), F(2, 62) = 8.16, p < .001, and for RT, F(2, 62) = 8.97, p < .001. There was a marginal advantage for the updated condition over the no correspondence condition on accuracy, F(1, 31) = 3.24, p = .081, but no advantage on RT, F(1, 31) = 1.97, p = .171. Performance was reliably more accurate, F(1, 31) = 18.6, p < .001, and faster, F(1, 31) = 23.0, p < .001, in the configuration condition compared with the no correspondence condition. Further, performance was reliably more accurate, F(1, 31) = 4.35, p = .045, and faster, F(1, 31) = 6.92, p = .013, in the configuration condition compared with the updated condition.
Translation + Expansion
The results from the translation + expansion condition of Experiment 4 are reported in Figure 6.
Same Trials
The effect of correspondence condition was reliable, both for accuracy F(2, 62) = 21.1, p < .001, and RT, F(2, 62) = 6.52, p = .003. There was an accuracy advantage for the updated condition over the no correspondence condition, F(1, 31) = 7.38, p = .011, but no difference on RT, F < 1. Performance was reliably more accurate, F(1, 31) = 48.3, p < .001, and faster, F(1, 31) = 9.75, p = .004, in the configuration condition compared with the no correspondence condition. Further, performance was reliably more accurate, F(1, 31) = 12.8, p = .001, and faster, F(1, 31) = 8.36, p = .007, in the configuration condition compared with the updated condition.
All Trials
The effect of correspondence condition was reliable for accuracy F(2, 62) = 6.35, p = .003, but not for RT, F < 1. There was no performance difference between the updated and no correspondence conditions, either on accuracy, F(1, 31) = 1.63, p = .211, or RT, F < 1. Performance was reliably more accurate, F(1, 31) = 13.9, p < .001, but not reliably faster, F(1, 31) = 1.36, p = .251, in the configuration condition compared with the no correspondence condition. Finally, performance was reliably more accurate, F(1, 31) = 4.45, p = .043, but no faster, F < 1, in the configuration condition compared with the no correspondence condition.
Comparison between translation and translation + expansion conditions
In Experiment 4, the addition of array expansion to array translation had minimal influence on the magnitude of the correspondence effect. The accuracy and RT data from “same” trials were entered into a 2 (translation, translation+expansion) × 3 (updated, configuration, no correspondence) repeated measures ANOVA. For accuracy, there was a reliable (though numerically very small) effect of expansion, F(1, 31) = 4.93, p = .034, with 88.4% correct in the translation condition and 87.1% correct in the translation+expansion condition. Expansion and correspondence condition did not interact, F < 1. For RT, there was no effect of expansion, F < 1, but there was a trend toward an interaction between expansion and correspondence, F(2, 62) = 2.79, p = .069. This marginal interaction was driven by a smaller configuration advantage in the translation+expansion condition than in the translation condition. This could indicate that expansion reduced the goodness of fit between the memory representation of the array and the test array. However, it was still the case that RT in the configuration condition remained numerically lower than RT in the two other conditions (and reliably so for same trials alone). In addition, the accuracy advantage for the configuration condition over the other two conditions remained reliable with array expansion, both for same trials and for all trials.
Discussion
The original position advantage observed in Experiments 1 and 2 was preserved when the array was systematically translated and expanded. Thus, the component of VWM that is relatively insensitive to object motion also appears to code object position within an abstract representation of the configuration of the array, consistent with earlier work on spatial structure in VWM (Hollingworth, 2007; Jiang et al., 2000; Olson & Marshuetz, 2005).
Experiment 5
In Experiments 3 and 4, the original-position advantage was preserved when relative position within the configuration was maintained, despite the fact that the original array-relative locations were not the original absolute locations. This suggests that binding to original location occurs in a scene-relative reference frame and is thus best understood as a scene-based coding rather than a purely space-based coding. However, neither experiment directly probed memory at the original absolute location, because all test objects occupied previously unoccupied absolute locations. If the component of VWM that is relatively insensitive to motion codes exclusively in an abstract, array-relative format, then one would not necessarily expect any binding at all to the original absolute location. Experiment 5 examined binding to the original absolute location using a converging array translation manipulation (see Figure 7).
Figure 7.
Top panels. Illustration of the method and correspondence conditions in Experiment 5. Bottom panel. Same-trial accuracy and RT data as a function of correspondence condition.
Three-object arrays were used. After the appearance of the initial colors, the array was translated. Unlike the array translation in Experiments 3 and 4, the object array maintained its original configuration throughout the motion. That is, the three objects were shifted as a coherent group, with the inter-object relationships maintained throughout the motion sequence. One object was randomly chosen to move to the location of one of the other objects in the array (in Figure 7, the object originally in the bottom right location moves to the location of the middle object). The other two objects moved the same distance and direction to new (previously unoccupied) locations. After motion, the same spatial configuration of objects was present. In addition, one location in the test image was the same as a location occupied in the initial image (i.e., there was one overlapping position). At test, one color square appeared, and again the task was to determine if the color was one of the old set or new.
The three correspondence conditions are illustrated in Figure 7. In the configuration condition, the test color that appeared in the overlapping position maintained the same position relative to the array configuration (in Figure 7, the color represented by the checkerboard is in the bottom-right position both in the initial array configuration and in the test array configuration). In addition, the test color was in the appropriate updated position given the motion. However, the test color was in a different absolute screen position in the initial array and test array. In the absolute condition, the test color that appeared in the overlapping position maintained the same absolute location from initial array to test array (in Figure 7, the color represented by vertical bars is in the same absolute location in the initial array and test array). However, the test color was not in the same position relative to the array configuration. Finally, in the no correspondence condition, the test color did not maintain its position relative to the configuration, did not maintain its absolute position, and was not in the appropriate updated position.
The critical comparison was between the absolute and no correspondence conditions. If the component of VWM that produced the original position advantage in previous experiments maintains at least some binding of color to absolute locations, performance should be higher in the absolute condition than in the no correspondence condition. However, if there is no binding at all to absolute locations, performance in the absolute condition should be no higher than that in the no correspondence condition. In addition, we expected highest performance in the configuration condition, because the advantages for consistent array-relative location and consistent updated location would both benefit this condition.
The basic design of Experiment 5 differed from most of the previous experiments in the use of a single test color. A single test color (instead of test colors in all the objects) was necessary to isolate memory performance at the one overlapping position, which was the only position at which we could present an item at the same absolute location in the initial and test arrays. If we had presented three test colors in the absolute condition, two of them would not have been in the original absolute locations, potentially masking an effect of binding to absolute locations.
Method
Participants
Twenty-four new participants from the University of Iowa community completed the experiment.
Stimuli
Each object array consisted of three boxes. The boxes were the same as in Experiment 1. A virtual circle was defined around central fixation, with a radius of 2.0°. The first box was placed at a random position on this circle. The other boxes were evenly spaced around the circle, separated by 120°. Then, each object was jittered both horizontally and vertically. The distance of the jitter was randomly chosen between −.85° and +.85° for each spatial dimension. Experiment 5 used the set of 7 colors from Experiment 1. Three colors were randomly chosen without replacement. When color changed, a new color was randomly chosen from the set of remaining colors.
On each trial, one of the three objects in the study array (object 1) was randomly chosen to move to the position of a second object (object 2), also randomly chosen. The position of object 2 was the overlapping position between the study array and the test array. The direction and distance necessary for object 1 to move to the position of object 2 was calculated. For the array motion, all three objects moved the same direction and distance (i.e., the array was systematically translated). All other motion parameters were the same as in Experiment 1. In the test array, object 1 now occupied the original absolute location of object 2. The two other objects occupied locations that had not been occupied in the initial array. The mean distance traveled by the objects was 3.5° (SD = 0.7°) at a mean rate of 7.0°/s.
In the test array, only one color was presented. It was either the same as a color in the initial array or new. Each of the three test locations was equally likely to contain the test color. And each of the three initial colors was equally likely to be the test color in the “same” condition. Thus, participants had no incentive to preferentially attend to any of the individual test locations.
Procedure
The sequence of events in a trial was the same as in previous experiments. Participants completed four demonstration trials, 18 practice trials, and an experiment session of 396 trials. In the experiment session, there were three possible test locations, and three possible colors that could appear at each location in the “same” trials, for a total of nine permutations of the “same” condition. In addition, three “new” trials occurred at each of the three locations, for nine permutations of the “new” condition. Participants completed 22 trials in each of the 18 permutations (for a total of 396). The position of central interest was the overlapping position, in which all three correspondence conditions were possible (configuration, absolute, and no correspondence). At the two non-overlapping positions, only the configuration and no correspondence conditions were possible. The entire session lasted approximately 50 min.
Results and Discussion
With a single test color, the correspondence between initial and test arrays on “new” trials was undefined, and thus the analyses were limited to “same” trials (the standard method in the object-file literature). Mean accuracy on “new” trials was 94.9%, and mean correct RT was 605 ms.
Same trials
Performance in the configuration and no correspondence conditions did not differ at the overlapping position compared with the two non-overlapping positions. Thus, for these two conditions, the data from all three positions were combined. Figure 7 shows percentage correct and RT results for the “same” trials as a function of correspondence condition. For percentage correct, there was not a reliable effect of correspondence condition, F < 1, although the numerical trend was toward an advantage for the configuration condition. For correct RT, there was a reliable effect of correspondence condition, F(2, 46) = 3.86, p = .028, with a reliable advantage for the configuration condition over the absolute condition, F(1, 23) = 5.36, p =.030, and a reliable advantage for the configuration condition over the no correspondence condition, F(1, 23) = 12.2, p = .002. Critically, there was no difference between the absolute and no correspondence conditions, F < 1.
If color was bound to the original absolute location, we should have observed an advantage for the absolute condition over the no correspondence condition. The absence of any such advantage suggests that there was no binding to the original absolute position. Thus, the component of VWM that generates the original-position advantage appears to code object location exclusively in an array-relative reference frame.
General Discussion
The literatures on VWM and object files have been centered around a common, core issue: How is information about visual objects maintained over brief perceptual disruption and change? In the present study, we examined the relationship between VWM and object files, asking whether the two are in fact equivalent. We developed a hybrid paradigm that probed short-term memory for object color (as in the VWM literature) and the updating of color memory with object motion (as in the object-file literature). Colors appeared briefly in a set of boxes, the empty boxes moved so as to trade locations, and test colors appeared, with all colors same or one color new (changed). The spatial correspondence between initial and test colors was manipulated (Figure 2). Test colors appeared either in the appropriate objects given the motion (updated condition) in the objects corresponding with the original positions of the colors (original condition) or in objects that corresponded with neither the updated nor the original positions (no correspondence condition).
There were two main findings in Experiments 1 and 2. First, performance was consistently superior in the updated condition compared with the no-correspondence condition, indicating that color memory was updated with motion so that the color came to be associated with the new location of the object. This finding is a direct replication of the Kahneman et al. (1992) same-object benefit, and this replication was observed in a paradigm for which we can be confident that object memory depends on the VWM system (Luck & Vogel, 1997). Thus, the object-file framework accounts for key features of object-position binding in VWM. With this connection, the literatures on objects files and VWM can be brought together and understood as investigating an overlapping set of mechanisms.
However, the overlap between VWM and object files is not complete. A key feature of object-position binding in VWM could not be explained by the object-file framework. Although color-to-object binding was indeed updated with motion, the most striking finding in Experiments 1and 2 was an advantage for the original condition over the other two conditions. In particular, binding to the original location of the object was more robust than binding to the new location of the object, despite clear perceptual evidence that the object had moved. This original-position advantage was largely insensitive to a variety of manipulations designed to improve updating performance. And binding to the original location could not be explained by an object-file mechanism with a highly limited updating capacity (in which only a subset of colors are updated with motion, the rest remaining bound to the original locations), because the original-position advantage was observed even when set size was reduced to two objects. Studies designed to estimate the capacity for updating color memory with motion have yielded an updating estimate of approximately two objects (Rasmussen & Hollingworth, 2008; Saiki, 2003). Thus, robust binding to the original location remained even when participants should have been able to update color memory for all visible objects.
In Experiments 3–5, we examined the nature of the spatial representation functional in generating the original-position advantage. We hypothesized that the component of VWM that is relatively insensitive to motion, generating the original-position advantage, is likely to be the same component probed in existing studies of spatial structure in VWM (Hollingworth, 2007; Jiang et al., 2000; Olson & Marshuetz, 2005). In these earlier experiments, object position in VWM was shown to be coded within an abstract, array-relative reference frame. Such abstract spatial structure would not support precise tracking of the absolute locations of objects and thus would be unlikely to show spatial updating with motion. Consistent with this hypothesis, the original-position advantage was found to be associated with array-relative locations within the abstract spatial configuration of the display. The original-position advantage (within the array configuration) survived both array translation and systematic expansion (Experiments 3 and 4), and there was no evidence of any binding at all to the original absolute location (Experiment 5).
Two Systems of Object-position Binding in VWM
A clear possibility raised by the present data is that there are two systems for the maintenance of object-position binding across brief delays and perceptual disruption. The first, the object-file system, is highly sensitive to the dynamic properties of objects and codes object location with sufficient precision to track the absolute positions of objects as they move (e.g., to support reaching or other motor interaction). However, this system has only a relatively sparse representation of the non-spatial perceptual features of objects, such as color, shape, or identity. To the extent that surface feature properties are encoded, they are updated so as to become associated with the new location of the object, consistent with the original Kahneman et al. framework, but this updating is highly limited.
This conception of the object-file system as supporting the spatial tracking of objects with only limited maintenance of visual surface feature properties is consistent with studies examining multiple-object tracking. Scholl, Pylyshyn, and Franconeri (1999) had participants track moving objects that differed in their surface features. At some point during motion, a tracked object was masked, and participants were asked to report the remembered surface features of that object. Performance on this task was quite poor; participants often failed to remember the surface feature properties of objects they were successfully tracking spatially, suggesting that the system responsible for tracking moving objects has a relatively sparse representation of object surface features. Similarly, Horowitz et al. (2007) and Pylyshyn (2004) found that participants’ capacity for spatial tracking was larger than their capacity for remembering the identities of tracked objects. Further, studies designed to estimate VWM capacity for the updating of object surface features and identity with motion (Rasmussen & Hollingworth, 2008; Saiki, 2003) have yielded estimates that are significantly smaller than 4–5 object estimates of spatial tracking capacity (Pylyshyn & Storm, 1988). Spatial tracking and MOT tasks tend to engage parietal regions of cortex in neuroimaging studies (Culham et al., 1998; Culham & Kanwisher, 2001), so it is plausible that the object-file system draws primarily from dorsal stream systems of spatial representation which maintain precise information about the locations of objects, are sensitive to motion, but have only limited access to ventral stream object features such as color, shape, or identity.
The second system of object-position binding in VWM would then maintain a more precise and robust representation of the surface features of an object, but would be relatively insensitive to object motion and would represent object position in an abstract format relative to the spatial context formed by the scene or array. A plausible locus for such a system is medial temporal lobe areas constituting the hippocampal complex, which receive strong input from ventral-stream regions coding the visual properties objects and their identities (e.g., Suzuki & Amaral, 1994). Damage to medial temporal lobe structures leads to deficits in memory for the binding of objects to scene locations (Parkinson, Murray, & Mishkin, 1988; Pigott & Milner, 1993). And a series of recent studies has shown that such deficits are evident even over relatively brief retention intervals, indicating a significant hippocampal contribution to VWM (Ezzyat & Olson, 2008; Hannula, Tranel, & Cohen, 2006; Olson, Moore, Stark, & Chatterjee, 2006; Olson, Page, Moore, Chatterjee, & Verfaellie, 2006). Spatial coding in medial temporal regions tends to be quite abstract, surviving metric changes that preserve object-to-object spatial relationships (O'Keefe & Conway, 1978; Shapiro, Tanila, & Eichenbaum, 1997). This component of VWM representation would likely be a precursor of longer-term scene representations coding the locations of objects relative to the intrinsic spatial structure of a scene (Mou & McNamara, 2002). In addition, abstract, array-relative coding of location would provide stability across changes in viewer position and orientation.
We propose, then, that binding to the original locations reflects a scene-based component of VWM. It is tempting to equate this scene-based component with earlier proposals of spacebased coding in perception and attention (Tipper et al., 1994). For example, Tipper et al. found that in the phenomenon of IOR, inhibition was associated both with the new location of a moved object and with the original location of that object. The latter effect was interpreted as reflecting a purely space-based system of selection, one that was independent of object or scene structure. However, in the present experiments, binding to the original locations was not purely spatial. Color memory was bound not to absolute locations but to object locations defined relative to the abstract spatial configuration of the array. Thus, the more accurate description of this type of binding is “scene-based”.
Further, it is important to note that there need not be any fundamental difference between binding to the original location in the present study and in studies examining IOR. In IOR experiments (Tipper et al., 1991; Tipper, Jordan, & Weaver, 1999; Tipper et al., 1994; Weaver, Lupianez, & Watson, 1998), objects traded absolute locations, and thus absolute and arrayrelative location were confounded (as in the present Experiments 1 and 2). Given evidence of significant overlap between IOR and VWM (Castel, Pratt, & Craik, 2003), we think it is likely that array translation and expansion manipulations would reveal that IOR at the original location is also scene-based rather than purely space-based, consistent with evidence for the importance of scene representations in IOR (Klein & MacInnes, 1999; Müller & Von Mühlenen, 2000; Takeda & Yagi, 2000)
In general, proposed dichotomies between “object-based” and “spaced-based” coding (or between “objects” and “locations”) have often exaggerated the extent to which objects and locations can be divorced from one another. For example, in the literature on visual attention, object- and space-based selection are unlikely to be dichotomous (Davis, Driver, Pavani, & Shepherd, 2000; Kramer & Jacobson, 1991; Kramer, Weber, & Watson, 1997; Kravitz & Behrmann, 2008; Vecera, 1994; Vecera & Farah, 1994). In the present study, we do not see the difference between object updating and original-location binding as a dichotomy between “objects” and “locations”. Even if memory is object-based in the sense that it is updated with object motion, memory is still associated with a location (the new location of the object). And the binding to the original location is not binding to a particular absolute location; it is binding to a particularly object location relative to the configuration of an array of objects. This does not allow for any straightforward division between objects and locations. Binding to the new location of an object and binding to the original location of an object are just two different mechanisms for coding the location of object properties.
Conclusion
In the present experiments, we found that the binding of objects to locations in VWM was formed by two components. First, we found evidence of a component consistent with the object-file framework of Kahneman et al. (1992). When an object moved, the remembered color of the object came to be associated with the new location of the object. Thus, the object-file framework can be understood as accounting for key properties of VWM. However, binding to the new location of the object was less robust than binding to the original location of the object, despite unambiguous perceptual evidence that the object had moved. It appears that object files are sensitive to motion, and they allow for the updating of object memory with changes in position, but this sensitivity to the dynamic properties of objects comes at the expense of memory for the surface feature properties of those objects. Robust binding of object surface features to the original location suggests a second component to object-position binding in VWM, which was found to survive metric alterations of the array that preserved abstract, object-to-object spatial relationships. This scene-based component maintains relatively accurate surface feature memory but at the expense of sensitivity to the dynamic properties of objects.
Supplementary Material
Acknowledgments
The research was supported by NIH grants R03MH65456 and R01EY017356.
Appendix A
Mean percentage correct and reaction time data for Experiments 1–4.
| Percentage Correct | Reaction Time (ms) | |||
|---|---|---|---|---|
| Same | New (Changed) | Same | New (Changed) | |
| Experiment 1 | ||||
| Updated | 87.9 | 78.5 | 791 | 791 |
| Original | 91.5 | 73.8 | 757 | 821 |
| No Correspondence | 80.9 | 80.1 | 820 | 803 |
| Experiment 2 | ||||
| Updated | 90.6 | 91.9 | 626 | 640 |
| Original | 94.2 | 91.0 | 595 | 638 |
| Experiment 3 | ||||
| Updated | 84.2 | 79.5 | 674 | 694 |
| Configuration | 90.7 | 78.5 | 655 | 710 |
| No Correspondence | 80.4 | 81.6 | 702 | 696 |
| Experiment 4 (Translation Only) | ||||
| Updated | 88.4 | 79.3 | 777 | 782 |
| Configuration | 93.5 | 77.2 | 717 | 791 |
| No Correspondence | 83.2 | 81.5 | 800 | 790 |
| Experiment 4 (Translation + Expansion) | ||||
| Updated | 86.7 | 77.8 | 768 | 784 |
| Configuration | 92.3 | 74.7 | 736 | 809 |
| No Correspondence | 82.2 | 79.6 | 777 | 790 |
Appendix B
Data from four follow-up experiments (see Experiment 2 Discussion section).
Footnotes
Aspects of this research were presented at the Sixth and Seventh Annual Meetings of the Vision Sciences Society, Sarasota, FL, May 2006 and May 2007, and at the Annual Meeting of the Experimental Psychology Society, Edinburgh, UK, July 2007.
Researchers have often used the term “visual short-term memory” to refer to this system. However, recent evidence that the content of VWM is under tight strategic control in the service of immediate task demands (Richard & Hollingworth, 2008) leads us now to prefer the term “visual working memory”.
The original naming latency version of the object-reviewing paradigm does not require memory. However, the test letter is much more likely to be one of the letters present in the preview than a letter not present in the preview, and participants therefore have an incentive to retain the preview letters in memory so as to anticipate the required response.
The random placement of objects generated significant differences in the distances between objects from trial to trial and thus generated differences in the speed of object motion. To examine the possible effect of motion speed on updating, we calculated the mean speed of the four objects on each same trial and grouped those trials by speed quartile. For the accuracy measure, speed quartile did not produce a reliable main effect, F(3,117) = 1.92, p = .13, nor did it interact with correspondence condition, F < 1. In the slowest quartile (mean speed = 6.3°/s), the advantage for the original condition over the no correspondence condition was 12.5%, and the advantage for the original condition over the updated condition was 4.5%. In the fastest quartile (mean speed = 9.5°/s), the advantage for the original condition over the no correspondence condition was 10.0%, and the advantage for the original condition over the updated condition was 3.7%. For the RT measure, speed quartile also did not produce a reliable main effect, F(3,117) = 1.06, p = .37, nor did it interact with correspondence condition, F < 1. In the slowest quartile, the advantage for the original condition over the no correspondence condition was 101 ms, and the advantage for the original condition over the updated condition was 30 ms. In the fastest quartile, the advantage for the original condition over the no correspondence condition was 26 ms, and the advantage for the original condition over the updated condition was 10 ms. Thus, the original position advantage was just as large or larger for trials on which the motion was relatively slow compared with trials on which the motion was relatively fast. This eliminates the possibility that the original position advantage was driven by a subset of trials on which relatively fast motion limited the ability to update object memory.
To ensure that this basic result extends to memory for the types of complex objects characteristic of real-world vision, we replicated Experiment 1 (N=24) using arrays of three natural objects (e.g., motorcycle, ice cream cone, and cell phone) drawn randomly from a set of 40 on each trial. The objects appeared in boxes, and the empty boxes moved to trade locations. In the test display, all objects were the same, or one object was replaced by a different object token (e.g., the original motorcycle was replaced by a different motorcycle). The results of Experiment 1 were replicated, with reliable advantages (both on accuracy and RT) for the updated and original conditions over the no correspondence condition. There was a trend toward better performance in the original condition than in the updated condition, but this was statistically reliable only for same-trial accuracy. This replication confirms that the abstract color memory task used in Experiment 1 taps the same visual systems functional in representing natural objects.
On possible concern with the design of Experiment 2 is that the objects tended to move faster in Experiment 2 (mean = 12.7°/s) compared with Experiment 1 (mean = 7.8°/s) which could have made updating more difficult in Experiment 2. We again conducted an analysis of the same trial data as a function of object speed quartile. There was no effect of speed quartile on accuracy, F < 1, and no interaction between speed quartile and correspondence condition, F < 1. The accuracy advantage for the original condition over the updated condition was 3.0% in the slowest quartile (mean speed = 9.8°/s) and 5.0% ms in the fastest quartile (mean speed = 15.6°/s). There was no effect of speed quartile on RT, F < 1, and no interaction between speed quartile and correspondence condition, F(3,69) = 1.10, p = .36. The RT advantage for the original condition over the updated condition was 63 ms in the slowest quartile and 27 ms in the fastest quartile. Thus, the benefit for the original condition was not driven by a subset of trials on which the object motion was relatively rapid.
We thank an anonymous reviewer for suggesting this possibility.
References
- Allen RJ, Baddeley AD, Hitch GJ. Is the binding of visual features in working memory resource-demanding? Journal of Experimental Psychology-General. 2006;135:298–313. doi: 10.1037/0096-3445.135.2.298. [DOI] [PubMed] [Google Scholar]
- Averbach E, Coriell AS. Short-term memory in vision. The Bell System Technical Journal. 1961;40:309–328. [Google Scholar]
- Castel AD, Pratt J, Craik FIM. The role of spatial working memory in inhibition of return: Evidence from divided attention tasks. Perception & Psychophysics. 2003;65:970–981. doi: 10.3758/bf03194827. [DOI] [PubMed] [Google Scholar]
- Chelazzi L, Miller EK, Duncan J, Desimone R. A neural basis for visual-search in inferior temporal cortex. Nature. 1993;363:345–347. doi: 10.1038/363345a0. [DOI] [PubMed] [Google Scholar]
- Cowan N, Elliott EM, Saults JS, Morey CC, Mattox S, Ismajatulina A, et al. On the capacity of attention: Its estimation and its role in working memory and cognitive aptitudes. Cognitive Psychology. 2005;51:42–100. doi: 10.1016/j.cogpsych.2004.12.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Culham JC, Brandt SA, Cavanagh P, Kanwisher NG, Dale AM, Tootell RBH. Cortical fMRI activation produced by attentive tracking of moving targets. Journal of Neurophysiology. 1998;80:2657–2670. doi: 10.1152/jn.1998.80.5.2657. [DOI] [PubMed] [Google Scholar]
- Culham JC, Kanwisher NG. Neuroimaging of cognitive functions in human parietal cortex. Current Opinion in Neurobiology. 2001;11:157–163. doi: 10.1016/s0959-4388(00)00191-4. [DOI] [PubMed] [Google Scholar]
- Davis G, Driver J, Pavani F, Shepherd A. Reappraising the apparent costs of attending to two separate visual objects. Vision Research. 2000;40:1323–1332. doi: 10.1016/s0042-6989(99)00189-3. [DOI] [PubMed] [Google Scholar]
- Desimone R, Duncan J. Neural mechanisms of selective visual attention. Annual Review of Neuroscience. 1995;18:193–222. doi: 10.1146/annurev.ne.18.030195.001205. [DOI] [PubMed] [Google Scholar]
- Duncan J, Humphreys G. Visual search and stimulus similarity. Psychological Review. 1989;96:433–458. doi: 10.1037/0033-295x.96.3.433. [DOI] [PubMed] [Google Scholar]
- Ezzyat Y, Olson IR. The medial temporal lobe and visual working memory: Comparisons across tasks, delays, and visual similarity. Cognitive, Affective, & Behavioral Neuroscience. 2008;8:32–40. doi: 10.3758/cabn.8.1.32. [DOI] [PubMed] [Google Scholar]
- Feigenson L, Carey S. On the limits of infants’ quantification of small object arrays. Cognition. 2005;97:295–313. doi: 10.1016/j.cognition.2004.09.010. [DOI] [PubMed] [Google Scholar]
- Flombaum JI, Scholl BJ. A temporal same-object advantage in the tunnel effect: Facilitated change detection for persisting objects. Journal of Experimental Psychology: Human Perception and Performance. 2006;32:840–853. doi: 10.1037/0096-1523.32.4.840. [DOI] [PubMed] [Google Scholar]
- Gajewski DA, Brockmole JR. Feature bindings endure without attention: Evidence from an explicit recall task. Psychonomic Bulletin & Review. 2006;13:581–587. doi: 10.3758/bf03193966. [DOI] [PubMed] [Google Scholar]
- Gordon RD. Attentional allocation during the perception of scenes. Journal of Experimental Psychology: Human Perception & Performance. 2004;30:760–777. doi: 10.1037/0096-1523.30.4.760. [DOI] [PubMed] [Google Scholar]
- Gordon RD, Irwin DE. What’s in an object file? Evidence from priming studies. Perception and Psychophysics. 1996;58:1260–1277. doi: 10.3758/bf03207558. [DOI] [PubMed] [Google Scholar]
- Hannula DE, Tranel D, Cohen NJ. The long and the short of it: Relational memory impairments in amnesia, even at short lags. Journal of Neuroscience. 2006;26:8352–8359. doi: 10.1523/JNEUROSCI.5222-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Henderson JM. Two representational systems in dynamic visual identification. Journal of Experimental Psychology: General. 1994;123:410–426. doi: 10.1037//0096-3445.123.4.410. [DOI] [PubMed] [Google Scholar]
- Henderson JM, Anes MD. Effects of object-file review and type priming on visual identification within and across eye fixations. Journal of Experimental Psychology: Human Perception and Performance. 1994;20:826–839. doi: 10.1037//0096-1523.20.4.826. [DOI] [PubMed] [Google Scholar]
- Henderson JM, Siefert ABC. Types and tokens in transsaccadic object identification: Effects of spatial position and left-right orientation. Psychonomic Bulletin & Review. 2001;8:753–760. doi: 10.3758/bf03196214. [DOI] [PubMed] [Google Scholar]
- Hollingworth A. Object-position binding in visual memory for natural scenes and object arrays. Journal of Experimental Psychology: Human Perception and Performance. 2007;33:31–47. doi: 10.1037/0096-1523.33.1.31. [DOI] [PubMed] [Google Scholar]
- Hollingworth A. Two forms of scene memory guide visual search: Memory for scene context and memory for the binding of target object to scene location. Visual Cognition. 2009;17:273–291. [Google Scholar]
- Hollingworth A, Franconeri SL. Object correspondence across brief occlusion is established on the basis of both spatiotemporal and surface feature cues. Accepted pending minor revision. Cognition. 2009 doi: 10.1016/j.cognition.2009.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hollingworth A, Franconeri SL. Object correspondence across brief occlusion is established on the basis of both spatiotemporal and surface feature cues. Accepted pending minor revision. Cognition. doi: 10.1016/j.cognition.2009.08.004. (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hollingworth A, Henderson JM. Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception and Performance. 2002;28:113–136. [Google Scholar]
- Hollingworth A, Richard AM, Luck SJ. Understanding the function of visual short-term memory: Transsaccadic memory, object correspondence, and gaze correction. Journal of Experimental Psychology: General. 2008;137:163–181. doi: 10.1037/0096-3445.137.1.163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Horowitz TS, Klieger SB, Fencsik DE, Yang KK, Alvarez GA, Wolfe JM. Tracking unique objects. Perception & Psychophysics. 2007;69:172–184. doi: 10.3758/bf03193740. [DOI] [PubMed] [Google Scholar]
- Hyun J-S, Luck SJ. Visual working memory as the substrate for mental rotation. Psychonomic Bulletin & Review. 2007;14:154–158. doi: 10.3758/bf03194043. [DOI] [PubMed] [Google Scholar]
- Hyun J-S, Woodman GF, Vogel EK, Hollingworth A, Luck SJ. The comparison of visual working memory representations with perceptual inputs. Journal of Experimental Psychology: Human Perception and Performance. doi: 10.1037/a0015019. (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Irwin DE. Memory for position and identity across eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1992;18:307–317. [Google Scholar]
- Irwin DE, Andrews RV. Integration and accumulation of information across saccadic eye movements. In: Inui T, McClelland JL, editors. Attention and performance XVI: Information integration in perception and communication. Cambridge, MA: MIT Press; 1996. pp. 125–155. [Google Scholar]
- Jiang Y, Olson IR, Chun MM. Organization of visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2000;26:683–702. doi: 10.1037//0278-7393.26.3.683. [DOI] [PubMed] [Google Scholar]
- Johnson JS, Hollingworth A, Luck SJ. The role of attention in the maintenance of feature bindings in visual short-term memory. Journal of Experimental Psychology: Human Perception and Performance. 2008:41–55. doi: 10.1037/0096-1523.34.1.41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kahneman D, Treisman A, Gibbs BJ. The reviewing of object files: Object-specific integration of information. Cognitive Psychology. 1992;24:175–219. doi: 10.1016/0010-0285(92)90007-o. [DOI] [PubMed] [Google Scholar]
- Klein RM, MacInnes WJ. Inhibition of return is a foraging facilitator in visual search. Psychological Science. 1999;10:346–352. [Google Scholar]
- Kramer AF, Jacobson A. Perceptual organization and focused attention: The role of objects and proximity in visual processing. Perception & Psychophysics. 1991;50:267–284. doi: 10.3758/bf03206750. [DOI] [PubMed] [Google Scholar]
- Kramer AF, Weber TA, Watson SE. Object-based attentional selection--Grouped-arrays or spatially invariant representations?: Comment on Vecera and Farah (1994) Journal of Experimental Psychology: General. 1997;126:3–13. doi: 10.1037//0096-3445.126.1.3. [DOI] [PubMed] [Google Scholar]
- Kravitz DJ, Behrmann M. The space of an object: Object attention alters the spatial gradient in the surround. Journal of Experimental Psychology-Human Perception and Performance. 2008;34:298–309. doi: 10.1037/0096-1523.34.2.298. [DOI] [PubMed] [Google Scholar]
- Kruschke JK, Fragassi MM. The perception of causality: Feature binding in interacting objects; Proceedings of the Eighteenth Annual Conference of the Cognitive Science Society; Hillsdale, NJ: Erlbaum; 1996. pp. 441–446. [Google Scholar]
- Luck SJ. Visual short-term memory. In: Luck SJ, Hollingworth A, editors. Visual Memory. New York: Oxford University Press; 2008. pp. 43–86. [Google Scholar]
- Luck SJ, Vogel EK. The capacity of visual working memory for features and conjunctions. Nature. 1997;390:279–281. doi: 10.1038/36846. [DOI] [PubMed] [Google Scholar]
- Makovski T, Sussman R, Jiang YHV. Orienting attention in visual working memory reduces interference from memory probes. Journal of Experimental Psychology: Learning Memory and Cognition. 2008;34:369–380. doi: 10.1037/0278-7393.34.2.369. [DOI] [PubMed] [Google Scholar]
- Mitroff SR, Alvarez GA. Space and time, not surface features, underlie object persistence. Psychonomic Bulletin & Review. 2007;14:1199–1204. doi: 10.3758/bf03193113. [DOI] [PubMed] [Google Scholar]
- Mitroff SR, Scholl BJ, Noles NS. Object files can be purely episodic. Perception. 2007;36:1730–1735. doi: 10.1068/p5804. [DOI] [PubMed] [Google Scholar]
- Mitroff SR, Scholl BJ, Wynn K. Divide and conquer: How object files adapt when a persisting object splits into two. Psychological Science. 2004;15:420–425. doi: 10.1111/j.0956-7976.2004.00695.x. [DOI] [PubMed] [Google Scholar]
- Mitroff SR, Scholl BJ, Wynn K. The relationship between object files and conscious perception. Cognition. 2005;96:67–92. doi: 10.1016/j.cognition.2004.03.008. [DOI] [PubMed] [Google Scholar]
- Moore CM, Enns JT. Object updating and the flash-lag effect. Psychological Science. 2004;15:866–871. doi: 10.1111/j.0956-7976.2004.00768.x. [DOI] [PubMed] [Google Scholar]
- Moore CM, Mordkoff JT, Enns JT. The path of least persistence: Evidence of object-mediated visual updating. Vision Research. 2007;47:1624–1630. doi: 10.1016/j.visres.2007.01.030. [DOI] [PubMed] [Google Scholar]
- Mou WM, McNamara TP. Intrinsic frames of reference in spatial memory. Journal of Experimental Psychology-Learning Memory and Cognition. 2002;28:162–170. doi: 10.1037/0278-7393.28.1.162. [DOI] [PubMed] [Google Scholar]
- Müller HJ, Von Mühlenen A. Probing distractor inhibition in visual search: Inhibition of return. Journal of Experimental Psychology: Human Perception and Performance. 2000;26:1591–1605. doi: 10.1037//0096-1523.26.5.1591. [DOI] [PubMed] [Google Scholar]
- Noles NS, Scholl BJ, Mitroff SR. The persistence of object file representations. Perception & Psychophysics. 2005;67:324–334. doi: 10.3758/bf03206495. [DOI] [PubMed] [Google Scholar]
- O'Keefe J, Conway DH. Hippocampal place units in the freely moving rat: Why they fire where they fire. Experimental Brain Research. 1978;31:573–590. doi: 10.1007/BF00239813. [DOI] [PubMed] [Google Scholar]
- Olivers CNL, Meijer F, Theeuwes J. Feature-based memory-driven attentional capture: Visual working memory content affects visual attention. Journal of Experimental Psychology: Human Perception & Performance. 2006;32:1243–1265. doi: 10.1037/0096-1523.32.5.1243. [DOI] [PubMed] [Google Scholar]
- Olson IR, Jiang Y. Is visual short-term memory object based? Rejection of the "strong object" hypothesis. Perception & Psychophysics. 2002;64:1055–1067. doi: 10.3758/bf03194756. [DOI] [PubMed] [Google Scholar]
- Olson IR, Marshuetz C. Remembering "what" brings along "where" in visual working memory. Perception & Psychophysics. 2005;67:185–194. doi: 10.3758/bf03206483. [DOI] [PubMed] [Google Scholar]
- Olson IR, Moore KS, Stark M, Chatterjee A. Visual working memory is impaired when the medial temporal lobe is damaged. Journal of Cognitive Neuroscience. 2006;18:1087–1097. doi: 10.1162/jocn.2006.18.7.1087. [DOI] [PubMed] [Google Scholar]
- Olson IR, Page K, Moore KS, Chatterjee A, Verfaellie M. Working memory for conjunctions relies on the medial temporal lobe. Journal of Neuroscience. 2006;26:4596–4601. doi: 10.1523/JNEUROSCI.1923-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parkinson JK, Murray EA, Mishkin M. A selective mnemonic role for the hippocampus in monkeys: Memory for the location of objects. Journal of Neuroscience. 1988;8:4159–4167. doi: 10.1523/JNEUROSCI.08-11-04159.1988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pashler H. Familiarity and the detection of change in visual displays. Perception & Psychophysics. 1988;44:369–378. doi: 10.3758/bf03210419. [DOI] [PubMed] [Google Scholar]
- Phillips WA. On the distinction between sensory storage and short-term visual memory. Perception & Psychophysics. 1974;16:283–290. [Google Scholar]
- Pigott S, Milner B. Memory for different aspects of complex visual scenes after unilateral temporal- or frontal-lobe resection. Neuropsychologia. 1993;31:1–15. doi: 10.1016/0028-3932(93)90076-c. [DOI] [PubMed] [Google Scholar]
- Pylyshyn ZW. Some puzzling findings in multiple object tracking: I. Tracking without keeping track of object identities. Visual Cognition. 2004;11:801–822. [Google Scholar]
- Pylyshyn ZW, Storm RW. Tracking multiple independent targets: Evidence for a parallel tracking mechanism. Spatial Vision. 1988;3:179–197. doi: 10.1163/156856888x00122. [DOI] [PubMed] [Google Scholar]
- Raffone A, Wolters G. A cortical mechanism for binding in visual working memory. Journal of Cognitive Neuroscience. 2001;13:766–785. doi: 10.1162/08989290152541430. [DOI] [PubMed] [Google Scholar]
- Rasmussen IP, Hollingworth A. The capacity for spatial updating in visual short-term memory. Paper presented at the Annual Meeting of the Vision Sciences Society; Naples, Fl. 2008. [Google Scholar]
- Richard AM, Hollingworth A. Strategic control of visual short-term memory during scene viewing. Paper presented at the Annual Meeting of the Vision Sciences Society.2008. [Google Scholar]
- Richard AM, Luck SJ, Hollingworth A. Establishing object correspondence across eye movements: Flexible use of spatiotemporal and surface feature information. Cognition. 2008;109:66–88. doi: 10.1016/j.cognition.2008.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Saiki J. Feature binding in object-file representations of multiple moving items. Journal of Vision. 2003;3:6–21. doi: 10.1167/3.1.2. [DOI] [PubMed] [Google Scholar]
- Scholl BJ, Pylyshyn ZW. Tracking multiple items through occlusion: Clues to visual objecthood. Cognitive Psychology. 1999;38:259–290. doi: 10.1006/cogp.1998.0698. [DOI] [PubMed] [Google Scholar]
- Scholl BJ, Pylyshyn ZW, Feldman J. What is a visual object? Evidence from target merging in multiple object tracking. Cognition. 2001;80:159–177. doi: 10.1016/s0010-0277(00)00157-8. [DOI] [PubMed] [Google Scholar]
- Scholl BJ, Pylyshyn ZW, Franconeri SL. When are spatiotemporal and featural properties encoded as a result of attentional allocation?. Paper presented at the The conference of the Association for Research in Vision and Ophthalmology.1999. [Google Scholar]
- Shapiro ML, Tanila H, Eichenbaum H. Cues that hippocampal place cells encode: Dynamic and hierarchical representation of local and distal stimuli. Hippocampus. 1997;7:624–642. doi: 10.1002/(SICI)1098-1063(1997)7:6<624::AID-HIPO5>3.0.CO;2-E. [DOI] [PubMed] [Google Scholar]
- Soto D, Heinke D, Humphreys GW, Blanco MJ. Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology-Human Perception and Performance. 2005;31:248–261. doi: 10.1037/0096-1523.31.2.248. [DOI] [PubMed] [Google Scholar]
- Soto D, Humphreys GW, Heinke D. Working memory can guide pop-out search. Vision Research. 2006;46:1010–1018. doi: 10.1016/j.visres.2005.09.008. [DOI] [PubMed] [Google Scholar]
- Sperling G. The information available in brief visual presentations. Psychological Monographs. 1960;74 (11, Whole no. 498). [Google Scholar]
- Suzuki WA, Amaral DG. Perirhinal and parahippocampal cortices of the macaque monkey: Cortical afferents. Journal of Comparative Neurology. 1994;350:497–533. doi: 10.1002/cne.903500402. [DOI] [PubMed] [Google Scholar]
- Takeda Y, Yagi A. Inhibitory tagging in visual search can be found if search stimuli remain visible. Perception & Psychophysics. 2000;62:927–934. doi: 10.3758/bf03212078. [DOI] [PubMed] [Google Scholar]
- Tipper SP, Driver J, Weaver B. Object-centered inhibition of return of visual-attention. Quarterly Journal of Experimental Psychology Section a-Human Experimental Psychology. 1991;43:289–298. doi: 10.1080/14640749108400971. [DOI] [PubMed] [Google Scholar]
- Tipper SP, Jordan H, Weaver B. Scene-based and object-centered inhibition of return: Evidence for dual orienting mechanisms. Perception & Psychophysics. 1999;61:50–60. doi: 10.3758/bf03211948. [DOI] [PubMed] [Google Scholar]
- Tipper SP, Weaver B, Jerreat LM, Burak AL. Object-based and environment-based inhibition of return of visual-attention. Journal of Experimental Psychology: Human Perception and Performance. 1994;20:478–499. [PubMed] [Google Scholar]
- Treisman A. Features and objects: The fourteenth Bartlett memorial lecture. Quarterly Journal of Experimental Psychology. 1988;40A:201–237. doi: 10.1080/02724988843000104. [DOI] [PubMed] [Google Scholar]
- Treisman A, Zhang WW. Location and binding in visual working memory. Memory & Cognition. 2006;34:1704–1719. doi: 10.3758/bf03195932. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vecera SP. Grouped locations and object-based attention: Comment on Egly, Driver, and Rafal (1994) Journal of Experimental Psychology: General. 1994;123:316–320. [Google Scholar]
- Vecera SP, Farah MJ. Does visual attention select objects or locations? Journal of Experimental Psychology: General. 1994;123:146–160. doi: 10.1037//0096-3445.123.2.146. [DOI] [PubMed] [Google Scholar]
- Weaver B, Lupianez J, Watson FL. The effects of practice on object-based, location-based, and static-display inhibition of return. Perception & Psychophysics. 1998;60:993–1003. doi: 10.3758/bf03211934. [DOI] [PubMed] [Google Scholar]
- Wheeler ME, Treisman AM. Binding in short-term visual memory. Journal of Experimental Psychology: General. 2002;131:48–64. doi: 10.1037//0096-3445.131.1.48. [DOI] [PubMed] [Google Scholar]
- Woodman GF, Luck SJ, Schall JD. The role of working memory representations in the control of attention. Cerebral Cortex. 2007;17:118–124. doi: 10.1093/cercor/bhm065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xu F, Carey S, Welch J. Infants' ability to use object kind information for object individuation. Cognition. 1999;70:137–166. doi: 10.1016/s0010-0277(99)00007-4. [DOI] [PubMed] [Google Scholar]
- Xu Y. Encoding color and shape from different parts of an object in visual short-term memory. Perception & Psychophysics. 2002a;64:1260–1280. doi: 10.3758/bf03194770. [DOI] [PubMed] [Google Scholar]
- Xu Y. Limitations of object-based feature encoding in visual short-term memory. Journal of Experimental Psychology: Human Perception and Performance. 2002b;28:458–468. doi: 10.1037//0096-1523.28.2.458. [DOI] [PubMed] [Google Scholar]
- Zelinsky GJ, Loschky LC. Eye movements serialize memory for objects in scenes. Perception & Psychophysics. 2005;67:676–690. doi: 10.3758/bf03193524. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.