Precision of working memory for visual motion sequences and transparent motion surfaces

Abstract

Recent studies investigating working memory for location, colour and orientation support a dynamic resource model. We examined whether this might also apply to motion, using random dot kinematograms (RDKs) presented sequentially or simultaneously.

Mean precision for motion direction declined as sequence length increased, with precision being lower for earlier RDKs. Two alternative models of working memory were compared specifically to distinguish between the contributions of different sources of error that corrupt memory (Zhang & Luck (2008) vs. Bays et al (2009)). The latter provided a significantly better fit for the data, revealing that decrease in memory precision for earlier items is explained by an increase in interference from other items in a sequence, rather than random guessing or a temporal decay of information. Misbinding feature attributes is an important source of error in working memory.

Precision of memory for motion direction decreased when two RDKs were presented simultaneously as transparent surfaces, compared to sequential RDKs. However, precision was enhanced when one motion surface was prioritized, demonstrating that selective attention can improve recall precision. These results are consistent with a resource model that can be used as a general conceptual framework for understanding working memory across a range of visual features.

Introduction

Visual working memory provides a limited, temporary storage system for relevant information in the visual scene (Baddeley, 2003). Understanding the limits and mechanisms underlying working memory has become fundamental to understanding perception(Simons and Rensink, 2005), attention (Awh and Jonides, 2001; de Fockert et al., 2001; Lepsien and Nobre, 2007) and visual search (Emrich et al., 2009, 2010). Motion is a fundamentally important source of information in the visual scene. From tracking the movements of a predator in the wild to computing the direction of moving cars when crossing a road, segregating objects using motion cues is critical to survival. However, compared to other visual features, e.g. colour, orientation or spatial location, we know far less about the nature of working memory for motion, particularly for sequences of moving stimuli over time.

Previous studies have shown that different features of visual motion, including direction of motion can be maintained in memory for several seconds (Magnussen and Greenlee, 1992; Blake et al., 1997), the information stored is spatially localized (Zaksas et al., 2001; Pasternak and Zaksas, 2003) and that visual motion between two temporally disparate arrays might assist to connect them into a single visual event (Song and Jiang, 2006). Furthermore, physiological studies provide evidence that motion processing systems involved in perception are also involved in the storage of visual motion. Neuronal activity during delay periods in a motion discrimination task, for example, reveals that the direction of motion is represented in MT neurons (Bisley et al., 2004). Recently transcranial magnetic stimulation (TMS) in human participants has revealed that activity in V5/MT+ reflects motion qualities of the items that are maintained in visual short term memory (Silvanto and Cattaneo, 2010).

However, less is known about the limits of working memory for visual motion. Kawasaki and colleagues (2008) conducted a change detection study for different features of items including colour, shape and direction of motion (Kawasaki et al., 2008). In such change detection tasks, participants are asked to detect the presence of supra-threshold changes among an array of items after a short retention period (Luck and Vogel, 1997; Vogel et al., 2001; Wilken and Ma, 2004; Awh et al., 2006).

Previously, studies using this design have found that observers are accurate for array sizes of up to 3 to 4 colours, shapes, orientations or integrated objects defined as conjunctions of these features (Luck and Vogel, 1997; Anderson et al., 2011; Luria and Vogel, 2011). Based on these results, item-limit models of memory have been proposed, which argue for a limited capacity of 3-4 independent memory “slots”, each storing information about an integrated visual object. However, in the study by Kawasaki and colleagues (2008), capacity limit for the direction of visual motion was found to be only 2 directions of motion.

However, change detection tasks with a fixed magnitude of change, might not be sensitive to changes in the fidelity of memory. Typically the magnitude of the change to-be-detected in such experiments is constant and arbitrarily large and observers are asked to make a binary (yes or no) response to the following question “Was there a change or not?” Recently studies have investigated the resolution with which visual features are stored in working memory using discrimination (Palmer, 1990; Lakha and Wright, 2004; Bays and Husain, 2008), adjustment (Wilken and Ma, 2004; Zhang and Luck, 2008; Bays et al., 2009; Fougnie et al., 2010) or change detection tasks, changing the magnitude of the information load of the to-be-remembered objects (Alvarez and Cavanagh, 2004).

The results from some of these studies have revealed that the precision with which items are stored in working memory depend on the number of items simultaneously held in memory (Alvarez and Cavanagh, 2004; Wilken and Ma, 2004; Bays and Husain, 2008; Bays et al., 2009; Brady et al., 2011). These results have begun to challenge the view that visual working memory might be limited to a fixed number of objects. Instead the results have shown that there is a graded decrease in the precision of memory as the number of items increase, even from one to two items, i.e., below the classical item limit of 3-4 memory “slots” (Bays and Husain, 2008; Bays et al., 2009, 2011; Brady et al., 2011). These results are compatible with a dynamic resource model of memory which argues that the resolution with which an item is stored in memory is proportional to the fraction of memory resource dedicated to that item. Hence as the set size increases, the fraction of memory allocated to each item decreases and each item is stored with less precision (Alvarez and Cavanagh, 2004; Wilken and Ma, 2004; Bays and Husain, 2008; Bays et al., 2009). Indeed, a study comparing change detection performance in humans and rhesus monkeys concluded that the results from both species are best explained using a continuous resource model where the precision of stored items is dependent on set size (Elmore et al., 2011).

An alternative to change detection tasks relies on observers remembering a visual feature and reproducing the exact qualities of the stored feature after a retention period, using a method of adjustment (Wilken and Ma, 2004; Zhang and Luck, 2008; Bays et al., 2009, 2011; Fougnie et al., 2010). Using this methodology, one can measure the precision with which an item is stored in working memory. Studies using precision as an index of working memory have so far examined recall for orientation, spatial location and colour presented simultaneously (Bays and Husain, 2008; Bays et al., 2009, 2011), and more recently for sequences of orientations (Gorgoraptis et al., 2011). To the best of our knowledge there has been no systematic investigation of the precision of memory for visual motion stimuli. In Experiment 1, we examine the nature of memory distribution for motion direction presented in sequences, investigating the effects of set size and serial position of target (where it appeared in a sequence) on precision in memory.

The method we use provides a sensitive measure of memory precision and allows us to test the predictions made by both classic item-limit and resource models of memory for sequences. Importantly, for the first time, we also directly test the two recently proposed models distinguishing errors in memory at recall (Zhang and Luck, 2008; Bays et al., 2009). Zhang and Luck (2008) proposed a revised version of the slots model, slots+ averaging model, where a memory resource is divided into a few fixed-resolution slots (<3). Below the limit of slots, bound items can be stored in more than one slot and averaged to provide a high resolution memory. However, beyond the slot limit no information is stored resulting in an increase in random responses for larger set sizes. Therefore, this model would predict that the errors in memory at recall arise as a result of either variable memory of the target object or guessing.

However, Bays et al (2009) proposed a model that also takes into account the misbinding errors in recall: errors resulting from incorrect conjunction of features that arise because non-target can systematically corrupt memory by biasing recall.

According to this scheme, larger set sizes not only result in a decrease in precision but also cause an increase in misbinding errors. In the present study, we test whether the addition of misbinding errors to the model can describe the results better. In Experiment 1, we used sequences of motion directions, for sequences of up to 4. For set sizes of up to 4, both models predict infrequent random responses and a decrease in precision for longer sequences. However, the model proposed by Bays et al (2009) would also predict an increase in misbinding errors. Therefore, the present experiment provides an opportunity to not only test the precision of working memory for motion sequences but also test whether taking into account errors corresponding to misbinding would improve models of memory.

One question that follows is how memory resources are distributed between two different visual motion directions when these are presented simultaneously, rather than sequentially, at the same location. Such motion transparency, involving two overlapping directions of motion, occurs naturally in the visual environment: from rain streaming down a window of a moving car to when an animal is moving behind foliage on a windy day. In Experiment 2, we probe the fidelity of memory for two overlapping, transparent motion surfaces and compare the findings with the condition when observers have to remember two motion directions presented sequentially. This paradigm also allowed us to examine the effects of attending to only one motion surface on the fidelity of memory representations.

Overall, our results for visual motion are consistent with a resource model of working memory. They demonstrate that previous inferences made by measuring the precision of memory for location, colour and orientation (Bays and Husain, 2008; Bays et al., 2009) can also be extended to motion. Thus the resource model appears to be a general conceptual framework that can be applied across a wide range of different visual features.

Experiment 1

Method

Participants

Eleven healthy individuals (4 male, 7 female) with an average age of 22 years (range: 18-28) participated in this study. All participants had normal or corrected to normal vision and reported normal colour vision. Participants provided written consent to the procedure of the experiment, which was approved by the local ethics committee.

Stimuli

Stimuli were generated by Cogent toolbox (www.vislab.ucl.ac.uk/Cogent/) for MATLAB and were displayed on a 14.1” flat panel display (resolution: 800 × 600 pixels, refresh rate: 60 Hz). Participants were seated approximately 60cm from the monitor in a dimly illuminated room.

In each trial, a sequence of Random Dot Kinematograms (RDKs) was presented at the centre of the screen on a black background (Fig. 1). RDKs consisted of 25 dots, each covering 0.1° of visual angle. Dots were displayed within an invisible circular aperture of 150 pixels in diameter (5.7 ° of visual angle). Dot lifetime was 500msec and dots reaching the edge of the circular aperture were re-positioned randomly on the other side of the aperture; therefore dot density was kept constant throughout the presentation.

Figure 1.

An example of a sample trial.

A sequence of 1-4 RDKs, moving coherently in different directions of motion, were presented. Any of the RDKs could be probed by colour of the response stimuli and participants were asked to adjust the orientation of the response stimuli to the direction of the motion of the RDK with similar colour.

Motion was 100% coherent (constant speed of 4.5 degrees/sec for all dots). Direction of motion for each RDK within a sequence was set at a random value between 0-360° while maintaining a minimum angular separation of 60° between different motion directions. RDKs were presented in a randomly selected colour from a selection of 5 different colours (red, green, blue, white and yellow) and within each sequence there was no repetition of colour.

Procedure

Each trial started with a fixation cross (500msec) followed by a sequence of RDKs. Sequences varied in length from 1 to 4 RDKs. Observers did not know beforehand how many RDKs would be displayed in each trial. Within a sequence, each RDK was presented for 500msec and was followed by a 500msec blank interval before the presentation of the next RDK. Participants were asked to remember the direction of motion of all RDKs. The last item in the sequence was followed by a 500msec blank interval before the probe display was presented.

The probe stimulus consisted of a circle (5.7 ° of visual angle in diameter) presented at the centre of the screen with an arrow positioned at a random orientation – drawn from the uniform distribution [0-360°] – within the circle. This probe stimulus was presented in the same colour as one of the RDKs in the sequence. Participants were asked to adjust the orientation of the coloured arrow, using a trackball, to match the direction of motion of the RDK presented in the same colour in the sequence.

The probability of probing any of the RDKs within the sequence was kept constant for all items in the sequence. Probe display was presented until response. Participants were told to respond as accurately as possible, with no time pressure. A schematic presentation of a sample sequence is presented in Figure 1.

Each participant completed 330 trials, 6 blocks of 55 trials. Each possible combination of sequence length and target position in the sequence (10 possible combinations) was presented for 30 trials throughout the whole experiment.

Control delay condition

30 additional trials consisted of the condition where a single RDK was presented followed by a long retention period (3000msec). The duration of the retention period in this condition is equal to the duration of the presentation of 3 RDKs. Therefore the duration between presentation of RDK and probe stimuli in this condition is equal to the duration between presentation of the first RDK in sequence of 4 items and the probe stimulus. Thus this condition served as a control to examine the effects of temporal decay on memory precision when one motion direction had to be remembered, for the same duration as the first item in sequences of 4.

Control for serial position probing

It might be argued that the results obtained here are systematically related to the method of probing: the colour of the response stimulus cues the participant which stimulus they had to recall. We therefore conducted a control experiment in which observers were probed by the number of the item in the sequence, rather than its colour. 6 participants participated in this version of this task. In each trial, similar to the main experiment, participants were presented with sequences of coloured RDKs (1-4). However, at the end of each trial, participants were probed using serial position number, e.g., participants might be asked to reproduce the motion direction of the second RDK. The probe display was similar to that used in the main experiment except for the following changes. First, the probe was presented in a colour that was not used in any of the sequences (i.e. white) and directly above the response stimulus presented in the main experiment. Second, the cued RDK was indicated in text (e.g., THIRD would indicate to the participant that the third motion in the sequence direction had to be recalled). This control condition was added to ensure that the serial position curves are not an artefact of the probing method.

Different conditions were randomly intermixed within each block. Participants were familiarized with the experimental apparatus and completed a practice block of 30 intermixed experimental trials prior to experiment.

Analysis

In each trial, error was calculated as the deviation of the response value (i.e. the reported motion direction of the target) from the target value (i.e. the real value of the target’s motion direction).

Following previous work (Bays and Husain, 2008; Bays et al., 2009) we defined precision as the reciprocal of the standard deviation of error measured using the method described by Fisher (1993) for calculating standard deviation in a circular parameter space (Fisher, 1993). This is a measure of variability of response; less variability in response corresponds to more precise memory. Using this definition, we calculated the precision of working memory for different sequence lengths and serial positions of the target for each participant in order to test the effects of sequence length and serial position of the target on variability in memory. Chance precision, that is the expected precision value if a participant responded at random in all trials, was calculated and subtracted from the precision values obtained for each condition.

In order to distinguish different sources of error in memory for tasks similar to the one we used (adjustment tasks), two probabilistic models have recently been proposed. According to Zhang and Luck’s (2008) model, there are two possible sources of error on each trial:

1. A Von Mises (circular Gaussian) distribution in memory centred on the target direction (Fig 2A).

2. A uniform distribution of error corresponding to random responses (Fig 2B).

Figure 2.

Three sources of error in memory used for modelling performance

A) A Von Mises (circular Gaussian) distribution with concentration parameter K, centred on the target direction of motion, capturing variability in memory for target direction of motion, with the area under the distribution (shaded) being proportional to the probability of responding to the target motion direction; B) A uniform distribution of error corresponding to random error, with the area under this distribution corresponding to the proportion of random responses and C) Von Mises distributions with concentration parameter K, centred on one of the non-target directions of motion, resulting from errors in identifying which motion direction belonged with the target colour (misbinding). The area under the distribution corresponds to the proportion of non-target responses

The model is described by the following equation:

$$p\left(\widehat{\theta }\right)=(1-\gamma ){\varphi }_{\kappa }(\widehat{\theta }-\theta )+\gamma \frac{1}{2\pi }$$ Equation (1)

where θ is the target motion direction and $\widehat{\theta }$ is the response direction. _κ is the Von Mises distribution with mean of zero and concentration parameter κ. The concentration parameter κ corresponds to the variability of recall of the target, where greater κ corresponds to lower variability in the distribution. γ corresponds to the proportion of trials where participants were guessing, i.e. responding at random.

Recently, an additional source of error has been identified by Bays et al. (2009) (see also http://www.sobell.ion.ucl.ac.uk/pbays/code/JV10/). According to this model, a proportion of errors in these tasks correspond to the probability of incorrectly responding with the motion direction of one of the other, non-target RDKs. These responses can be captured by Von Mises distributions centred on each of the non-target motion directions in a sequence (Fig 2C).

The model is described by the following equation:

$$p\left(\widehat{\theta }\right)=\alpha {\varphi }_{\kappa }(\widehat{\theta }-\theta )+\beta \frac{1}{m}\sum _i^m{\varphi }_{\kappa }(\widehat{\theta }-{\phi }_i)+\gamma \frac{1}{2\pi }$$ Equation (2)

Probability of responding with the target direction is given by α, and β corresponds to the probability of misbinding errors, i.e. trials on which a participant responds with a non-target direction as a result of incorrect conjunction of features (here colour and motion direction) in memory. {φ₁,φ₂, … φ_m} correspond to motion directions of the m non-target items. Probability of responding at random (γ) is calculated as 1-α-β.

Maximum likelihood estimates of parameters α, β, γ and κ were obtained using Expectation Maximization (Myung, 2003) for each participant and each experimental condition.

It’s important to note that neither of the two models described here include noise in sensory and motor systems that may affect performance at recall. However, one would assume that such noise remains constant across different sequence lengths and serial positions and hence any effects observed on modelling parameters can be reliably attributed to noise in memory rather than sensory or motor systems.

Model Comparison

In order to determine which of the two models provides a better fit of the data, we used likelihood ratio tests: used to compare two models in cases where one is a special case of the other (i.e. “nested” models). Here, the model proposed by Zhang and Luck (2008) (the null model) is a special case of the model proposed by Bays et al. (2009) (the alternative model), in which the β parameter (probability of non-target responses) is fixed at zero. The likelihood ratio test statistic (D) was calculated separately for each sequence length and serial position, collapsing data across subjects, and statistical significance tested by comparison to the χ² distribution with 1 d.f.

Results

Behavioural results

We first examined the distribution of errors in relation to the target direction for different sequence lengths. As illustrated in Figure 3, in longer sequences the proportion of responses falling close to the target direction decreased; observed as a reduction in the responses made around the target as sequence length increases. Furthermore, the longer tails of the distribution in longer sequences provide evidence for additional sources of error (either guessing or misbinding errors) that may occur in longer sequences (Fig. 3).

Figure 3.

Distribution of errors relative to the target direction for different sequence lengths Response probability is plotted as a function of the difference between response and target direction (black dots), for different sequence lengths (1 – 4 items). As sequence lengths increase, the variability in recall of the target direction (width of the distribution) increases and the peak of the distribution centred around zero decreases. Red line indicates the response probability predicted by the mixture model proposed by Bays et al, (2009) (see Analysis section).

We have also calculated bias that is the mean deviation from the target direction in our data for different sequence lengths; the bias is <3 degrees for all set sizes. For sequences of 1 item it was 2.2° (SD 0.36); for 2 items it was 1.1 ° (SD 0.60); for 3 it was 2.3 ° (SD 0.49), and for 4 items it was 2.6 ° (SD 0.62).

We then investigated whether precision of memory for motion direction was affected by the number of items in a sequence. Mean precision for each sequence length (collapsing across all serial positions) was calculated. Precision decreased significantly as the number of items in the sequence increased; one-way ANOVA: F(3, 106)= 21.406, P<0.001 (Fig. 4A). More importantly, overall precision decreased significantly when sequence length was increased from only 1 to 2 RDKs (t(10)= 4.881, p= 0.001), contrary to what would be predicted by item-based models of memory.

Figure 4.

Precision of memory for different sequence lengths

A) Average precision for each sequence length: precision decreased with increasing number of items in a sequence (* p< 0.05, ** p< 0.01); B) Within each sequence, the last item was always remembered best (recency effect); C) Precision decreased as sequence length increased at any of the serial positions within a sequence. Note the privileged precision with which the last item was recalled. Zero precision indicates chance performance, error bars demonstrate SEM across participants (N=11).

A similar pattern of results was observed for comparisons between other sequence lengths; two vs. three (t(10)= 6.238, p<0.001) and three vs. four (t(10)= 2.39, p= 0.038, n.s. after Bonferroni correction). Performance was significantly better than chance for all sequence lengths (e.g. sequence length 4: t(10) = −15.13; p<0.001) and in all participants.

We also examined how precision of recall is affected by the serial position of a target stimulus, i.e., when the target was presented in the sequence. There was a significant effect of serial order on precision of recall (Fig. 4B; two-way ANOVA, main effect of serial order, F(3, 100)= 10.47, p< 0.001). Precision was best for the last item, demonstrating a strong recency effect. Interestingly, when the data from the last item in each sequence was excluded, all other items in the sequence were remembered with similar precision (two-way ANOVA, F(2,50)= 0.65, p=0.53).

Importantly, the precision of recall for the last item in a sequence was influenced by sequence length. This item was remembered with higher precision when presented in shorter sequences (Fig. 4B, last items; one-way ANOVA of the last item; F(3, 40)= 4.05, p= 0.01). Note that although several studies of visual working memory using sequences have also demonstrated recency effects (Phillips and Christie, 1977; Wright et al., 1985; Neath, 1993; Hay et al., 2007; Botvinick et al., 2009; Blalock and Clegg, 2010), using precision as an index of recall allowed us to observe that the magnitude of the recency effect was modulated by sequence length.

A similar effect was observed for other serial positions. Precision of memory for items presented 2^nd to last in the sequence was significantly lower when part of a longer sequence (Fig. 4B; one-way ANOVA; F(2, 30)= 10.77, p<0.001). t-test analysis on the 3^rd to last items for sequence lengths three and four illustrated that precision for the 3^rd to last item was significantly higher for items presented in a sequence of three compared to those presented in a sequence of four; t(24)= 3.106, p= 0.011 (Fig. 4B). This fall in precision in all serial positions was driven purely by the overall number of items in the sequence, compatible with the dynamic resource model of memory. Thus, regardless of the serial position of the target, memory for motion direction was determined by the fraction of memory resource allocated to that item, which was determined by the overall number of items in the sequence.

A similar pattern of results was found in a control condition using serial position number as the probe (Figure 4C). Furthermore in order to test whether participants’ responses were biased by non-targets closer to the target in the sequence, Mean Square Error (MSE) in relation to each of the non-targets for all sequence lengths and for targets in different serial positions were calculated. There was no significant difference between MSEs for non-targets at different proximity to the target in different sequences. Therefore, in the present study, the distance between the target and the non-targets does not influence the extent to which non-targets influence error in memory.

Loss in precision for earlier items in the sequence might be caused by either temporal decay of memory or interference from other items in the sequence (Hole, 1996; Berman et al., 2009; Lewandowsky et al., 2009). To test these alternative hypotheses, we compared precision of memory for one item followed by either a short or a long (equal to presentation duration of 3 items) retention period, our control condition.

Precision of memory was not affected by increasing the retention interval (t(10)= 1.658, p=0.128). However, precision decreased significantly when three items were presented in the retention interval, i.e., trials where the target RDK was the 1^st item presented in a sequence of 4 RDKs (Fig. 5; t(10)= −8.132, p<0.001). Therefore the loss of precision observed for earlier items in the sequence was not caused by a temporal decay of memory but rather the interference of items that follow the earlier items.

Figure 5.

Effect of temporal decay versus interference

No loss of precision in memory with increasing retention interval (left and middle bar), but precision decreased significantly when 3 items were presented in the retention interval (right bar). Error bars indicate SEM across participants (N=11).

Model Comparisons

We applied the two alternative models, i.e. models proposed by Zhang and Luck (2008) and Bays et al (2009) to the data (see Analysis and (Bays et al., 2009) for details). Maximum likelihood estimates of the probability of responding at random and variability in recall of the target direction under both models were estimated. The probability of responding with a non-target motion direction was estimated using Bays et al’s (2009) model (see also www.sobell.ion.ucl.ac.uk/pbays/code/JV10/).

The concentration parameter (κ) which captures variability in memory for target direction, did not differ significantly between the two models for all different sequence lengths (Fig. 6A, two-way ANOVA, main effect of model type; F(1,80)= 0.15, p=0.7). Furthermore, there was no significant difference between the proportion of target responses estimated by the two models for all sequence lengths (Fig. 6B, two-way ANOVA, main effect of model type; F(1,80)= 0.99, p=0.32). Thus, with respect to these parameters, the models are reassuringly equivalent.

Figure 6.

Modelling parameters analyzed for different sequence lengths

A) Concentration parameter decreased significantly as sequence length increased therefore memory for target is more variable in longer sequences, B) Probability of responding with the target direction of motion was significantly lower in longer sequences, C) Probability of responding with a non-target motion direction increased significantly in longer sequences. D) No difference in random responses for different sequence lengths. Error bars indicate SEM across participants (N=11).

However, overall, the probability of random responses was significantly higher when estimated by the model proposed by Zhang and Luck (2008) compared to that of Bays et al (2009); two way ANOVA, main effect of model type; F(1,80)=33.55, p <0.001 (Fig. 6C).

In particular, there was a significant increase in random responses estimated by Zhang and Luck’s (2008) model, in longer sequences (Fig. 6C- dashed black line; one-way ANOVA, F(3,40)= 18.310, p<0.001), increasing to up to 11% for items presented in a sequence of 4. Comparisons between the proportion of random responses for each sequence length confirmed a significant increase in random responses estimated by Zhang and Luck’s model in sequences of 3 (t(10) = 2.99, p<0.02) and 4 items (t(10) = 6.62, p<0.001).

By contrast, random responses estimated by the model proposed by Bays et al (2009) were infrequent (<3%) and did not significantly differ for different sequence lengths (Fig. 6C- red line; one-way ANOVA, F(3,40) = 1.95, p = 0.14). Crucially, however, there was an increase in probability of non-target responses, the extra parameter proposed by Bays et al (2008), in longer sequences (Fig 6D; one-way ANOVA, F(3,40)= 16.85, p<0.001).

For each sequence length (2-4), log likelihood values (LL) were calculated under both models to test which provided a better fit to the data. Since there are no non-target items in a sequence of 1, the log likelihood values (LL) for this condition under both models are identical and hence will not be reported. Table 1 shows the LL values under each model and the probability that the null model (Zhang and Luck, 2008) provides a better fit for the data based on a likelihood ratio test.

Table 1. Log likelihood (LL) values for each model and the p-values of the likelihood ratio test for different sequence lengths.

Sequence Length	Zhang and Luck (2008) LL value	Bays et al (2009) LL value	p-value
2	−1209.66	−4.67	<0.001
3	−1815.60	−914.94	<0.001
4	−2425.86	−1856.32	<0.001

As illustrated in Table 1, for all sequence lengths, Bays et al’s (2009) model provides a significantly better fit for the data compared to Zhang and Luck’s model (2008). Furthermore, we calculated LL values for each serial position condition for all participants collapsed together. Table 2 provides the LL values for each serial position condition under both models and the significance of the likelihood ratio test.

Table 2. Log likelihood (LL) values for each model and the p-value of the likelihood ratio test for different serial order positions.

Sequence Length	Serial Position	Zhang and Luck (2008) LL value	Bays et al (2009) LL value	p-value
2	First	−27.56	−25.47	<0.05
2	Last	25.27	28.92	<0.01
3	First	−126.92	−118.06	<0.001
3	Second	−151.11	−137.21	<0.001
3	Last	−2.545	−1.37	0.13
4	First	−197.07	−191.24	<0.001
4	Second	−241.70	−223.20	<0.001
4	Third	−160.17	−154.90	<0.002
4	Last	−21.71	−20.65	0.145

For all earlier items in a sequence, the model proposed by Bays et al (2009) provides a significantly better fit for the data compared to the model proposed by Zhang and Luck (2008). As for last items in the sequence, Bays et al’s (2009) model is a significantly better fit for the data only for the last item presented in a sequence of 2. However, it’s important to note that as shown recently (Gorgoraptis et al., 2011), last items in a sequence are significantly less prone to misbinding errors. Therefore, the reason why the model proposed by Bays et al (2009) does not provide a better fit for last items, is because the proportion of non-target responses is near zero for these items. Memory for the last item is therefore less likely to be corrupted by other items in the sequence.

Given that overall the model proposed by Bays et al (2009) is a significantly better fit for the present data, we will use parameters estimated by this method to distinguish the possible sources of error in recall and investigate the effects of sequence length and serial position on these sources of error. Note that this model includes the target error and random errors just as in Zhang and Luck (2008) but simply extends that model to consider the possibility of non-targets also influencing errors. If we had relied on the model by Zhang and Luck (2008) for our analysis, errors systematically biased by non-targets would have simply been subsumed as ‘random’ errors, and we would have no index of misbinding – attributing the feature belonging to a non-target to the target.

We therefore applied the three-component model of response errors to our data (see Analysis and (Bays et al., 2009) for details). Maximum likelihood estimates of the probability of responding at random, the probability of responding with a non-target motion direction and variability in recall of target direction were estimated (see also www.sobell.ion.ucl.ac.uk/pbays/code/JV10/).

There was a significant decrease in κ as sequence length increased from one to four (Fig. 6A, red line; one-way ANOVA, F(3,40)= 11.66, p< 0.001), demonstrating more variability in memory for the target direction in longer sequences. However it’s important to highlight that the decline in κ is not proportional to 1/set size. Assuming for example that a “memory resource” is represented by a pool of neurons, storage of multiple items in memory would result in the sharing of this pool amongst the to-be-rememebred items. However, the firing rates of neurons are corrupted by noise (Bialek and Rieke, 1992), therefore an increase in the number of items in memory will result in an increase in the variability of the population estimate. Theoretical studies using maximum likelihood decoding scheme have shown that the relationship between precision (1/s.d.) and the number of neurons follows a power-law (Seung and Sompolinsky, 1993). Behavioural studies investigating the relationship between number of to-be-remembered items and precision have also shown a power-law relation between the number of items and precision of memory (Bays and Husain, 2008; Bays et al., 2009), consistent with the findings of the present study showing that the number of items and precision of memory do not follow a linear relationship.

The probability of target responses, i.e., responses correctly centred on the target direction of motion, decreased significantly in longer sequences (Fig. 6B, red line; one-way ANOVA, F(3,40)= 17.841, p< 0.001). Participants were more likely to respond with the target direction when presented in shorter sequences, while the probability of non-target responses, i.e., responses incorrectly centred on the directions of motion of non-targets displayed in a particular sequence, increased with increasing sequence lengths (Fig. 6D; one-way ANOVA, F(3,40)= 16.85, p<0.001, red line). This increase in probability of responding to non-target directions can be attributed to misremembering the correct conjunctions of colours and motion directions, i.e. misbinding features of stimuli. Responding at random (i.e. guessing) was very infrequent (<3% of responses) and there was no significant difference between the probability of guessing for different sequence lengths (Fig. 6C, red line; one-way ANOVA, F(3,40) = 1.95, p = 0.14).

There was no effect of serial position on the concentration parameter κ (F<2, p= 0.126) (Figure 7A). Therefore, there was no difference in variability in memory for targets presented at different serial positions within each sequence. However, probability of responding to non-target directions was significantly higher for items presented earlier in a sequence (Fig. 7B; two-way ANOVA, main effect of serial position, F(3,100)= 8.55, p<0.001). This was accompanied by a decrease in probability of responding to target direction (Fig. 7C; two-way ANOVA, main effect of serial position, F(3,100)= 8.50, p<0.001). Therefore, for earlier items in a sequence, participants were more likely to misremember which colour was associated with the target motion direction. There was no significant difference in probability of responding at random at different serial positions of target (F<2, p= 0.378) (Fig. 7D).

Figure 7.

Modelling parameters calculated for each serial position for each participant

A) Concentration parameter for each sequence length and serial position of the target; significant decrease in concentration parameter for longer sequences for the last item only; B) Probability of responding with the target direction for all sequence lengths and serial positions of the target; C) Probability of responding with a non-target direction of motion for all sequence lengths and serial positions of the target; D) Probability of random responses; no significant difference in random responses for different sequence lengths and serial positions. Error bars indicate SEM across participants (N=11).

Together, these results illustrate that the loss in precision observed for earlier items in a sequence is driven by a significant increase of incorrect conjunctions of colour and direction – misbinding – rather than an increase in random responses or a more variable memory for the target direction.

Variability in memory for the target direction presented last in a sequence increased significantly in longer sequences (Fig. 7A; F(3,40)= 3.62, p= 0.02). However, no significant difference was observed for probability of responding to non-target directions or responding at random for last items in different sequence lengths (F< 2, p= 0.126). Therefore for items presented last in the sequence, only κ decreased significantly in longer sequences. On the other hand, for other serial positions in the sequence, there was no effect of sequence length on the κ parameter (Fig. 7A).

What about items earlier than the last one? There was a significant increase in probability of responding to non-target directions for earlier items presented in longer sequences. Probability of responding to non-target direction for the penultimate item was significantly larger when presented in longer sequences (F(2,30)= 5.16, p= 0.012) while probability of responding to target direction was significantly smaller in longer sequences (F(2,30)= 7.81, p= 0.002). A similar pattern of result was observed for 3^rd to last item, with probability of responding to target being larger for 3^rd to last items when presented in sequence of 4 compared to sequence of 3 (t(10)= 2.703, p= 0.022), along with a significant decrease in probability of responding to target in sequence of 4 (t(10)= 2.234, p=0.049) (Fig. 7C).

These findings illustrate two separate sources of error in memory that result in loss of precision in longer sequences for the last item and earlier items in the sequence. For the last item, the loss in precision in longer sequences is a result of an increase in variability in memory, while for items earlier than the last item the loss in precision is primarily caused by an increase in misremembering the correct conjunctions of colours and motion directions – misbinding.

In Experiment 2, we aimed to extend our findings to stimuli that share both spatial and temporal properties using transparent motion, i.e., motion stimuli that overlap at the same spatial location. The design provides a unique paradigm for directly comparing the predictions made by item-limit and dynamic resource models for set sizes below 3-4 items, the capacity limit of the visual system claimed to exist by many previous investigations (Luck and Vogel, 1997; Zhang and Luck, 2008; Anderson et al., 2011; Luria and Vogel, 2011).

Experiment 2

Method

Participants

Eleven healthy volunteers (5 male) with an average age of 29 years old (range: 20-70 years) participated in this experiment. They all had normal or corrected to normal vision and reported normal colour vision. They all provided written consent of the procedure of the experiment approved by the local ethical committee.

Stimuli

The stimuli in this experiment were generated by Cogent toolbox for MATLAB and displayed on a 14.1” flat panel display (resolution: 800 × 600 pixels, refresh rate: 60 Hz). Participants were seated approximately 60cm from the screen in a dimly lit room. This experiment consisted of 3 experimental conditions.

In the “transparent motion” condition, a RDK consisting of 50 dots was presented. The properties of the dots were consistent with those in Experiment 1. Half of the dots (25 dots) were presented in a colour different than the other half and moved in a different direction (coherently). This resulted in the percept of two sheets of transparent motion. In situations where two dots of different colour had identical spatial position, one dot was chosen at random to be presented on top of the other. In the control “sequential motion” condition, a sequence of 2 RDKs were presented at fixation. The properties of the two RDKs were identical to those in Experiment 1.

When participants are asked to estimate the direction of motion of superimposed motion stimuli moving at an acute angle to one another, the angle between the two motion stimuli is often misperceived by few degrees. This phenomenon is called motion repulsion (Levinson and Sekuler, 1976; Marshak and Sekuler, 1979; Mather and Moulden, 1980; Hiris and Blake, 1996; Rauber and Treue, 1999). In order to minimise motion repulsion, minimum angular separation between the two motion directions was set to be above 90 degrees, above which repulsion effect is minimized (Rauber and Treue, 1999; Braddick et al., 2002). A schematic presentation of transparent motion stimuli is presented in Figure 8.

Figure 8.

A schematic representation of transparent motion stimuli.

Two sets of differently coloured dots moved coherently at an oblique angle to one another.

We also used a cued condition to examine the effects of cuing one motion direction as the one to-be-remembered from the two transparent moving surfaces. Thus the “transparent motion cued” condition was identical to the “transparent motion” condition, except that at the beginning of each trial, the colour of the fixation point was used as a 100% valid cue for the colour of the target motion direction that would be probed.

Procedure

Experimental procedure for the sequential motion condition was similar to Experiment 1 except for the following difference. After the display of fixation cross (500msec), a sequence of 2 RDKs were presented, each presented for 2000 msec. The duration of presentation was increased to match the duration of presentation of the stimuli in the “transparent motion” condition.

Each trial in the “transparent motion” condition started with a fixation cross (500msec) followed by the transparent motion stimuli (4000msec). Presentation duration was increased to ensure complete perception of the two motion directions. This was followed by a 500msec blank interval before the probe display was presented.

Each trial on the “transparent cued” condition started with a coloured fixation cross (500msec) that acted as a 100% valid cue for the target motion direction. Participants were asked to only attend to the motion direction with similar colour to the colour of the fixation cross. Transparent motion was presented for 4000msec and was followed by a 500msec blank before the presentation of the probe display.

The probe display was identical to that presented in Experiment 1. Each motion direction had equal probability of being probed in sequential and transparent motion conditions. In the cueing condition, the probe was presented in the same colour as the cued colour. Participants were told to respond as accurately as possible and reaction times were not measured.

Participants were familiarized with the procedure of the experiment by completing 10 trials of each experimental condition prior to the experiment. Each participant completed 60 trials per condition, divided into 2 blocks of 30 and intermixed randomly.

Results

Precision of memory (1/standard deviation of error) was calculated for each condition per participant. We first compared precision of memory for two motion direction presented either in a sequence or simultaneously (transparent condition). Precision was significantly lower for motion directions presented in the transparent motion condition compared to when presented in a sequence (Fig. 9A; t(10)= 3.20, p= 0.01).

Figure 9.

Results for transparent versus sequential stimuli.

A) Precision was significantly higher for items presented in the sequential condition compared to transparent condition; B) Estimated modelling parameter, κ (concentration parameter) illustrates that variability in memory around the target motion direction was significantly lower in the sequential condition compared to the transparent condition; C) Probability of non-target responses were significantly higher in the transparent condition compared to the sequential condition. Error bars indicate SEM across participants (N=11).

In order to distinguish possible sources of error that could result in the observed modulation of precision, we applied the three-component model of error in memory describe previously (see Analysis for details; Bays et al., 2009). Variability of memory around the target direction was significantly greater in the transparent condition compared to the sequential condition (i.e. K was reduced; Fig. 9B; t(10)= 2.54, p= 0.03). Furthermore, the probability of responding with non-target directions of motion i.e. misbinding errors increased significantly in the transparent motion condition (Fig. 9C; t(10)= 2.32, p= 0.04). There was no significant effect of condition on probability of responding with the target direction (t(10)= 1.72, p= 0.12) or responding at random (t(10)= 0.42, p= 0.68). Thus the stored memory of an item was more variable and more prone to misbinding errors in the transparent compared to sequential condition.

We then looked at the effect of attention on precision of memory. Precision significantly improved when the colour of the target direction was cued prior to stimuli presentation in the transparent motion condition (Fig. 10A; t(10)= 7.93, p <0.001). This increase in precision in the cued condition was associated with both a significant decrease in variability of memory around the target direction (Fig. 10B; t(10)= 2.59, p = 0.027) and a significant decrease in probability of responding with non-target directions (Fig. 10C; t(10)= 2.41, p = 0.037) in the cued condition. Furthermore, this decrease in probability of responding to non-target directions was accompanied by a significant increase in probability of responding with the target direction; t(10)= 2.753, p = 0.020. No significant difference between the random responses in both condition was observed (t(10)= 1.826, p = 0.098).

Figure 10.

Results for transparent cued versus uncued stimuli

A) Precision was significantly higher for items presented in the cueing condition compared to transparent condition; B) Estimated modelling parameter, κ (concentration parameter) illustrates that variability in memory around the target motion direction was significantly lower in the cueing condition compared to the transparent condition; C) Probability of non-target responses were significantly higher in the transparent condition compared to the cueing condition. Error bars indicate SEM across participants (N=11).

Together these results show that motion directions presented simultaneously as transparent motion are more variable and prone to misbinding errors. Prioritizing one transparent motion sheet enhances precision, causing a decrease in both variability of memory and misbinding errors. These findings demonstrate the impact of selective attention on working memory precision in this task.

Discussion

Recent studies on precision of visual working memory for orientation, colour and location (Bays and Husain, 2008; Bays et al., 2009, 2011) have provided evidence for a resource model of memory where memory resource is dynamically allocated between visual objects. Here we extend the findings to another visual feature, visual motion, by investigating the precision of memory for motion directions presented both sequentially and simultaneously as transparent motion.

Precision of memory declines in longer sequences

In Experiment 1, motion directions were presented in sequences of varied lengths. We demonstrated a decline in precision for motion directions presented in longer sequences (Fig. 4A). Importantly, a significant drop in precision was observed even when sequence length was increased from one to two items. These findings are contrary to predictions of the item-limit models of memory which propose that visual working memory has a capacity limit of 3-4 “slots” (Luck and Vogel, 1997; Cowan, 2001; Luria and Vogel, 2011) leading to the prediction that the fidelity of memory will not change below the proposed capacity limit. Instead the results are compatible with a limited memory resource (Bays and Husain, 2008; Bays et al., 2009, 2011), where the resource allocated to each item is determined by the number of to-be-remembered items. By extending the scope of the dynamic resource model – which has previously been successfully applied to spatial location, colour and orientation – to the domain of motion, we demonstrate that this model can be taken as a general conceptual framework for visual working memory.

It is important to highlight that in the present study, all participants performed above chance level even for sequence lengths of up to 4 items. However, a previous study has reported a small memory capacity for motion directions, i.e. two items, with memory capacity measured using a change detection procedure (Kawasaki et al., 2008). Although such paradigms have been very useful in understanding visual working memory, they provide a binary-correct/incorrect response measure. We would argue that measuring the precision of memory is potentially a more sensitive index of working memory, allowing us also to test any modulations of the fidelity of the stored items by set size.

Serial order of target influences precision of memory

Serial position or order at which an item is presented within a sequence can also affect the precision with which it was recalled. Last item in a sequence was remembered with higher fidelity compared to other items in a sequence. This finding is in line with previous studies reporting a benefit for the last item presented in a sequence known as the recency effect (Phillips and Christie, 1977; Wright et al., 1985; Neath, 1993; Hay et al., 2007; Blalock and Clegg, 2010). However, measuring precision of working memory, i.e. a measure sensitive to small changes in fidelity of memory, showed that the recency effect was affected by the number of preceding items, with less precision in longer sequences (Fig. 4B, for each serial position). This modulation of last item precision by sequence length appeared to be driven purely by an increase in variability in memory for the last item in longer sequences (Fig. 7A).

For items earlier than the last one in each sequence, precision of memory did not vary significantly between different serial positions, but was significantly lower compared to the last item (Fig. 4B). This drop in precision cannot be explained by the temporal decay of memory (Fig. 5) but rather is a result of interference of other items in the sequence that follow earlier items. Thus it appears that each time an item is added, the resources dedicated to previous items have to be re-distributed to accommodate the added item with enough resources for encoding. Equal precision for all items except the last item in each sequence suggests that memory resource is shared equally between earlier items.

In line with earlier studies investigating working memory (Wright et al., 1985; Neath, 1993; Burgess and Hitch, 1999; Hay et al., 2007; Botvinick et al., 2009; Blalock and Clegg, 2010), we show a recency effect which more importantly was modulated by the number of preceding items (Fig. 4B). Several models have been proposed in the literature to account for the well-replicated effect of recency arguing for retroactive interference whenever a new item is added (Nairne, 1988) or a decrease in temporal distinctiveness for earlier items, resulting in a better available representation for the last item (Glenberg et al., 1983; Burgess and Hitch, 1999; Brown et al., 2000). However, here we show that the magnitude of the recency effect is affected by the number of preceding items. Therefore, the recency effect can also be explained in terms of the allocation of a “left over” share of the memory resource that is determined by the number of previous items.

Interestingly, similar decline of precision was observed at other serial positions (last, penultimate, first, etc.), there was a significant decrease in precision for items presented in longer sequences (Figs. 4B). Therefore at each serial position, the precision of memory and the amount of resource dedicated to that item is influenced by the overall number of to-be-remembered items in that trial. These results are consistent with research investigating the precision of working memory for sequences of orientations (Gorgoraptis et al., 2011), now extending those conclusions to the domain of visual motion.

Models of error at recall in working memory

Recently, two alternative models of working memory distinguishing possible sources of error in recall have been proposed. According to the model proposed by Zhang and Luck (2008), one can distinguish between two possible errors that can result in lower precision for larger set sizes. In larger set sizes, variability of memory for the target direction is higher. Furthermore, participants are more likely to respond at random (‘guessing’) in longer sequences since the probability of not storing an item beyond the capacity limits of memory, as proposed by Zhang and Luck (2008) and “slot” models of memory, is higher for larger set sizes.

However, in tasks similar to the one used in the present study, successful performance depends not only on remembering target motion direction but also on remembering the correct conjunction of colour. This raises the possibility of another type of error that may occur in working memory: misremembering the correct conjunction of colour and motion direction, i.e., misbinding errors. In other words, other items in a sequence that are not probed (non-targets) can systematically bias recall if a feature associated with them (direction of motion) is attributed to the target instead.

This important potential source of error is accounted for in the model proposed by Bays et al (2009) by adding the probability of non-target responses to the model proposed by Zhang and Luck (2008). In order to find the model that best describes our data we compared to the two models (Fig 6). Our analysis demonstrated that the model proposed by Bays et al (2009) provides a significantly better fit (Tables 1 and 2), providing evidence for the existence of a source of error in working memory that was not accounted for in the Zhang and Luck (2008) model. In their schema, misbinding errors (those systematically biased by non-targets) would simply have been subsumed under the category of ‘random’ responses.

Misbinding of object features

As stated previously, successful performance on the precision task used here depends not only on remembering target motion direction but also on remembering the correct conjunction of colour and motion. Therefore, responses to non-target motion directions arise when participants incorrectly bind features across two objects. In this instance, participants may incorrectly bind the colours and motion directions resulting in responses centred around one of the non-target motion directions. Previously, it has been shown that increasing the load of memory, either by increasing the number of items (Bays et al., 2009) or by introducing an extra set of features in memory (Bays et al., 2011), results in misremembering the correct conjunction of features between objects.

In the present study, we show that items presented early in a sequence and in longer sequences are more prone to misbinding errors as opposed to other items in the sequence. Note that participants did not know before each trial how long the sequence would be, so there is no reason to believe they pre-allocated resources for each item in a sequence prior to each trial. Thus they would be expected to allocate all resources to the first item. But if a new item is presented, according to the resource model, some of the resources devoted to the first item now have to be allocated to that item, and so on if further items follow. The findings presented here suggest that the re-distribution of memory resource for earlier items comes with a cost, specifically making these more susceptible to misbinding errors.

But why would such reallocation of resources lead to misbinding? Some insights into this process might be offered from the results of a recent report which investigated precision of memory for objects of different colour and orientation (Bays et al., 2011). Errors in memory for colour and orientation increased with the number of items to be stored but crucially were not correlated, suggesting that these features are stored separately-‘unbound’. Importantly, misbinding errors also increased with memory load but again occurred independently in each feature dimension, suggesting that binding information might in fact be stored independently of feature dimensions, If this is indeed the case, it might explain why, in the present study, increasing number of items to be held in memory is associated with increasing vulnerability to misbinding a non-target motion direction to the target colour. Corruption of feature bindings would be expected to increase in noisy neural representations as the number of items that are stored increases.

Binding and attention

Simultaneous presentation of motion directions as transparent motion surfaces resulted in a decrease in precision as compared to sequential presentation. Precision of memory for motion directions was observed to be more variable and prone to misbinding errors (Fig. 9C), both of which declined by prioritizing (cuing) a motion surface beforehand (Fig. 10C).

Although the experiment was designed to minimize motion repulsion, misperceiving target direction might have contributed to the increase in memory variability around the target direction. However, misbinding errors cannot be explained in terms of such misperception effects. In the transparent motion (without cuing) condition participants had to actively segregate the two directions of motion and encode both directions. Our results demonstrate that this process during encoding has a cost on the precision of memory, explained by an increase in misbinding errors. Conversely, prioritizing one transparent sheet by cuing it in advance led to improvement in precision of memory associated with a significant decrease in misbinding errors.

These findings suggest that selective attention can have an important role in correct binding of features, extending previous reports demonstrating the importance of attention in binding of features in visual working memory (Wheeler and Treisman, 2002). Previous reports have illustrated the dependence of working memory capacity on the ability to attend to task-relevant information (Awh et al., 2006; McNab and Klingberg, 2008), and reported elevation of precision by cuing either the location or colour of the relevant items (Bays and Husain, 2008; Gorgoraptis et al., 2011). Together these results highlight the close relationship between attention and working memory although the exact nature of this connection remains unclear.

Conclusion

Together the results from the present study point towards a limited memory resource that can be dynamically distributed and re-distributed when presented with novel visual objects over time. The re-allocation of resources comes with a cost, manifested in misbinding errors observed for earlier items in longer sequences and the modulation in variability in memory for last items. Furthermore, our results highlight the importance of selective attention in visual working memory specifically for correct binding of visual features in memory. These findings extend previous literature on temporary storage of visual motion and provide further insight into the dynamics of memory distribution for objects presented over time or simultaneously. Results from both experiments challenge item-limit models of memory and provide consistent evidence for the dynamic allocation of resources in memory, now across a range of visual features, including motion.