Attention is Required for Maintenance of Feature Binding in Visual Working Memory

Nahid Zokaei; Maike Heider; Masud Husain

doi:10.1080/17470218.2013.852232

. 2014 Jun 1;67(6):1191–1213. doi: 10.1080/17470218.2013.852232

Attention is Required for Maintenance of Feature Binding in Visual Working Memory

Nahid Zokaei ^1,^2,^✉, Maike Heider ^1,², Masud Husain ^1,²

PMCID: PMC4047630 PMID: 24266343

Abstract

Working memory and attention are intimately connected. However, understanding the relationship between the two is challenging. Currently, there is an important controversy about whether objects in working memory are maintained automatically or require resources that are also deployed for visual or auditory attention. Here we investigated the effects of loading attention resources on precision of visual working memory, specifically on correct maintenance of feature-bound objects, using a dual-task paradigm. Participants were presented with a memory array and were asked to remember either direction of motion of random dot kinematograms of different colour, or orientation of coloured bars. During the maintenance period, they performed a secondary visual or auditory task, with varying levels of load. Following a retention period, they adjusted a coloured probe to match either the motion direction or orientation of stimuli with the same colour in the memory array. This allowed us to examine the effects of an attention-demanding task performed during maintenance on precision of recall on the concurrent working memory task. Systematic increase in attention load during maintenance resulted in a significant decrease in overall working memory performance. Changes in overall performance were specifically accompanied by an increase in feature misbinding errors: erroneous reporting of nontarget motion or orientation. Thus in trials where attention resources were taxed, participants were more likely to respond with nontarget values rather than simply making random responses. Our findings suggest that resources used during attention-demanding visual or auditory tasks also contribute to maintaining feature-bound representations in visual working memory—but not necessarily other aspects of working memory.

Keywords: Visual working memory, Attention, Executive resources, Binding failures

What do working memory and attention systems share? While many authors have pointed out the intimate connections between these two processes—in terms of both behaviour and neural substrate (e.g., Awh & Jonides, 2001; Chun, 2011; Chun, Golomb, & Turk-Browne, 2011; Lavie, 2005; Rensink, 2000; Wheeler & Treisman, 2002)—it remains unclear how much overlap there is between them. Some go as far as defining visual working memory as active maintenance of attention to visual information (Chun, 2011) highlighting the very close relationship between these processes. Indeed, several studies have shown that spatial working memory load interferes with concurrently performed visual search tasks (e.g., Emrich, Al-Aidroos, Pratt, & Ferber, 2010; Oh & Kim, 2004; Woodman & Luck, 2004). Recently, an important controversy has arisen about whether attention is required for the maintenance of feature-bound objects in working memory (Baddeley, Allen, & Hitch, 2011; Chun et al., 2011; Fougnie & Marois, 2009; Luck & Vogel, 1997). Here, we investigate this issue using a relatively new technique that allows us to examine the precision of recall and decompose the types of error made when participants retrieve an item from working memory (Bays, Catalao, & Husain, 2009; Zhang & Luck, 2008).

A central quality of working memory is its limited capacity (Cowan, 2001), a property shared by attention processes (Chun et al., 2011). Cowan has argued for a capacity-limited system for maintenance of information available for conscious awareness, with a limit of about four chunks of information in healthy adults (Cowan, 1998, 2001, 2005). In the visual domain, the chunks are expressed as the number of integrated objects—that is, with all features that belong to an object correctly bound together. In fact, in line with this framework, object-based theories of visual working memory propose that there are a limited number of discrete memory slots, each storing or maintaining information regarding an individual object consisting of different features (Anderson, Vogel, & Awh, 2011; Luck & Vogel, 1997; Luria & Vogel, 2011).

However, within object-based theories of working memory, the role of attention has yet to be established. One line of research has argued that maintenance of bound features in working memory is automatic and demands the same amount of resources as maintaining individual features (Luck & Vogel, 1997; Vogel, Woodman, & Luck, 2001). In a series of pioneering experiments, Luck and Vogel (1997) reported that working memory performance was unaffected by the number of features in the objects to be remembered. Therefore, according to these object-based models of working memory, although initial feature binding relies on attention, maintenance of stored items can be achieved in the absence of attention resources.

Support for such a proposal comes also from studies that employ working memory and attention tasks concurrently. The rationale behind such dual-task studies is to demonstrate overlap in cognitive processes. For example, if memory for binding depends more on attention resources, loading these resources should result in larger impairments in memory in trials where information was stored in a bound form than for “disintegrated” features. Some previous studies have reported similar levels of impairment in memory for independent features and feature-bound objects using dual working memory/attention tasks (Allen, Baddeley, & Hitch, 2006; Allen, Hitch, Mate, & Baddeley, 2012; Baddeley et al., 2011; Johnson, Hollingworth, & Luck, 2008; Luck & Vogel, 1997; Yeh, Yang, & Chiu, 2005; see also Delvenne, Cleeremans, & Laloyaux, 2010; Gajewski & Brockmole, 2006). However, it has recently been argued that the extent to which automatic feature binding occurs in working memory depends on whether the feature-bound objects are perceived as a coherent object (Ecker, Maybery, & Zimmer, 2012). These findings would therefore be consistent with automatic maintenance of bound features in working memory, at least under some conditions.

Contrary to these claims, other researchers have argued for a role of attention in working memory maintenance, emphasizing that in situations where attention is withdrawn, object representations collapse into disintegrated features in working memory (Rensink, 2000; Wheeler & Treisman, 2002). Wheeler and Treisman (2002) proposed a two-stage model of working memory: Within each feature dimension, working memory is limited to a few items but binding is maintained only by relying on attention processes. This model was based on the results of a study where participants were asked to detect either a change among individual features or a change in feature conjunction (binding condition) following a retention period. Crucially, they found that change-detection performance was impaired in trials where participants were asked to detect a change in the binding condition compared to changes in individual or multiple features (Wheeler & Treisman, 2002). The authors argued that since maintenance of integrated objects is more attentionally demanding, more errors in performance arise under the binding condition. Note that these findings were observed only when the whole memory array was presented at probe. Thus the significance of this effect is highly dependent on the methodology employed.

More recently, Chun and colleagues have put forward a taxonomy for attention, arguing that attention can be directed to internal representations—in the absence of sensory information—to maintain feature binding in integrated objects (Chun, 2011; Chun et al., 2011). According to this view, maintenance of bound features within a limited set of objects is dependent on “internal attention” resources. Consistent with this proposal, some studies using dual-task designs have shown large decrements in memory performance for integrated objects, compared to disintegrated features (Brown & Brockmole, 2010; Fougnie & Marois, 2009; Stefurak & Boynton, 1986).

As a result of these contradictory findings in the literature, there still remains a lack of consensus on the role of attention in maintenance of bound objects, despite its highlighted role in many theories of working memory (Chun, 2011; Wheeler & Treisman, 2002). One reason for divergent results might lie in the methodology used. In dual-task studies, attentional resources have so far been taxed using different tasks in the auditory or visual domains—performed either throughout the working memory task (i.e., at encoding, maintenance, and response phases) or at a specific phase (i.e., during maintenance and response phases only; e.g., Allen et al., 2012; Brown & Brockmole, 2010; Fougnie & Marois, 2009; Johnson et al., 2008). Further, until relatively recently, working memory performance has been measured using the change-detection paradigm, with comparisons made between performance across different conditions (e.g., memory for features vs. bound objects). Although change detection has been instrumental in advancing our understanding of working memory, limitations in this design may have resulted in the discrepancies observed in the literature. Recently, it has been shown that the magnitude of change critically influences change-detection performance (Keshvari, van den Berg, & Ma, 2012). Importantly, therefore, small changes can go undetected, particularly with increasing set sizes.

In studies that have focused on the issue of maintenance of bound objects, however, the magnitude of change has been set at an arbitrary value. Moreover, this value is different across different features. This is most vividly demonstrated by varied performance across features: Change detection is better for some features than for others (e.g., Allen et al., 2012; Wheeler & Treisman, 2002). Thus, interpreting performance in a binding condition, where one of two features of an object can change in any given trial, becomes problematic if the change is not equated across feature dimensions. Performance in the binding condition not only is subject to averaging across varied baselines but crucially also depends on the magnitude of change set for each feature in each experiment (Keshvari et al., 2012).

In addition, change-detection performance is influenced by probe presentation: Whole probe displays result in an decrease in accuracy in the binding condition, while single probe displays do not (Wheeler & Treisman, 2002). These limitations of change-detection design, together with differences in methodology and analysis between reported studies, can lead to discrepancies in findings (Allen et al., 2012). Thus, in order to understand the relationship between attention and feature binding in working memory, it is essential to use a task that allows us to investigate both feature only and feature-binding components of working memory without any changes in task difficulty confounding interpretation.

Tasks that measure the fidelity of working memory using adjustment techniques can potentially provide a more sensitive measure than change-detection paradigms (Bays & Husain, 2008; Wilken & Ma, 2004; Zhang & Luck, 2008). In adjustment tasks, participants are asked to reproduce the exact qualities of a stored feature. Using this methodology, it is possible to measure the fidelity or precision of memory on a continuous, analogue scale—rather than depend on binary change or no-change report. Crucially, instead of examining memory performance under tasks of varied difficulty (memory for features or integrated objects), we kept the working memory task constant and instead systematically manipulated the demand of the attention task during the maintenance period using a method previously shown to successfully load attentional resources (e.g., Forster & Lavie, 2008; Lavie, 2005; Lavie, Lin, Zokaei, & Thoma, 2009). Moreover, by applying a recent analytical technique (Bays et al., 2009; Zhang & Luck, 2008) we can distinguish different sources of errors that are systematically modulated by the demand of the attention task. This allows us to directly test whether altering attention demands affects feature binding of objects already maintained in working memory or alternatively influences the number of guesses (random responses) or resolution of feature representations.

In the present study, we examined the effects of deploying increasing levels of attention in a visual search task in maintenance of two objects defined by motion and colour (Experiment 1) and orientation and colour (Experiment 2), and for high levels of visual search difficulty (Experiment 3). In Experiment 4 we aimed to extend the findings from Experiments 1–3 to set sizes above 2. We controlled for increase in attention load rather than working memory load in the visual search task in Experiment 5. Finally, in Experiment 6, we used an auditory task of varied difficulty performed in the maintenance period to examine whether an attention-demanding auditory task would have an effect on feature binding of items maintained in visual working memory.

Experiment 1

First, we investigated whether systematic increase in attentional load of a visual search task during working memory maintenance influences memory resolution. Specifically, we tested whether this manipulation results in an increase in the proportion of misbinding errors, in this case for colour and direction of motion.

Method

Participants

Twelve healthy individuals (5 male) with an average age 25 years (range: 20–35), recruited from University College London participant pool, participated in this experiment. All had normal or corrected to normal vision and reported normal colour vision. They provided written consent to the procedure of the experiment, which was approved by the local ethics committee.

Stimuli and procedure

A schematic representation of a trial of our dual-task design is presented in Figure 1. The general design consisted of participants being asked to hold in memory two different moving stimuli and during the delay perform an attentionally demanding task (high load/low load compared to no load) before being probed on the memory array. Participants were asked to perform both memory and attention tasks to the best of their abilities.

Each trial started with a fixation cross (500 ms) followed by the presentation of two random dot kinematograms (RDKs) on either side of fixation cross (10° of visual angle away from the centre) for 500 ms. Each RDK consisted of dots of a single colour, presented on a grey background on either side of the fixation cross. RDKs were presented in a randomly selected colour from a selection of five easily distinguishable colours (white, red, green, blue, and yellow). Participants were asked to hold in memory the directions of motion of these coloured RDKs (memory array).

Each RDK consisted of 50 dots, each covering 0.1° of visual angle. Dots were displayed within an invisible circular aperture of 150 pixels in diameter (5.7° of visual angle). Dot lifetime was 500 ms, equal to the presentation duration of RDKs. Dots reaching the edge of the circular aperture were repositioned randomly on the other side of the aperture; therefore the averaged dot density was kept constant throughout the presentation. Motion was 100% coherent (constant speed of 4.5 degree/s for all dots). Motion direction was chosen randomly from a value between 0–360° for both RDKs.

In two thirds of the trials, following a blank interval (800 ms), a visual search array was presented briefly for 195 ms, and participants were asked to perform a letter search task. They were required to press X and Z keys on the keyboard if they detected the letter X or Z, respectively, across both high and low load conditions.

In low and high load conditions, the search array consisted of an array of six letters, each positioned on a virtual circle at a radius of 2.5 cm from the centre of the screen. The letters were one of the two target letters (Z or X) and five nontargets or distractors (five Os in low load condition and letters R, K, V, S, L in high load condition). Different distractor letters made discrimination more difficult in the high load condition. Letters were presented in Ariel font and were 40 points in size (each subtending 0.6 by 0.4°). In 50% of the trials the target letter was letter “Z”, and in the remaining trials the target letter was “X”. Participants were asked to respond as fast and as accurately as they could. Auditory feedback (correct or incorrect) was provided on performance in this task. Separation between the two RDK motion directions was 86° on average (SD = 9) for low load trials and 91° (SD = 13) for high load trials. Following the visual search task, a blank interval was presented for 800 ms before the presentation of the working memory probe.

In the remaining third of trials, no visual search task was presented (Figure 1, no load condition). Separation between the two RDKs was 87° on average (SD = 10) in this condition. Following the presentation of the two RDKs, a blank interval of 1795 ms was displayed before the presentation of the probe display. All trial types were randomly interleaved within a block.

The probe stimulus consisted of a circle (5.7° of visual angle in diameter) presented at the centre of the screen with an arrow positioned at a random orientation—drawn from the uniform distribution (0–360°)—within the circle. The probe stimulus was presented in the same colour as one of the two RDKs in the memory array. Participants were asked to adjust the orientation of the arrow, using a mouse, to match the direction of motion of the RDK presented in the same colour in the memory array. Participants then had to press the left mouse key to confirm their response. The probability of probing any of the RDKs was kept constant for each RDK. The probe display was presented until response. Participants were informed to give equal weight to both the working memory and the visual search task in each trial.

Stimuli were generated by Cogent toolbox (www.vislab.ucl.ac.uk/Cogent/) for MATLAB and were displayed on a 21-inch CRT monitor (refresh rate: 60 Hz). A chin rest, positioned at 60 cm from the screen, ensured a 60-cm distance from the monitor.

Participants completed three blocks of 60 randomly intermixed trials per visual search condition (i.e., no, low, and high conditions) in a dimly illuminated room. Prior to the start of the experiment, participants were acquainted with the experimental apparatus and conditions by a gradual increase in the complexity of the practice trials.

Analysis

In each trial, both reaction times (RTs) and accuracy in the visual search task under both load conditions were calculated. In addition, overall performance in the working memory task was computed for each attention load condition as 1/standard deviation of error in recall, labelled precision. The parameter space for motion direction is circular; therefore we used Fisher's definition of standard deviation for circular data (Fisher, 1993), subtracting this value form the expected value for chance. Therefore zero precision corresponds to chance-level precision.

Overall performance (defined here as precision of recall) does not provide information about the sources of error. In order to distinguish between these, we applied a probabilistic model proposed previously by Bays et al. (2009). This allows us to isolate the sources of error affected by loading attention resources. According to this model, there are three sources of errors in recall:

A Von Mises (circular Gaussian) distribution in memory centred on the target value (Figure 2A).
A Von Mises distribution in memory centred on nontarget value—that is, around the direction of motion of dots in the RDK that was not probed (Figure 2B).
A uniform distribution of error corresponding to random error or guessing (Figure 2C).

Figure 2. — Three sources of error in memory used for modeling performance. (A) A Von Mises (circular Gaussian) distribution with concentration parameter κ, centred on the target value, capturing variability in memory for target, with the area under the distribution (shaded) being proportional to the probability of responding to the target; (B) Von Mises distribution with concentration parameter κ, centred on one of the non-target value, resulting from errors in identifying which target value belonged with the target colour (misbinding). The area under the distribution corresponds to the proportion of non-target responses and (C) A uniform distribution of error corresponding to random error, with the area under this distribution corresponding to the proportion of random responses.

The model is described by the following equation:

μ (σ) = α ϕ_{κ} (θ - 0) + β \frac{1}{m} \sum_{i}^{m} ϕ_{κ} (σ - ϕ_{i}) + γ \frac{1}{2 π}

Where θ corresponds to the target direction, σ is the response direction, and $ϕ_{κ}$ is the Von Mises (circular Gaussian) distribution with a mean of zero and concentration parameter κ. The concentration parameter κ corresponds to the variability of recall of the target computed by the model, where greater κ corresponds to lower variability in the distribution. (Note that we use the term “precision” here as a measure of an individual's precision of recall. This provides no information regarding the exact sources of error. The model estimate for variability around the target item, labelled as concentration parameter κ, provides an index of one source of error contributing to overall behavioural precision.) The probability of responding with the target direction is given by α [also referred to as p(T)].

Successful performance in tasks similar to the one here requires memory for correct combination of motion direction and colour. Therefore error can arise as a result of incorrect conjunction of colour and motion direction (misbinding): trials where participants make an error centred on the nontarget; β corresponds to the probability of responding with such misbinding errors [also referred to as p(NT), or probability of making nontarget errors]. ${ϕ_{1}, ϕ_{2}, \dots ϕ_{m}}$ correspond to motion directions of m nontarget items (here m = 1). Probability of responding at random (γ) is calculated as 1 – α − β and corresponds to the proportion of trials where participants were guessing [also referred to as p(U)]. Maximum likelihood parameters of α, β, γ, and κ were obtained using expectation maximization (Myung, 2003) for each participant and under each search condition (full details in Bays et al., 2009).

However, it is important to note that model comparison between different model estimates is not possible to conduct in a straightforward manner. Hence a null effect of attentional load on one parameter and not on another does not necessarily mean that there is a difference between the effects of attention on the two model parameters (Gelman & Stern, 2006).

Results and discussion

Effect of load manipulation on visual search performance

High load visual search task resulted in significantly longer mean RT and decrease in mean accuracy compared to the low load condition (744 vs. 596 ms and 83% vs. 94%), t(11) = 6.168, p < .001, for accuracy, and t(11) = 9.368, p < .001, for RT. This confirmed that attentional load was successfully manipulated in the visual search task. Trials with an incorrect response in the visual search task were excluded from analyses of the working memory task performance.

Visual working memory performance

The key question we wished to address is whether increasing attention load in the search task would affect memory for items already stored in working memory. Precision of working memory representations—that is, 1/distribution of error in response (see Analysis section)—was affected by visual search load manipulations [main effect of search load, F(2, 22) = 5.63, p = .011]. There was a significant decrease in the fidelity of memory representations under high load compared to no load, t(11) = 3.688, p = .004, and low load conditions [t(11) = 2.454, p = .032; ns after Bonferroni correction]. There was no significant difference in performance between no load and low load search conditions, t(11) = 0.949, p > .05. Thus, overall working memory performance is influenced by visual search load/difficulty.

However, the overall performance does not inform us on the sources of error that are affected by visual search load—that is, whether decrease in working memory performance was due to increase variability in response for the target feature (κ), changes in proportion of target responses [p(T)], proportion of nontarget responses [p(NT)], or increased random responses [p(U)]. Thus, we next examined the distribution of errors in relation to the target direction under different conditions (i.e., no, low, and high load conditions). As illustrated in Figure 3A, the proportion of responses falling close to the target direction decreased systematically as the attentional load increased. This is illustrated as a decrease in the peak of response distribution around zero—that is, target value under high load condition. Furthermore, the longer tails of the distribution under high load condition provide evidence for additional source of error (either guessing or misbinding errors) that may occur in this condition (Figure 3A). Importantly, the proportion of responses centred on the nontarget motion direction increased in the high load condition (Figure 3B). Therefore, under high load conditions, increased errors arose as a result of responding to the nontarget direction. In other words, participants erroneously misbound the colour of the probed RDK (target) with the direction of motion of the RDK that was not probed (nontarget). This is clearly illustrated in an increase in the peak of response distributions around the nontarget value under high attention load.

Figure 3. — Distribution of errors relative to target and non-target motion direction, Experiment 1. (A) Frequency of response as a function of the difference between the response and the target motion direction. Under high load condition, the variability in recall of the target direction (width of the distribution) increase and the peak of the distribution centred around target value (zero) decreases. (B) Frequency of response as a function of the difference between the response and the non-target motion direction. There is a larger proportion of responses around the non-target direction under high load condition compared to the other two conditions. Error bars indicate SEM (N = 12).

To quantify the possible sources of error in memory affected by visual search load, we applied a three-component model of response error to our data (see Analysis section). Maximum likelihood estimates of the probability of responding at random, the probability of responding with the target and nontarget motion direction, and variability (concentration parameter, κ) in recall of target direction were estimated. Subsequent to model fitting, outlier values were excluded. Model estimates for κ and random responses for one participant were 2.5 standard deviations above the mean values, and the probability of target responses for the same participant was 2.5 standard deviations below the mean value. Therefore for the remaining analysis, this participant was excluded.

There was no significant difference in κ (i.e., variability in performance for the target motion direction; Figure 4A). Proportion of target responses varied significantly under different load conditions, F(2, 30) = 3.61, p < .039. There was a significant decrease in proportion of target responses under high load compared to no load, t(10) = 4.9, p < .001, and low load conditions [t(10) = 2.291, p = .045, ns after Bonferroni correction]. Further there was a marginally significant decrease in proportion of target responses under low load compared to no load condition, t(10) = 2.303, p = .052. The modulation in proportion of target responses under different load conditions was accompanied by changes in the proportion of nontarget responses. There was a significant increase in proportion of nontarget responses under high load condition compared to no load, t(10) = 6.5, p < .001, and low load conditions, t(10) = 3.5, p = .006 (Figure 4B). There was no significant difference in proportion of random responses under different search conditions (Figure 4B). The same pattern of results was obtained when all trials, regardless of search accuracy, were included in the analysis.

The results from Experiment 1 demonstrate an increase in the proportion of responses centred on the nontarget motion direction in trials where the visual search load/difficulty of the secondary task was high without any change in variability of recall for the target motion direction. Therefore, loading visual attention (by a secondary visual search task) during working memory maintenance for motion directions specifically results in feature-binding failures for items maintained in memory.

Experiment 2

In order to establish the generality of the relationship between attention and feature binding in working memory, we aimed to extend these findings to another visual feature. In Experiment 2, we applied similar methodology to a visual working memory task but now for remembering the orientation of coloured bars, rather than remembering the direction of movement.