Visual Working Memory Capacity: From Psychophysics and Neurobiology to Individual Differences

Abstract

Visual working memory capacity is of great interest because it is strongly correlated with overall cognitive ability, can be understood at the level of neural circuits, and is easily measured. Recent studies have shown that capacity influences tasks ranging from saccade targeting to analogical reasoning. A debate has arisen over whether capacity is constrained by a limited number of discrete representations or by an infinitely divisible resource, but the empirical evidence and neural network models currently favor a discrete item limit. Capacity differs markedly across individuals and groups, and recent research indicates that some of these differences reflect true differences in storage capacity whereas others reflect variations in the ability to use memory capacity efficiently.

The Rise of Visual Working Memory

For many decades, the concept of working memory capacity has played a central role in large-scale theories of cognition [1, 2] and in explaining individual differences in cognitive ability [3]. Originally, research on working memory was dominated by verbal paradigms, such as digit span tasks that require participants to repeat back a series of digits and complex span tasks in which participants must alternate between two tasks that require processing information and storing items in memory [4]. However, the last fifteen-year period has seen an explosion of research on visual working memory (VWM, as defined in Box 1).

Box 1. What is Visual Working Memory?

The term working memory is used in many different ways, and it is therefore important for researchers to define exactly what they mean when they use this term. In this article, we define visual working memory as the active maintenance of visual information to serve the needs of ongoing tasks. We are not suggesting that this is the only valid definition of visual working memory. Instead, this definition is intended to provide a clear statement of the scope of the memory system that we are addressing in this review. We also believe that it reflects the typical usage of the term by researchers who come from a vision science perspective.

Our definition has three key components. First, to qualify as VWM, it is not sufficient that the information was acquired through the visual modality; the representation of the information must be visual in nature. If the observer stores a verbal or amodal conceptual representation of the sensory input, we no longer consider it to be a visual memory. Second, VWM is based on active maintenance. That is, a VWM representation is maintained by a change in sustained, energy-requiring neural activity rather than by a change in synaptic strength (which can be verified with physiological recordings). This distinguishes VWM representations from passively stored, longer-term memories. Third, the representations must be used in the service of broader cognitive tasks. This is theworking part of VWM.

In this article, we will review some of the key findings of this research, focusing on the cognitive and neural mechanisms of VWM capacity and on individual and group differences in VWM capacity. We begin by asking why vision needs a working memory system. We then discuss whether capacity is constrained by a limit on the number of discrete items that can be represented or by a limit on a resource that can be divided among large numbers of items. We then discuss how and why capacity varies among individuals and between groups. Finally, we discuss the neural mechanisms that may determine the capacity of visual working memory. Our overall perspective is that limits on VWM capacity reflect the broader problem of maintaining multiple active representations in networks of interconnected neurons. This problem can be solved by maintaining a limited number of discrete representations, which then impacts almost every aspect of cognitive function.

Why Study Visual Working Memory?

There are at least four major reasons for the explosion of research on VWM capacity. First, studies of change blindness in the 1990s (Figure 1a) provided striking examples of the limitations of VWM capacity in both the laboratory and the real world [5, 6].

Figure 1.

(a) Example of a change blindness task. Many cycles are required before an observer notices the difference between the two images. Reprinted from [10] by permission of Oxford University Press, USA. (b) Example of a change detection task [8]. A brief sample array is followed by a blank delay and then a test array. The test array is either identical to the sample array or differs in one feature of one of the objects, and the observer indicates whether a change was present. In the change localization variant, a change is always present and the subject indicates which item changed [47, 60, 79]. (c) Hypothetical results for an observer with a capacity (K_max) of 4 items, assuming a slot model. Accuracy (% correct) is perfect when the set size (N) is less than K_max (assuming that changes in color are very large, when present). When N > K_max, the changed item will be present in memory on N/K_max trials, and subjects will fail to detect the change when the changed item is not in memory. Accuracy will therefore decrease systematically as N increases above K_max. By taking into account guessing, it is possible to estimate the number of items that the observer must have had in memory (K) at each set size [80–83]. (d) Data from an actual experiment with college student subjects [8]. (e) Scatterplot of the relationship between storage capacity (K_max) measured in a 10-minute change localization task and a measure of broad cognitive function (the T score from the MATRICS battery) in a sample of subjects including both schizophrenia patients and matched controls [13]. The correlations were similar in both groups, justifying an aggregated analysis.

Second, the change detection paradigm (Figure 1bcd) was popularized to provide a means of studying the same basic phenomenon with more precisely controlled visual displays [7–9]. This paradigm made it possible to quantify the capacity of VWM and to link VWM to the enormous body of research on vision [10]. Moreover, this task is so simple that even pigeons can do it [11].

Third, estimates of VWM capacity have excellent psychometric properties when optimal methods are used [12, 13]. For example, a 10-minute change localization task yielded test-retest reliability of 0.77 with testing episodes separated by 1.5 years [13]. In addition, VWM capacity is highly correlated with measures of broad cognitive function, accounting for 43% of individual differences in a global fluid intelligence [14] and 46% of individual differences in overall performance on a broad battery of cognitive tasks [13] (Figure 1e). These high correlations are particularly striking given that the change detection paradigm provides a relatively simple measure of VWM capacity, with little or no impact of long-term memory when canonical task parameters are used. Specifically, there is little or no effect of proactive interference [15, but see 16, 17] or medial temporal lobe lesions [18, but see 19] with canonical parameters.

Finally, researchers have discovered neural correlates of VWM maintenance that are strongly correlated with individual differences in VWM capacity. In studies of VWM in nonhuman primates, neurons in several brain areas exhibit elevated firing rates and increased synchrony during the delay interval [20, 21]. In human event-related potential (ERP) studies, an analogous sustained change in voltage is observed during the delay interval of the change detection task [22, 23]. This effect is called contralateral delay activity (CDA) because it is found in the hemisphere contralateral to a set of lateralized objects that are being remembered over a delay period (see Figure 2). CDA amplitude increases as the set size increases, reaching an asymptote at the capacity limit (typically 3–4 items). This is true both at the group level and the single-subject level, with a very strong correlation between an individual’s behaviorally measured VWM capacity and that individual’s CDA asymptote point (Figure 2d). An analogous effect can be seen in functional magnetic resonance imaging (fMRI) studies, where the signal in the intraparietal sulcus (IPS) during the delay period increases as the set size increases, reaching an asymptote at the individual subject’s VWM capacity [24, 25].

Figure 2.

(a) ERP paradigm for recording contralateral delay activity (CDA) [22]. Subjects are instructed to remember the colors of the items on the side indicated by the arrow and report whether a color has changed on that side in the test array. (b) ERP waveforms from ipsilateral versus contralateral electrode sites relative to the side of the array that was encoded into memory. Time zero is the onset of the test array, and the CDA is the difference in voltage between the ipsilateral and contralateral waveforms during the delay period. Negative is plotted upward. (c) CDA amplitude as a function of the number of items on the to-be-remembered side, averaged over subjects. Note that CDA amplitude reaches asymptote near the average working memory capacity limit. (d) Scatterplot of individual subjects, showing that individual differences in working memory capacity (K_max) are correlated with differences in CDA asymptote (quantified as the difference in CDA amplitude between set sizes 2 and 4).

The Role of Working Memory in Vision

Visual information is typically acquired during short periods of fixation (usually 200–500 ms in duration) separated by saccadic eye movements that temporarily suppress processing and then shift the retinal image. Some kind of memory is needed to bridge the temporal gaps and spatial shifts created by eye movements [26]. Seminal research by Irwin and his colleagues [27, 28] demonstrated that iconic memory does not survive eye movements but that VWM can be used to link the pre-saccade representation of an object at one retinal location with the post-saccade representation of that object in a different retinal location. More recent research has shown that the target of an upcoming eye movement is automatically stored in VWM, and then after the eye movement this VWM representation is compared with the newly fixated object [29]. In addition, eye movements may be biased toward objects that match the current contents of VWM [30], and even the simplest saccades are faster if the saccade target matches the current contents of VWM [31].

VWM also plays a key role in higher-level visual tasks, demonstrating that it is truly a working memory. For example, when the target for a visual search task is cued in a trial-by-trial manner, the cue is stored in VWM, leading to a CDA in the interval between the cue and the search array [32]. In this situation, search performance is impaired if VWM is filled to capacity by a secondary object memory task [33]. However, after several trials of searching for the same target, the CDA disappears [32]. Moreover, search performance is no longer impaired by a concurrent VWM load when the target remains the same on trial after trial [34]. These results indicate that the “search template” is transferred from VWM into a longer-term memory store when the target remains constant over several trials.

It is natural to assume that working memory’s sole purpose is to store items that are no longer present, but recent research indicates that the same system is also used to represent information that is currently visible. For example, both VWM capacity limits and neural indices of VWM activation have been observed in tasks in which the items remain visible throughout the trial, such as visual search [35, 36] and multiple object tracking [37, 38]. A recent study took this a step further and showed that when observers were asked to remember the colors of items in a static array that remained visible for the entire trial, their VWM capacity was indistinguishable from trials in which the items disappeared during a 1-s retention period [39]. Moreover, the CDA was the same whether or not the stimuli were visible during the retention period, indicating that the same neural mechanisms are used to represent visual objects whether or not they are currently visible. Together, these findings suggest that VWM may not really be a memory system per se, but may instead be a general-purpose visual representation system that can, when necessary, maintain information over short delays.

The Nature of Capacity Limitations in Visual Working Memory: Slots versus Resources

When memory for simple, highly discriminable colored squares is tested, the typical college student has a capacity of only 3–4 objects’ worth of information [40]. What is the nature of this limit? Object complexity plays a clear role: Task performance is less accurate for complex objects than for simple objects in most cases [41, 42]. However, complexity is not a very well defined term [10, 43], and it is much easier to understand capacity limits for simple, unidimensional features.

Two main classes of theories of VWM capacity have been proposed, a discrete slots class and a continuous resource class [10, 44–46]. Figure 3a shows how a display of 6 colored squares would be represented in VWM according to these two classes of theories. Slot-based theories assume that a limited number of items, K_max, can be stored in VWM; if the number of items in the sensory input is greater than K_max, then K_max of the items are stored in VWM and no information about the other items is stored in VWM. Note, however, that internal and external sources of variability will cause each representation to be imperfect and may cause K_max to vary from trial to trial.

Figure 3.

(a) Essence of the continuous resource and discrete slot classes of models. (b) Example of the continuous report task with color stimuli. The cue (thicker box) indicates which item should be reported by clicking on the color wheel. (c) Hypothetical distribution of response errors (difference between actual color and reported color) according to the slot model [48]. If the cued item is present in memory (violet line), the errors will be normally distributed around the correct value (the Von Mises distribution is used for circular dimensions such as hue). If the cued item is not remembered (brown line), errors will be random (a uniform distribution). The actual data consist of a weighted sum of these two distributions (black line). (d) Observed data from set sizes 3 and 6, and estimates of the parameters of the underlying distributions [48]. (e) Continuous report task for orientation [49]. The sample array contains circles with gaps; when the test display appears, the subject reports the remembered orientation of the gap in the item that is cued by the thicker circle; the report the orientation by clicking on the corresponding location on the cue circle. (f) Standard deviation (SD) of the distribution of response errors in the task shown in (e) as a function of set size. The group data are well fit by a function that rises linearly, has an inflection point at the average K_max, and is then flat. (g) Inflection point as a function of K_max for individual subjects, showing that the point at which the SD reaches asymptote for a given subject is predicted by that subject’s VWM capacity.

Resource-based theories assume that VWM capacity is a flexibly divisible resource that can be spread among all the items in the display, but with fewer resources per item and therefore reduced precision as the set size increases. These theories can also be framed in terms of an increase neural noise as the set size increases.

Evidence for Discrete Slots from Continuous Report Experiments

The most obvious way to distinguish between these classes of theories is to determine whether VWM representations become less precise as the set size increases, and this is most easily assessed with the continuous report paradigm shown in Figure 3b [47, 48]. Each trial begins with a sample array followed by a retention interval and then a test display. The test display includes a cue at one location, and the observer clicks on a color wheel to report the remembered color of the cued item. If the cued item is present in memory, the reported color should be close to the actual color (i.e., the magnitude of the error will be small), and the distribution of error magnitudes over many trials would be expected to be approximately normal (see Figure 3c). The width of this distribution (quantified as the standard deviation, SD) is inversely related to the precision of the memory representation. The same method can be used for other feature dimensions, such as orientation [49, but see 50] and shape [48].

When the set size exceeds the number of items that can be stored in VWM (K_max), the cued item will not be present in memory on a subset of trials. When this happens, the observer will guess randomly, leading to a uniform distribution of errors (Figure 3c). Because the data from a set of trials may contain a mixture of in-memory and out-of-memory trials, the observed distribution of errors will consist of a mixture of a normal distribution (from in-memory trials) and a uniform distribution (from out-of-memory trials). This is equivalent to a normal distribution that has been shifted vertically by an amount that is related to the probability that the cued item was absent from memory. From the observed mixture, it is possible to derive 2 key VWM parameters: P_mem, the probability that the cued item was present in memory; and SD, the width of the normal distribution. The number of items being held in memory for a given set size (K) is simply P_mem x set size.

Figure 3d shows that increasing the set size from 3 to 6 produces a large vertical shift in the distribution of responses, with no substantial change in the width of the normal portion of the distribution [48]. Thus, increasing the set size appears to decrease the probability that the cued item is present in memory (P_mem) without changing the precision (SD) of the representation. These data are consistent with discrete slot models and are incompatible with most resource-based models.

Although precision did not change significantly between set sizes 3 and 6, precision did improve when the set size was reduced below three items. This is best illustrated by a study in which subjects remembered orientation information rather than color information (Figure 3e) [49]. SD increased linearly as set size increased up to a point and then reached an asymptote (Figure 3f). Interestingly, the infection point in this bilinear function was closely related to VWM capacity (Figure 3g). These results suggest that resources can be shared among items until a maximum number of items (K_max) is reached [49, 51].

Evidence for Continuous Resources

Evidence against discrete slots and in favor of continuous resources was provided by Bays and Husain [46] using variations on the spatial memory paradigm shown in Figure 4a. A sample array of colored squares was presented, followed after a delay by a probe stimulus, and the task was to report whether the probe stimulus was displaced to the left or to the right of the corresponding item in the sample array. Slot-based models predict that observers should make errors at large set sizes even for very large displacements (when no information about a given item is present in VWM). In contrast, resource-based models predict that performance should remain near perfect for sufficiently large displacements. The latter pattern was found (Figure 4b). Moreover, the precision of the representations declined monotonically as the set size increased.

Figure 4.

(a) Spatial VWM paradigm of Bays and Husain [46]. Observers report whether the probe was displaced leftward or rightward relative to the corresponding sample item. (b) Results from set size 8 in a replication experiment [52]. The X axis shows the amount of displacement of the probe relative to the original item, with negative values indicating a leftward displacement and positive values indicating a rightward displacement. The Y axis shows the probability that the subject reported a rightward displacement. When the displacement was large, subjects were nearly perfect: they nearly always reported a rightward displacement for a large rightward displacement and almost never reported a rightward displacement for a large leftward displacement. Bays and Husain argued that this nearly perfect memory for large displacements at set size 8 was strong evidence against the slot model and in favor of the resource model, but later research showed that these results could be explained by a guessing strategy [52]. (c) Results when the task was changed slightly to eliminate the guessing strategy. Observers were no longer nearly perfect for large displacements. Panels (b) and (c) were reprinted from [52] with kind permission from Springer Science and Business Media.

Although this appears to be strong evidence for continuous resources, it appears to reflect a guessing strategy. If, as shown in Figure 4a, the probe is near the left edge of the display, the subject can guess that it was a leftward shift even if the corresponding sample item was not stored in memory. Indeed, a subsequent study [52] showed that near-perfect performance can be obtained for large displacements when this guessing strategy is possible (Figure 4b), but performance does not reach ceiling when the task is modified to prevent this strategy (Figure 4c).

A more compelling challenge to slot-based models was provided by van den Berg et al. [44], who proposed a new resource-based model in which the precision varies randomly from trial to trial. According to this model, the distribution of errors in continuous report tasks (e.g., Figure 3d) reflects the average of many different normal distributions with different widths (SDs). A careful analysis showed beyond a doubt that a single normal distribution systematically misfits the actual distribution of responses and that a mixture of multiple normal distributions more accurately fits the observed distribution.

This model also proposes that the amount of variation in the SD increases as the set size increases. At large set sizes, the memories will sometimes be so imprecise that very large errors will occur, making it seem as if observers are guessing randomly. Consequently, the frequent occurrence of extreme errors at set size 6 in Figure 3d may reflect very poor memory precision on a subset of trials, not the complete absence of a representation of the tested item. However, it remains to be seen whether this variable-precision resource model fits the data better than a slot-based model in which precision is allowed to vary from trial to trial (as would be expected in any imperfect storage system) but does not increase with set size.

Sims et al. [53] proposed a very different resource-based model in which VWM capacity can be conceived in terms of classic information theory. In this model, the sensory input is optimally recoded so that it can be represented in terms of a specific number of bits of information. The model predicts that, because of optimal recoding, observers will be able to retain more precise information when the range of possible values is small than when the range of possible values is large, and this prediction was confirmed. If visual information can be arbitrarily recoded in abstract bits, this naturally brings up the question of whether the representations are still visual. It remains to be seen whether the storage of this recoded information occurs in visual cortex or instead occurs in a more generic, amodal working memory system [54].

Another possibility is the VWM capacity limits are a by-product of competition between similar representations [55, 56]. However, this would predict that capacity would be lower when the to-be-remembered items are similar to each other, which is inconsistent with the available evidence [41, 57].

Additional Evidence for Discrete Slots

Converging evidence from multiple experimental paradigms will be needed to distinguish between the broad classes of slot-based and resource-based models. The data from set size manipulations such as those shown in Figures 3 and 4 have not yet been conclusive, but three additional approaches have provided evidence in favor of discrete slots.

First, Rouder et al. [45] recorded confidence judgments from observers in a change detection task so that receiver operating characteristic (ROC) curves could be constructed. When large change magnitudes are used, the all-or-none storage posited by slot models should lead to a linear ROC curve, whereas low-resolution memory representations will lead to a bowed ROC curve. The observed ROC curves were close to linear, supporting the slot model’s assumption of all-or-none memory encoding.

Second, Zhang and Luck [48] combined the color wheel paradigm shown in Figure 3b with a spatial cuing manipulation. The sample array contained two items along with a spatial cue, which could either be valid, invalid, or neutral. Resource-based models would predict that precision would be very high on valid trials, intermediate on neutral trials, and very low on invalid trials (because the invalidly cued objects should have “just a few drops” of resources). Slot-based models, however, predict that precision should be the same on neutral and invalid trials (because it’s not possible to have “just a few drops” of resources in these models). Instead, the probability of having any representation at all should decline on invalid trials. This is exactly what was found.

Third, a more recent study [58] asked whether observers could trade precision for capacity, increasing the number of items stored in VWM beyond the typical K_max by decreasing the precision of the representations. Observers were never able to increase K_max by reducing precision, even when given monetary incentives to do so. This is strong evidence against the idea that resources can be allocated flexibly to increase the number of VWM representations.

Fourth, if observers devoted all of their resources to the items in the display, no matter whether 1 or 20 items were present, then it is difficult to explain why ERP and fMRI measures of VWM delay activity increase as the set size increases from 1 up to the individual observer’s K_max and then reach an asymptote [22, 24]. Resource models would instead predict that delay period activity should be constant as long as observers are devoting all their resources to the task.

Multiple sources of evidence therefore support the idea of discrete slots. However, state-of-the-art resource models [44, 53] have not yet had an opportunity to explain these other results. Thus, significant progress is being made and many specific models have been ruled out, but additional research will be needed to provide a definitive evidence distinguishing between these two broad classes of models.

Individual and Group Differences in Visual Working Memory Capacity

As discussed earlier, VWM capacity is a stable individual difference and is impressively correlated with measures of higher cognitive function (Figure 1e). In addition, substantial differences in VWM capacity can be observed across groups. For example, K_max is lower in people with schizophrenia than in healthy control subjects [59, 60], with a very large effect size (Cohen’s d) of 1.11 in a study of 99 patients and 77 controls [13]. Moreover, the reduction in K_max accounted for approximately 40% of the patient group’s impairment on a measure of broad intellectual function. Assuming that reduced capacity causes reduced intellectual function (see Box 2), this finding suggests that 40% of the cognitive deficit in people with schizophrenia could be eliminated by a treatment that normalized their VWM capacity.

Box 2. Outstanding Questions.

• Debate is continuing about whether VWM is best conceived as a set of discrete slot-like representations or as a flexible, continuous resource. Substantial progress has been made, and many specific models have been ruled out by the data. However, creative new experimental designs and analytic procedures are needed before we can definitively distinguish between these broad classes of theories.

• Most models assume that trial-by-trial variations in memory (e.g., the distribution of errors shown in Figure 2d) reflect neural noise. However, much of this variance could instead reflect systematic differences across trials (e.g., differences in the specific stimuli being remembered). The source of this variance is therefore an important issue for future research.

• Studies of individual differences typically assume that having more memory capacity causes people to perform better on broader tests of cognitive ability. However, the direction of causation may actually run in the opposite direction. That is, smarter people may figure out better ways to perform working memory tasks. Determining the actual direction of causality will be vitally important in future research.

• If variations in VWM capacity actually cause variations in overall cognitive ability, then the next obvious question is whether it is possible to improve VWM capacity and thereby improve overall cognitive ability. This is a currently a hot topic among working memory researchers, but no clear answer has yet emerged.

Several studies have attempted to determine the underlying causes of individual and group differences in VWM capacity. Among healthy young adults, Vogel and colleagues have found that the differences in VWM capacity can be attributed to differences in attentional processes that are responsible for filtering out irrelevant information. For example, when given arrays containing both relevant and irrelevant objects, low-capacity individuals tend to encode irrelevant information into VWM to a greater extent than high-capacity individuals [23]. An fMRI study indicated that this reflects impaired connectivity among prefrontal cortex, the basal ganglia, and parietal cortex [61]. In addition, low-capacity individuals are slower than high-capacity individuals to recover after attention is captured by irrelevant information [62]. Thus, differences in K_max among healthy young adults may not reflect the capacity of VWM per se, but may instead reflect variations in filtering processes that control the encoding of information into VWM.

Although attentional differences explain the bulk of the between-subject variability in VWM capacity among healthy college students, other factors appear to contribute to group differences. For example, K_max is reduced in patients with Parkinson’s disease compared to healthy control subjects, but only part of this can be explained by impaired filtering; these patients also appear to have a reduction in VWM storage capacity per se [63]. Similarly, K_max declines in aging, as does filtering efficiency, but the timing of the filtering efficiency differences between younger and older adults is not the same as the timing of differences in filtering efficiency between low- and high-K_max younger adults [64, 65].

In addition, the reduced K_max exhibited by schizophrenia patients does not appear to reflect impaired filtering at all. First, schizophrenia patients show the same ability to exclude irrelevant distractor items as control subjects [60]. Second, CDA amplitude is actually greater in schizophrenia patients than in control subjects when a single object is stored in memory, but is decreased in patients compared to controls when 3 or 5 items must be stored [66]. This difference was observed even in subsets of patients and controls with equivalent K_max values. The patient impairment may reflect a tendency to hyperfocus on a small number of items, directly reducing VWM capacity. Schizophrenia patients may also tend to hyperfocus on perceptually salient information: They exhibit impaired filtering [67] and slowed disengagement [68] when faced with high-salience distractors.

Neural Mechanisms of Visual Working Memory

The simplicity of the change detection task makes VWM amenable to neural network modeling and electrophysiological recordings. Most neural network models assume that VWM representations are maintained by recurrent feedback loops, in which information flows from one set of neurons to another and then back again [69] (see Figure 5a). This explains the persistence of the memories, the increased neural activity observed during the retention interval, and the finding of oscillations as the activity bounces back and forth among neural populations.

Figure 5.

Neural representation of three cell assemblies (groups of neurons coding separate objects in VWM). Each cell assembly consists of a group of neurons from one or more cortical areas. In some models, neurons are recruited to a specific cell assembly at the moment of encoding to represent the features of the object being encoded, and a given neuron may be allocated to different cell assemblies depending on the information being stored in memory. (a) Groups of neurons coding a given object form local recurrent loops within an area (small U-shaped arrows) and long-range recurrent loops between areas (large arrows). The recurrent connections cause the activity to be maintained over time, and the activity oscillates as it bounces back and forth between neurons (both within and between cortical areas). Most models include only one or two cortical areas (e.g., inferotemporal and prefrontal cortex), but many different areas are likely synchronized in this manner. (b) The neurons in a given cell assembly spike together briefly (represented by vertical lines), and then the activity decays. The different cell assemblies spike at different times, minimizing interference between them. However, a given cell assembly must spike again before it decays too far (in which case the cell assembly stops firing and the VWM representation is lost). This limits the number of cell assemblies that can be simultaneously active without either interfering with each other or decaying into oblivion.

A recurrent feedback loop can easily maintain a single item, but it is more difficult to keep representations of multiple different items from collapsing into a single representation. To solve this problem, all the neurons that represent a given item are linked together in a synchronously firing cell assembly, and only one cell assembly to fire at a given moment in time (see Figure 5b). Synchrony within a cell assembly helps maintain the recurrent activation, and asynchrony between different cell assemblies avoids interference between the representations of different items. A synchronous cell assembly effectively serves as a “slot” in VWM. The cell assemblies are formed dynamically, combining whatever set of neurons is necessary to represent a given object.

In these models, a cell assembly passively decays after each time it fires, and the representation will be lost if too much time passes before it fires again. Consequently, the number of items that can be maintained is limited by the need to keep multiple cell assemblies from firing at the same time, while also preventing long delays between successive firings of a given cell assembly so that it does not decay too far [70, 71]. Realistic biophysical parameters lead to an average capacity of 3–4 discrete objects, with some stochastic variation in the number of items stored on each trial [71]. This model can also explain the fact that multidimensional objects can be remembered as easily as single-dimension objects [8], because the neurons coding different dimensions can be synchronized into a single cell assembly [71, 72]. In general, models of this nature can explain how slot-like behavior can arise from the dynamics of a continuous neural network [56].

It is difficult to test these models from neural recordings because of the difficulty of recording from dozens of individual neurons at the same time and determining how they are linked together. Nonetheless, electrophysiological recordings from both humans and nonhuman primates have provided evidence that synchronized spikes and gamma-band oscillations represent the individual cell assemblies [73], which are then sequenced by means of coupling with theta-band oscillations [74–77]. Alpha-band oscillations may also play an important role. For example, asymmetric modulations of alpha amplitude may contribute to sustained slow waves such as the CDA [78].

Putting it All Together

Differences in VWM capacity among healthy individuals are strongly predictive of broad cognitive abilities [13, 14], and impairments in VWM capacity in patient groups may also provide an important key to understanding their real-world cognitive impairments [13]. It is remarkable that memory for simple stimuli such as colored squares is so strongly predictive of broader measures of cognitive ability and so clearly impaired in a variety of groups. But this is quite fortunate, because VWM for simple colored squares is amenable to rigorous psychophysical measurement, neural network modeling, ERP and fMRI experiments in humans, and invasive measures of neural activity in animals. Consequently, we are rapidly gaining a detailed, mechanistic understanding of the factors that determine VWM capacity, and this may in turn lead to major advances in understanding individual and group differences in broader cognitive function.

Highlights.

• Visual working memory capacity can be assessed reliably with very simple tasks

• Capacity for simple visual features is highly correlated with cognitive ability

• Capacity varies substantially across individuals and groups

• Researchers debate whether capacity reflects discrete slots or a flexible resource

• Capacity limits can be understood in terms of recurrent neural networks

Glossary and Abbreviations

Contralateral delay activity

A sustained ERP response during the delay period of a visual working memory task that is observed over the hemisphere contralateral to the items being maintained in memory

Cell assembly

A set of neurons that together represent a single item (e.g., a set of red-selective neurons and a set of vertical-selective neurons that together represent a red vertical bar)

Complex span tasks

Tasks that assess working memory capacity by requiring subjects to switch back and forth between a memory encoding task and a processing task. In the operation span task, for example, subjects see a simple math problem followed by a word that is to be stored in memory. On each trial, a sequence of these pairs is presented, and subjects are asked to recall the words at the end of the trial. Memory span is quantified in terms of the number of words that can be recalled at the end of the trial

Event-related potential

ERPs are specific event-related brain responses that are embedded within the electroencephalogram (EEG). They arise from the summed postsynaptic potentials of many thousands of neurons and are conducted through the brain and skull to the scalp, where they can be recorded noninvasively via surface electrodes

Functional magnetic resonance imaging

This technique takes advantage of the different magnetic properties of oxygenated and deoxygenated hemoglobin to localize changes in blood flow that are triggered by changes in neural activity

K

The number of items stored in working memory on a given trial type

K_max

The maximum number of items that a given individual can store in working memory. This is a measure of working memory capacity

P_mem

The probability that a given item is present in memory at the time of test

Standard Deviation

A measure of the spread of a distribution of values. In VWM research, standard deviation is used to quantify the distribution of memory errors, and it is inversely related to the precision of the memory representation

Set size

The number of items in an array

Single-unit recordings

Recordings of the action potentials of individual neurons via the tip of an electrode placed just outside the cell body of a neuron

Visual working memory

The active maintenance of visual information to serve the needs of ongoing tasks