Accounting for stimulus-specific variation in precision reveals a discrete capacity limit in visual working memory

Abstract

If we view a visual scene that contains many objects, then momentarily close our eyes, some details persist while others seem to fade. Discrete models of visual working memory (VWM) assume that only a few items can be actively maintained in memory, beyond which pure guessing will emerge. Alternatively, continuous resource models assume that all items in a visual scene can be stored with some precision. Distinguishing between these competing models is challenging, however, as resource models that allow for stochastically variable precision (across items and trials) can produce error distributions that resemble random guessing behavior. Here, we evaluated the hypothesis that a major source of variability in VWM performance arises from systematic variation in precision across the stimuli themselves; such stimulus-specific variability can be incorporated into both discrete-capacity and variable-precision resource models. Participants viewed multiple oriented gratings, and then reported the orientation of a cued grating from memory. When modeling the overall distribution of VWM errors, we found that the variable-precision resource model outperformed the discrete model. However, VWM errors revealed a pronounced “oblique effect”, with larger errors for oblique than cardinal orientations. After this source of variability was incorporated into both models, we found that the discrete model provided a better account of VWM errors. Our results demonstrate that variable precision across the stimulus space can lead to an unwarranted advantage for resource models that assume stochastically variable precision. When these deterministic sources are adequately modeled, human working memory performance reveals evidence of a discrete capacity limit.

Working memory serves an essential role in cognition, providing an active mental workspace for maintaining task-relevant information to support one’s immediate goals. This workspace, although flexible and adaptive, is severely limited in its capacity (Baddeley, 2003; Luck & Vogel, 1997; Miller, 1956). In recent years, major advances have been made in understanding the cognitive and neural bases of visual working memory (Ester, Sprague, & Serences, 2015; Harrison & Tong, 2009; Luck & Vogel, 2013; Ma, Husain, & Bays, 2014; Pratte & Tong, 2014; Serences, Ester, Vogel, & Awh, 2009; Sreenivasan, Curtis, & D’Esposito, 2014). Recent attempts to characterize the precision of VWM across varying memory loads has led to the development of mathematical models that aim to capture the cognitive architecture of visual working memory.

Prominent models of VWM broadly fall into two distinct classes: discrete capacity models and continuous resource models. Discrete capacity models posit that only a few items can be concurrently maintained in VWM with high precision (Cowan, 2001; Cowan & Rouder, 2009; Luck & Vogel, 1997; Rouder et al., 2008; Thiele, Pratte, & Rouder, 2011; Zhang & Luck, 2008) such that increases in set size beyond a person’s capacity limit will lead to more frequent responses driven by pure guessing. In contrast, continuous resource models assume that VWM has no discrete limit. Instead, performance is determined by a limited central resource that is allocated to all items in the display (Bays & Husain, 2008; Fougnie, Suchow, & Alvarez, 2012; van den Berg, Shin, Chou, George, & Ma, 2012; Wilken & Ma, 2004). As set size increases, there is less resource to go around, resulting in diminished performance for each individual item but no guessing behavior.

To investigate the relationship between accuracy and working memory load, Zhang and Luck (2008) presented 1, 2, 3 or 6 colored squares to participants, and cued participants to report the color of a probed item after a brief delay by clicking on a color wheel. This method of continuous report revealed that participants were highly accurate at small set sizes, but sometimes made gross errors at large set sizes, such as reporting that a green stimulus was red. They found that the distribution of errors in responses was well described by a mixture model consisting of accurate responses centered at the studied color, and pure-guessing responses following a uniform distribution for items that were presumably forgotten. These findings were initially taken as strong evidence in favor of the discrete capacity model, as standard resource models cannot readily account for high proportions of guess-like responses.

Although the continuous resource model was thought to be incompatible with the large memory errors observed in continuous report data, van den Berg et al. (2012) developed a new version of this model that accounted very well for the observed distribution of VWM errors (see also Fougnie et al., 2012). This model assumes that all items in a visual display are assigned some amount of central resource, but that this amount varies randomly across items and trials. Consequently, some items may happen to receive a lot of resource, leading to high memory precision, whereas other items may receive very little resource, which can lead to large errors that resemble random guessing behavior. In recent studies, this augmented version of the continuous resource model, referred to as the variable precision (VP) model, has either been shown to provide a statistically superior fit over the discrete-capacity mixture model (Keshvari, van den Berg, & Ma, 2013; van den Berg, Awh, & Ma, 2014; van den Berg et al., 2012) or a statistically indistinguishable fit (van den Berg et al., 2014). Such findings have led some researchers to question the longstanding view that visual working memory has a discrete capacity limit (see Ma et al., 2014), a concept that has been central to most major theories of working memory (Baddeley, 2003; Cowan, 2001; Luck & Vogel, 1997, 2013; Miller, 1956).

In the present study, our goal was to provide a rigorous evaluation and comparison of the discrete-capacity mixture model and the variable-precision resource model. In particular, we wanted to understand what components of each model might confer an advantage in its ability to fit participants’ VWM errors. The variable-precision resource model is founded on the central assumption that working memory has no item limit, but its superior ability to fit VWM errors could arise either from this lack of a discrete item limit, or from other attributes of the model.

A key feature of the VP model is that variability arises from random variation in precision across items and trials. Although the average precision declines with set size in a predictable manner, each item in a given display is presumed to receive a stochastically allotted amount of resource, resulting in variable precision. At large set sizes, the VP model yields predicted error distributions that resemble a high proportion of random-guessing responses due to the fact that items are frequently stored with low but non-zero precision. Note that the VP model does not directly specify the precision with which an individual item is stored, as it is fitted to the overall distribution of errors. The model also does not specify the potential sources of this variability in precision, although it is commonly described as reflecting variability in how a continuous pool of memory resources is allocated to particular items within a trial (Fougnie et al., 2012; van den Berg et al., 2012). However, this variable precision mechanism can account for many sources of trial-to-trial variability in precision, regardless of their source, including those that arise from deterministic rather than stochastic processes. By contrast, the standard discrete capacity model lacks any such mechanism to account for variable precision, as it assumes that the precision provided by each slot is the same for all items and trials.

We hypothesized that a major source of variability in VWM performance arises from systematic variation in precision across the stimulus space, effects that can be readily incorporated into both discrete capacity and continuous resource models. We investigated working memory for orientation to test this hypothesis, as considerable research has demonstrated systematic variability in the precision of orientation processing. In particular, there is both psychophysical and neural evidence that horizontal and vertical orientations are more accurately encoded by the early visual system than oblique orientations (Appelle, 1972; Furmanski & Engel, 2000; Girshick, Landy, & Simoncelli, 2011; Li, Peterson, & Freeman, 2003; Shen, Tao, Zhang, Smith, & Chino, 2014). This so-called “oblique effect” leads to superior perceptual discrimination of cardinal orientations over oblique orientations, which could confer a similar advantage in VWM tasks.

In this study, participants were briefly presented with arrays of 1, 2, 3 or 6 oriented gratings, and after a delay period, reported the orientation of a randomly cued grating from memory by rotating a central probe (Figure 1). Each participant performed multiple test sessions to obtain sufficient data for characterization of the precision of working memory performance across the full range of orientation space. We observed a prominent oblique effect at all set sizes, as evidenced by the fact that VWM responses were more tightly clustered around the true orientation when the tested grating was near vertical or horizontal (Figures 2A–C, black lines) than when it was near 45° or 135° (Figures 2A–C, red lines). This deterministic source of variable precision can be anticipated equally well by discrete capacity or continuous resource theories, for example, as resulting from early perceptual processes. However, the VP model includes a mechanism that can account for this variability (Figures 2D–F), whereas the standard discrete capacity model has no such mechanism. Consequently, this stimulus-driven variation may confer an undue advantage to the variable-precision resource model.

Figure 1.

Experimental design and stimuli. Example of a working memory trial with a set size of 3 items. Participants rotated the probe using a keyboard interface to report the orientation of the cued grating from memory.

Figure 2.

Distributions of errors for working memory responses, with participant-averaged data and model predictions for set sizes 2, 3, and 6 (left, middle and right columns, respectively). (AC) Density estimates of participant-averaged data for each set size are plotted, separately for studied orientations within ±10° of cardinal orientations (black curves) and within ±10° of oblique orientations (red curves). (D–F) Depiction of the variable precision model, showing gamma distributions of precision values (inset) for set sizes 2, 3, 6, and corresponding von Mises distributions for the 5^th, 50^th and 95^th percentiles of the gamma distributions. (G–I) Predicted density functions for the discrete-capacity model with oblique effects in precision incorporated, plotted separately for cardinal (black) and oblique (red) orientations.

We performed a simulation study to demonstrate this advantage. In Figure 3A, data were generated from either the discrete capacity model (blue) or the variable precision model (red), each with various magnitudes of oblique effects included in the model. Positive values on the y-axis indicate that the discrete capacity model provides a better account of the data than the variable precision model. Thus, blue points should have positive values, as they are generated from a discrete capacity model. Alternatively, red points should have negative values, as they are generated from the variable precision model. When oblique effects are small, the majority of blue points indeed lie above zero and red points below zero, indicating that the correct data-generating model was identified. However, as the oblique effect becomes more prominent in magnitude, statistical model comparison systematically prefers the VP model over the discrete capacity model, even when the data were generated from a discrete capacity model. Consequently, even if the discrete capacity model generated a set of working memory data, the presence of an oblique effect will make the results appear as if the VP model provides a superior account of the data.

Figure 3.

Simulation results. (A) Data were simulated from a discrete capacity model (DC, blue) or a variable precision model (VP, red) with oblique effects ranging from zero to one, encompassing typical values estimated from the experimental data (individual’s estimates denoted by vertical lines along the abscissa). The y-axis shows the difference in BIC between fitted discrete capacity and variable precision models, such that positive numbers indicate that the discrete capacity model provides a superior account of the data and negative numbers support the VP model. When oblique effects are small, the correct models are recovered as being the generating model. However, as the oblique effect grows, BIC begins to favor the variable precision model even when the data were generated from a discrete capacity model. (B) The same data were fitted with the discrete capacity and variable precision models that included oblique precision effects, with positive BIC values again supporting the discrete capacity model. Incorporating the oblique effect in precision provides for accurate recovery of the true generating model, even with large oblique effects.

We performed a series of analyses on experimental data to test whether this deterministic source of variable precision, across stimulus space, might confer an undue advantage for the variable-precision resource model. First, we fitted both the discrete-capacity model and the variable-precision model to the overall distribution of VWM errors, independent of the studied orientation. Statistical model comparison indicated that the VP model outperformed the discrete capacity model, consistent with other recent reports (van den Berg et al., 2014; van den Berg et al., 2012). However, the VP model can account for variability in precision due to the oblique effect (Figures 2D–F) whereas the discrete-capacity model cannot, so we next augmented both models to directly include these effects. We characterized the precision of VWM across orientation space for each participant (Figure 4A), and incorporated this stimulus-specific variability into both models by allowing the precision of responses to depend on the specific orientation tested on each trial. When this predictable source of variability was accounted for, we found that the discrete capacity model outperformed the variable precision model.

Figure 4.

Stimulus-specific effects of orientation, showing changes in VWM precision across orientation space (A) and bias in mean responses across orientation space (B). Data were binned (10° bin widths) by studied orientation for each participant, and a von Mises distribution was fitted to the data in each bin to obtain estimates of mean bias and precision for each bin. (A) Estimated precision for each bin was averaged over participants at each set size, revealing a substantial oblique effect. Lines show model fits to participant-averaged data, assuming a fixed oblique effect for all set sizes, allowing only mean precision to vary across set size according to Eqn. 3. (B) The circular difference between the true center for each bin and the estimated mean response in that bin reveals systematic response biases away from the cardinal axes. The line shows the bias model fitted to these averaged data, in which the magnitude and shape of the bias effect is fixed across set sizes according to Eqn. 4.

Finally, we implemented a hybrid version of the two models, by constructing a discrete capacity model that allowed for additional variability in precision beyond the oblique effect. This model significantly outperformed both the discrete capacity model and the variable-precision resource model. Taken together, these findings imply that there are multiple sources of precision variability in VWM, with stimulus-dependent variability comprising one major source. When these sources are taken into account, we find that visual working memory is best characterized as having a discrete item limit.

Methods

Participants

Twelve observers (ages 20 to 35 years) participated in the experiment, including two authors (RR and YP). All participants, except for the authors, were paid for participation. All participants reported normal or corrected-to-normal visual acuity, and provided informed consent prior to participation. This study was approved by the Institutional Review Board at Vanderbilt University.

Stimuli

Visual stimuli were displayed on a gamma-corrected CRT monitor with 1024 × 768 resolution and 75Hz refresh rate. The stimuli were generated using MATLAB and the Psychophysics toolbox (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007). The stimuli were presented against a grey background of luminance 35.8 cd/m2 at a viewing distance of 57 cm in an otherwise darkened room. Throughout the experiment, observers were instructed to fixate centrally on a black and white bull’s eye (0.5° of visual angle in diameter). Stimuli consisted of circular gratings (2° in diameter) with spatial frequency of 2 cycles/° and 50% Michelson contrast, with a wide Gaussian envelope (σ=2°). A sample array consisted of 1, 2, 3, or 6 gratings that appeared at randomly determined locations on a virtual circle at an eccentricity of 4° from the fixation point (see Figure 1), with the restriction that any two gratings were separated by a minimum of 1°. Gratings were presented at random orientations, ranging from 1° to 180° clockwise from vertical in 1° steps.

Procedure

Gratings were presented for 200ms or 2000ms. Following a memory retention interval of 3 seconds, cues appeared for 500 ms as white outlines (0.04°) at the circumference of all studied gratings. The to-be-reported item was indicated by a thicker (0.10°) outline. A probe grating (2° in diameter) was presented centrally at an initially random orientation (1°–180°), which could be subsequently rotated by small (0.5°) or large (1°) steps by using key presses. Observers confirmed their final answer by pressing the space bar, initiating the fixation display for the next trial.

Each one-hour experimental session consisted of 320 trials. Trials with different study durations (200 and 2000 ms) were presented in 8 alternating blocks of 40 trials each. In each block, different set sizes (1, 2, 3 and 6) were randomly intermixed across trials. Across 8 sessions, a total of 2560 trials were collected for each participant, providing 640 observations at each of the 4 set sizes. In all analyses, our main findings of the oblique effect and response biases were present at both exposure durations, and exhibited the same general pattern. Consequently, the data were collapsed across exposure duration conditions.

Discrete Capacity Model

According to the discrete-capacity model, the observer’s responses (denoted y) are assumed to arise from a mixture of random-guess responses that follow a uniform distribution, and in-memory responses that follow a von Mises (VM) distribution centered at the studied orientation θ with precision κ_n for the nth set size. Guess responses at the nth set size occur with probability g_n and in-memory responses occur with probability 1 – g_n.

$$P(y=Y)=\frac{g_n}{2\pi }+(1-g_n)\ast \text{VM}(\theta ,{\kappa }_n).$$

The guess rate g_n varies as a function of the set size (N), governed by a free parameter (K) that denotes the subject’s working memory capacity:

$$g_n=1-\frac{K}{N}$$ (1)

According to the slots-plus-averaging model (Zhang & Luck, 2008), memory precision will vary as a function of set size when the number of studied items is less than a subject’s capacity limit. This is based on the assumption that an item may be stored in multiple independent slots, leading to superior precision for that item due to the benefits of averaging. The number of slots (S) devoted to the studied item is a latent mixture (van den Berg et al., 2012):

$$S_k=\{\begin{array}{ll}\lfloor \frac{K}{N}\rfloor ,\hfill & \text{with}\text{probability}1-\frac{\mathit{KmodN}}{N},\hfill \\ \lfloor \frac{K}{N}\rfloor +1,\hfill & \text{with}\text{probability}\frac{\mathit{KmodN}}{N},\hfill \end{array}$$

where ⌊x⌋ denotes the floor of x. This assignment of slots results in two corresponding concentration parameters

$$\begin{array}{ll}\hfill {\kappa }_{\mathit{low}}(N)& ={\kappa }_1\lfloor \frac{K}{N}\rfloor ,\hfill \\ \hfill {\kappa }_{\mathit{high}}(N)& ={\kappa }_1\left(1+\lfloor \frac{K}{N}\rfloor \right),\hfill \end{array}$$

where κ₁ denotes the precision for items maintained by a single slot. If the number of studied items is equal to or exceeds capacity (N ≥ K), each stored item will have precision equal to κ₁. Thus, the discrete capacity model has two free parameters for each subject, capacity (K) and the associated precision for one slot (κ₁). Parameters for all models were estimated separately for each participant using maximum likelihood estimation procedures.

Variable Precision Model

In the variable precision model, responses again follow a von Mises distribution, but with a precision that is assumed to vary stochastically from item-to-item and trial-to-trial:

$$\begin{array}{ll}\hfill P(y=Y)& =\text{VM}(\theta ,{\kappa }_i),\hfill \\ \hfill {\kappa }_i& ~\text{Gamma}({\alpha }_n,\tau ).\hfill \end{array}$$

The precision (or amount of resource) assigned to any item in a display of N items is assumed to arise from a gamma distribution parameterized by its mean (α_n), which varies as a function of set size, and by a shape parameter τ that remains fixed across set sizes. Mean precision varies as a function of set size, following a power function with two free parameters:

$${\alpha }_n=\frac{\mu }{N^{\beta }}$$ (2)

where μ is the precision for a single item, and β determines the rate with which this precision declines as a function of set size N. The average estimates of μ and β in our experiment were approximately 25 and 1.3, respectively.

The variable precision model has three free parameters, the precision for a single item (μ), the rate at which this precision declines as a function of set size (β), and the shape of the gamma distribution that describes the variability in precision (τ).

Hybrid Model

We constructed a hybrid model to determine whether visual working memory might be better described by a model that has a discrete item limit but still allows for variability in the precision of VWM across items and trials beyond that caused by the oblique effect. This model followed the architecture of the discrete capacity model as presented above, but with the key difference that the precision of each slot was allowed to vary from trial to trial according to a gamma distribution. In this hybrid model, the mean of the gamma distribution varies as a function of set size according to the slots-plus-averaging framework, rather than by a power law as in the variable precision model. Thus, the theoretical foundation of the hybrid model is a discrete capacity model, as both guess rate and changes in precision across set size follow the strict predictions of the discrete model based on the individual’s estimated capacity K. Thus, assigning multiple slots to represent a specific item will lead to improved mean precision for that item, but mean precision remains constant for all set sizes above a person’s capacity.

The hybrid model has three free parameters for each participant: capacity (K), average precision for a single slot (μ), and the shape parameter for the gamma distribution on precision (τ).

Incorporating Orientation-Specific Oblique Effects

In all models under consideration, we examined two effects of orientation: 1) the oblique effect whereby precision varies as a function of the studied orientation, and 2) a bias in responses away from the cardinal axes. To model the oblique effects on precision, we used an exponentiated cosine function:

$$O(\theta \mid \lambda ,\gamma )=e^{\lambda +\gamma \ast cos(2\theta )},$$ (3)

where λ is the mean precision for all orientations θ, and γ is the size of the oblique effect on precision. Exponentiation of the cosine ensures that precision values are always above zero, and produces a multiplicative effect between mean precision and orientation. This oblique effect in mean precision is incorporated into the discrete capacity model by allowing the precision for a single slot (κ₁) to vary as a function of studied orientation according to equation 3. The oblique effect was incorporated within the VP model and the hybrid model by allowing the mean precision parameter (μ) to vary according to equation 3.

In addition to changes in precision across orientation space, previous work has shown that observers sometimes exhibit consistent biases toward or away from cardinal orientations in visual perception tasks (Andrews, 1965; Girshick et al., 2011; van Bergen, Ma, Pratte, & Jehee, 2015; Wei & Stocker, 2015). To ensure that such biases did not influence our modeling of the oblique effects in precision, we incorporated a bias term and performed model comparison both with and without this bias parameter. We characterized these bias effects by allowing the response distributions to be shifted away from (or toward) nearby cardinal orientations, relative to the original studied orientation, following a cosine function

$$B(\theta \mid \eta )=\eta cos(2\theta -\pi /2),$$ (4)

where the free parameter η reflects the magnitude and direction of the bias. This shift was added to the center of the von Mises response distributions for all models with the bias term.

We found that these parameterizations of the oblique effect in precision and the bias provided reasonable fits to participants’ VWM data (denoted by lines in Figure 4). In order to accurately estimate the oblique effect, we first estimated how precision varies with orientation (γ) and the magnitude of bias (η) for each subject at set size 1, and then used these parameter estimates when fitting the full working memory models to set sizes 2, 3 and 6. This approach capitalized on the highly reliable precision estimates observed at set size 1, and minimized the potential impact of non-uniform guessing distributions which could distort estimates of precision and bias across orientation space at large set sizes.

Model Estimation

All models were fit to the data from each participant by maximizing the likelihood function using standard gradient descent optimization. This approach was made possible for the variable precision and hybrid models by noting that the marginal distribution of a von Mises distribution with gamma distributed precision follows a wrapped Student’s t distribution (Fougnie et al., 2012). Although the density function for the wrapped t distribution involves an infinite sum, appropriately approximating this sum (Pewsey, Lewis, & Jones, 2007) provides a deterministic likelihood such that a reliable maximum may be obtained.

To incorporate oblique effects into the models, the data from set size 1 were first used to estimate the magnitude of oblique effects in precision alone for each participant. At set size 1 the discrete capacity model reduces to a von Mises distribution (assuming K>1). Likewise, β drops out of the VP model at set size 1. In addition, for typical estimates of τ such as we observed here (mean = 4.4), and the high precision values at set size 1 (see Figure 4A, in blue), the wrapped t distribution approaches and is accurately approximated by a von Mises distribution. Consequently, we measured the size of each participant’s oblique effect by fitting a von Mises distribution to set size 1, with a mean precision and oblique effect in precision following equation 3. For models with oblique effects in both precision and bias, we included an additional parameter for the bias effect by modifying the mean of the von Mises (Eqn. 4, Figure 4B), conditioned on the tested orientation. The estimated oblique effects for each subject were then used when fitting the working memory models to all other set size data (excluding set size 1).

Simulation Studies

A series of simulation studies were conducted to 1) investigate how failing to model oblique effects can produce distorted conclusions, and 2) to examine whether explicitly incorporating these effects within both models produces accurate conclusions. In Figure 3A, data were simulated from the discrete capacity model (blue points) or the variable precision model (red points). Oblique effects on precision were incorporated within each simulation according to Equation 3 (oblique effects in bias were excluded from the simulations for simplicity), with effect magnitudes ranging from no oblique effects (λ=0) to large effects (λ=1). The size of these oblique effects covered the span of typical effects that were estimated from the experimental data (the estimated magnitudes of oblique effects for individual subjects are denoted by black vertical bars along the abscissa). Parameters for each simulation were the parameters estimated from fitting each model to the data aggregated over participants. Simulations were modeled after our experimental design for each participant, with each including 640 trials from set sizes 1, 2, 3 and 6.

For each simulation, the standard discrete capacity and variable precision models without oblique effects were fitted to the simulated data from set sizes 2, 3 and 6. The relative ability for each model to account for the data was measured with the BIC statistic (Schwarz, 1978); positive values in Figure 3A indicate that the discrete capacity model was superior, negative values indicate that the variable precision model was superior. When oblique effects are small, the true data-generating models are accurately identified (blue points are above zero, red points are below zero). However, as oblique effects grow in size (x-axis), the BIC statistics begin to favor the variable precision model, even when the discrete capacity model generated the data (indicated by the blue points below zero).

In Figure 3B the same data were fitted with discrete capacity and variable precision models that explicitly accounted for oblique effects in precision. The procedure for doing so was as described above, in which oblique effects were estimated at set size 1, and incorporated into the models for fitting data from set sizes 2, 3 and 6. With oblique effects properly accounted for in both models, model comparison statistics now identify the correct model, regardless of oblique effect magnitude. The size of the VP win in BIC does appear to get smaller at very large oblique effects. We suspect this reflects the fact that when oblique effects are very large, responses to oblique orientations at high set sizes will be from such low precision that they will resemble guessing, whereas responses to cardinals will be somewhat accurate. This pattern may give the appearance of a mixture of guessing and in-memory responses, providing some evidence for a discrete capacity limit. Critically, however, even at very large oblique effects the models are accurately recovered in the majority of simulations, such that even if the presence of large oblique effects in the data make them appear to exhibit near-guessing on some trials, the discrete capacity and variable precision models with oblique effects can still be accurately identified as having generated the data.

Results

Behavioral Performance

The accuracy of participants’ working memory reports revealed a prominent oblique effect across all set sizes in this study. The precision of VWM was higher for cardinal than oblique orientations, as can be seen in the scatterplots shown in Figure 5, based on the tighter clustering of reported responses around orientations 0° (vertical) and 90° (horizontal). To examine how performance varied according to stimulus orientation, we binned the data according to the orientation tested on each trial (10° bin widths) and calculated the precision and mean bias of VWM responses for each bin. The results of this analysis revealed strong oblique effects in terms of VWM precision across all set sizes (Figure 4A), as well as a modest effect of response bias away from cardinal orientations (Figure 4B).

Figure 5.

Scatterplots showing reported orientations plotted as a function of the studied orientation for each set size, collapsed across participants. Responses tended to fall along the identity line. However, the bowtie shape of the deviations from the identity line indicates a substantial oblique effect. At set size 6, responses appear more scattered throughout the space of all possible orientation responses.

Comparison of Discrete-Capacity and Variable-Precision Resource Models

We performed statistical model comparison to determine how well each model could account for participants’ patterns of VWM errors across changes in set size, and critically, across changes in orientation. Models were compared using the Bayesian information criterion (BIC) statistic (Schwarz, 1978), which reflects the log likelihood of the observed data given the model and its estimated parameters, plus a penalty term to account for model complexity (a function of the number of parameters). Thus, lower BIC values indicate better model performance. The difference in BIC scores between two models provides a metric of the better model, with a difference of zero indicating equivalent performance and differences greater than 10.0 typically taken as strong evidence in favor of the lower scoring model. We present BIC results in two ways. First, we aggregated the results across participants by summing likelihoods and computing a total BIC score for each model. These aggregated BIC scores are shown in Table 1. Second, we calculated BIC values separately for each participant, and determined which of the 9 tested models provided the best fit for each participant. The numbers of participants for which each model provided the best fit are shown in parentheses in Table 1. Similar results were obtained using other model comparison statistics such as Akaike information criterion (AIC) (Akaike, 1974).

Table 1.

	Discrete Capacity	Variable Precision	BIC Difference	Hybrid
Original Model	1181 (0)	813 (0)	368	733 (2)
With Oblique Precision Effect	510 (1)	616 (0)	−106	150 (3)
With Precision And Bias Effects	366 (2)	571 (0)	−205	0 (4)

Models Without Oblique Effects

We first applied each model to the overall distribution of VWM errors across changes in set size, without consideration of the specific stimulus tested on each trial, following the approach of other recent studies that have compared the discrete capacity and variable precision models (van den Berg et al., 2014; van den Berg et al., 2012). Table 1 provides BIC values for all models tested in this study, reported as difference scores relative the to best fitting model (hybrid model with oblique precision effect plus bias, as is described below). The top row of Table 1 indicates the performance of the original versions of the discrete capacity and variable precision models, as well as the difference in performance between the two models. As can be seen, the results of this analysis replicate previous reports: The variable precision model provided a better account of the data than the discrete capacity model (ΔBIC=368). Likewise, when BIC values for these models are computed and compared for each participant, the VP model provided a better fit than the discrete capacity model for 8 out of 12 participants.

Models With Oblique Effects

For this analysis, we determined how well each model could account for participants’ VWM errors, taking into consideration not only set size but also the specific orientation tested on each trial. To do so, we incorporated oblique effects into each model prior to performing statistical model comparison. Accounting for these oblique effects led to marked improvements in fit for both models. BIC scores were considerably lower for the discrete capacity model with oblique effects in precision, in comparison to the original version of this model (ΔBIC=671). The variable precision model with oblique effects in precision also performed better than the original version of this model (ΔBIC=197), indicating that there was additional variance to be absorbed by explicit modeling of these stimulus-specific variations in precision. However, as we anticipated, specifying this deterministic source of variable precision a priori proved to be less beneficial for the VP model than for the discrete capacity model. This is because the original VP model allows for latent variability in precision, and therefore it already has the ability to account for some of the variance arising from predictable sources such as the oblique effect (Figures 2D–F). When this stimulus-specific source of precision variability is explicitly specified in both models, our results indicate that the discrete capacity model now outperforms the VP model (ΔBIC=−106). We inspected how well these models fitted the data of individual participants, and found that the discrete capacity model with oblique effects provides a better account than the VP model with oblique effects for the majority of participants (7 out of 12). The predictions of this discrete capacity model are shown in Figure 2G–I separately for cardinal (black lines) and oblique orientations (red lines), and these predictions align well with the observed data (Figures 2A–C).

In addition to oblique effects in precision, we considered models that included orientation-specific effects in bias, as some participants showed small but consistent deviations in their responses away from the true orientation, typically with shifts away from cardinal orientations (Figure 4B). Similar effects of bias have been reported in studies of visual perception (Andrews, 1965; Girshick et al., 2011; Wei & Stocker, 2015). For these analyses, the mean of the von Mises distribution was allowed to shift away from the true orientation, by incorporating a mean bias component across orientation space. The inclusion of this bias effect, in addition to the oblique effect on precision, led to a further improvement in the fits for both the discrete capacity model (ΔBIC=144) and variable precision model (ΔBIC=45). With both of these sources of stimulus-specific variability now incorporated into both models, model comparison indicated that the discrete capacity model with precision plus bias effects provides a much better account of VWM responses than does the variable precision model with these effects (ΔBIC=−205). Moreover, the discrete capacity model now provides a superior fit for 8 of the 12 participants, in contrast to the advantage observed for the standard version of the VP model over the standard discrete capacity model.

Hybrid Model

The above results support the presence of an item limit for working memory: When stimulus-specific variation in VWM precision is taken into account, the discrete capacity model provides a better account of VWM responses than the variable-precision resource model. However, it remains an open question as to whether additional sources of variable precision might exist for a discrete model with item limit, over and above the stimulus-specific variation in precision that we account for by modeling the oblique effect.

To address this question, we constructed a hybrid model that included both a discrete capacity limit as well as variable precision. Specifically, the hybrid model followed the core architecture of the discrete capacity model, while allowing the precision of each slot to vary from trial-to-trial according to a gamma distribution. The mean of this gamma distribution varied as a function of set size, according to the slots-plus-averaging framework, and set sizes exceeding capacity were modeled by predicted changes in guess rate. If sources of variability other than the oblique effect are present, then this hybrid model should outperform the discrete capacity model with oblique effects included.

We found that the hybrid model, with discrete capacity limit and latent variable precision, outperformed both the standard discrete capacity model (ΔBIC=448) and the standard variable precision model (ΔBIC=80). When the oblique effects of precision and bias were incorporated within all models, the hybrid model showed a clear advantage over both the discrete capacity model (ΔBIC=366) and the variable precision model (ΔBIC=571) with these effects included.

When we consider all 9 models and their ability to account for the performance of individual participants (parentheses in Table 1 indicate the number of participants for which each model provided the overall best fit), several notable patterns emerge. First, the best-fitting model for all of the 12 participants had as part of its central architecture a discrete capacity limit (either the discrete capacity or hybrid models). Second, the majority of best-fitting models included the oblique effect on precision, or both an oblique effect on precision and an orientation bias effect (10 out of 12 participants). Finally, some form of the hybrid model provided the best account for 9 out of 12 participants. Taken together, the conclusions drawn from these results are that: 1) VWM is best characterized by a discrete capacity limit, 2) a major source of variable precision arises from stimulus-specific differences in VWM precision such as the oblique effect, and 3) there are additional sources of variable precision that are not accounted for by our model of the oblique effect.

Discussion

Discrete capacity and continuous resource models have provided dichotomous accounts of whether there is any limit to the number of items that can be actively maintained in working memory. Despite such divergent starting assumptions, these models yield very similar predicted patterns of performance on VWM tasks, presenting a challenge for discerning their respective qualities.

Here, we found that the variable precision model does outperform the discrete slots-plus-averaging model, when applied to the distribution of VWM errors independent of the stimulus tested. However, the superior performance of the variable precision model did not arise from its central assumption that VWM has no item limit. Instead, the variable precision model contains a mechanism that can account for multiple sources of variability, regardless of whether they arise from stochastic or deterministic sources. Our behavioral results revealed that working memory performance was consistently more precise for cardinal orientations than for oblique orientations, indicating that stimulus-specific factors can strongly modulate VWM precision. Once this source of systematic variability was incorporated into both models, we found that the discrete capacity model provided a better account of the patterns of VWM errors, and outperformed the variable-precision resource model. These findings indicate the importance of considering deterministic sources of variability when modeling VWM performance, otherwise resource models with stochastically variable precision may benefit from an unwarranted advantage.

Our implementation of a hybrid version of the two models provides further evidence to support a discrete capacity limit. Recent work has suggested that such a hybrid model with discrete item limit may perform as well as or better than a variable precision model with no item limit (Donkin, Kary, Tahir, & Taylor, 2016; Nosofsky & Donkin, 2016; van den Berg et al., 2014). Here, when oblique effects were taken into account in all models, the hybrid model clearly outperformed both the discrete capacity model and the variable-precision resource model (see Table 1). These results suggest the presence of other sources of variable precision in our study, in addition to those arising from the oblique effect. Critically, however, our hybrid model with a discrete item limit outperformed the variable precision models that have no discrete upper limit. Our results therefore fail to support the notion that all items in a visual display can be maintained with some degree of precision. Instead, we find that gross errors often occur at large set sizes that exceed presumed capacity, with error distributions better described by an increasing predominance of random guessing behavior. These findings provide compelling new evidence in favor of discrete capacity models of visual working memory.

We focused here on visual working memory for orientation, as variation in precision across orientation space has been extensively studied and is well understood (Appelle, 1972; Furmanski & Engel, 2000; Girshick et al., 2011; van Bergen et al., 2015). The prevalence and consistency of the oblique effect across people allowed us to incorporate this effect within complex models of working memory in a principled manner, allowing for a rigorous comparison of the discrete capacity and variable precision models. However, there are almost certainly other sources of item-level and trial-to-trial variability (Rademaker, Tredway, & Tong, 2012). For our orientation experiment, the superiority of the hybrid model, with oblique effects included, suggests that there are other sources of trial-to-trial variability beyond just the oblique effect. There could be other stimulus-dependent effects arising from interactions among stimuli, such as ensemble coding effects (Brady & Alvarez, 2015a, 2015b; Johnson, Spencer, Luck, & Schoner, 2009), perceptual grouping effects (Woodman, Vecera, & Luck, 2003; Xu & Chun, 2007) and effects of visual crowding (Tamber-Rosenau, Fintzi, & Marois, 2015). For example, if some elements in a particular visual display can be perceptually grouped or “chunked” as a single unit for storage, this could alter the effective number of items that can be maintained in working memory, which in turn would lead to variability in VWM performance across displays containing the same physical number of items. There is also evidence of item-specific variability in VWM performance for other stimulus materials such as color (Bae, Olkkonen, Allred, Wilson, & Flombaum, 2014; Brady & Alvarez, 2015a; Morey, 2011), which could arise from low-level differences in perceptual sensitivity across color space, as well as from the categorical nature of color processing and naming (Bae, Olkkonen, Allred, & Flombaum, 2015; Hardman, in press). Finally, the allocation of attention across a visual display could lead to additional sources of variability. For example, participants might be predisposed to attend to a particular item in a multi-element display if it appears far from the other items, leading to uneven distribution of attentional resources within a trial. Likewise, temporal fluctuations of attentional vigilance could lead to variable performance across trials (Adam, Mance, Fukuda, & Vogel, 2015), even for displays containing identical visual items.

The goal in statistical model comparison is to reward models for fitting the data well and to penalize models for their complexity, as more complex models will always provide a better fit than less complex models. We performed model comparison using the BIC statistic which, like AIC (Akaike, 1974), penalizes models for their complexity based on the number of free parameters in the model. Whereas the discrete capacity model has two free parameters (capacity and precision), the variable precision model has three free parameters (mean precision, variability in precision, rate of set size decline), and this additional complexity must be taken into account when comparing the models. Both AIC and BIC are theoretically well motivated, and simply counting parameters provides a convenient heuristic to quantify model complexity. In some cases, however, this coarse measure may not capture the true difference in flexibility across models. For example, when data were simulated from a discrete capacity model without oblique effects, the fitted discrete capacity model produced BIC scores that were superior by about 35 points. However, when the data were generated from the variable precision model, the fitted variable precision model produced BIC scores that were superior by about 95 points (see Figure 3). This asymmetry was equivalent if AIC was used, and indicates a bias in model selection in favor of the variable precision model. Presumably, the VP model has greater flexibility and complexity than the discrete capacity model, and the BIC statistic does not sufficiently penalize the VP model for this added flexibility. The particular choice of model comparison statistic employed here is not so critical, as we demonstrate that incorporating oblique effects leads to a complete reversal in the conclusion. However, for more critical applications such as when the discrete capacity and variable precision models are simply compared in their ability to account for a data set, our simulations suggest that a more careful approach for capturing model complexity is warranted (e.g., Wagenmakers, Ratcliff, Gomez, & Iverson, 2004)

Constructing and comparing mathematical models provides for a powerful way to study the cognitive structures that underlie visual working memory. However, it is critical that any statistical advantage of one model over another reflects a meaningful theoretical advantage. To date, studies that have sought to compare models of working memory have neglected the potential contributions that stimulus-specific processing can play in such model comparisons. Here, we show that stimulus-specific variations in memory precision can be substantial, and can distort the conclusions drawn from model comparison when they are not taken into account. Looking forward, we believe that explicit characterization and modeling of these multiple sources of variability will provide for a more accurate picture of the underlying functional architecture of visual working memory.

Statement of Public Significance.

Visual working memory refers to the ability to remember visual information over periods of seconds. This memory system is flexible and adaptive, but severely limited in terms of the amount of information it can maintain. A major goal in psychology is to understand the nature of this limit: How many items can be stored, and how precisely can each item be stored? In this work, we develop new mathematical modeling techniques for characterizing the precision of memory for different visual stimuli, and find that some stimuli are remembered with much higher accuracy than others. By modeling and accounting for this large source of variability in memory performance, we obtain new evidence to support the theory that visual information is lost from working memory in a discrete, all-or-none manner, such that information about previously viewed stimuli can vanish entirely from memory within moments of viewing.