Optimal Allocation of Finite Sampling Capacity in Accumulator Models of Multialternative Decision Making

Abstract

When facing many options, we narrow down our focus to very few of them. Although behaviors like this can be a sign of heuristics, they can actually be optimal under limited cognitive resources. Here, we study the problem of how to optimally allocate limited sampling time to multiple options, modeled as accumulators of noisy evidence, to determine the most profitable one. We show that the effective sampling capacity of an agent increases with both available time and the discriminability of the options, and optimal policies undergo a sharp transition as a function of it. For small capacity, it is best to allocate time evenly to exactly five options and to ignore all the others, regardless of the prior distribution of rewards. For large capacities, the optimal number of sampled accumulators grows sublinearly, closely following a power law as a function of capacity for a wide variety of priors. We find that allocating equal times to the sampled accumulators is better than using uneven time allocations. Our work highlights that multialternative decisions are endowed with breadth–depth tradeoffs, demonstrates how their optimal solutions depend on the amount of limited resources and the variability of the environment, and shows that narrowing down to a handful of options is always optimal for small capacities.

1. Introduction

The problem of allocating finite resources to determine the best of several options is common in decision making, from deciding which vaccine candidates to fund for further research to choosing a movie for Saturday night. In these cases, planning, and thus resource allocation, needs to be made in advance, well before feedback about the success of the choice is observed. Consequently, two important questions arise: How many options should we examine? And, for how long? When resources are limited, such as number of participants who can be tested with vaccines in a short time, or weekend free time in the previous examples, a decision maker should balance breadth, how many options to sample, and depth, how much to sample each. This ubiquitous decision‐making problem under constrained resources is what has been called the breadth–depth dilemma (Moreno‐Bote, Ramírez‐Ruiz, Drugowitsch, & Hayden, 2020; Horowitz & Sahni, 1978; Miller, 1981).

In the face of many alternatives, humans quickly narrow down the number of considered options to around two to five (Beach, 1993; Hauser & Wernerfelt, 1990; Levin, Jasper, & Forbes, 1998; Olshavsky, 1979; Payne, 1976), and, when presented with more than six options, experienced overload produces suboptimal choices in certain conditions (Iyengar & Lepper, 2000; Scheibehenne, Greifeneder, & Todd, 2010). Models describe this behavior by assuming that considering more options incurs search or mental costs (Hauser & Wernerfelt, 1990; Mehta, Rajiv, & Srinivasan, 2003; Stigler, 1961), but why people consider small sets in a wide range of environments is still a matter of debate. While this could be explained by strict small capacity limits in attention or working memory (Cowan et al., 2005; Miller, 1956), the nature of this small capacity would still need to be addressed (Ma, Husain, & Bays, 2014). Another possibility is that capacity is not necessarily small, but rather that sampling few options and ignoring the vast majority, in either an automatic or in a conscious manner, is actually an optimal policy that favors depth over breadth (Moreno‐Bote et al., 2020). This possibility is supported by the fact that neuronal resources devoted to decision making are not precisely low, as dozens of brain areas and several billions of neurons are involved in even simple decision‐making tasks (Rushworth, Noonan, Boorman, Walton, & Behrens, 2011; Siegel, Buschman, & Miller, 2015; Vickery, Chun, & Lee, 2011; Yoo & Hayden, 2018). Thus, processing bottlenecks could be reflections of close‐to‐optimal policies to breadth–depth dilemmas.

Bounded rationality accounts (Gershman, Horvitz, & Tenenbaum, 2015; Griffiths, Lieder, & Goodman, 2015; Russell & Wefald, 1991; Simon, 1972) surmise that many features of cognition arise from the finite limits of the nervous system. This must also be the case for the nature of the policies chosen by people in decision making, but oftentimes, the constraints imposed by the limited resources are not made explicit. Indeed, choices stemming from sequential sampling between two or three options have been typically modeled as optimal stopping problems (Callaway, Rangel, & Griffiths, 2021; Drugowitsch, Moreno‐Bote, Churchland, Shadlen, & Pouget, 2012; Gold & Shadlen, 2007; Jang, Sharma, & Drugowitsch, 2021; Krajbich & Rangel, 2011; Ratcliff & Murdock, 1976; Sepulveda et al., 2020; Tajima, Drugowitsch, Patel, & Pouget, 2019; Vul, Goodman, Griffiths, & Tenenbaum, 2014), where agents should optimally balance the prospect of learning the value of the options with the costs of sampling them, but they do so without computational or capacity constraints. In these works, the objective is to maximize accumulated reward, typically by introducing a sampling cost. Therefore, by fixating largely on the sequential nature of the tasks, these studies focus only on a particular efficiency–performance tradeoff known as the speed–accuracy tradeoff (Del Giudice & Crespi, 2018). In many complex decisions, however, there are several other functional tradeoffs that involve other properties of the agent–environment loop, such as limited sampling resources, limited interactions with the environment, and delayed feedback (Moreno‐Bote et al., 2020). The effect of these resource limitations on decision making might not be important when there are only two or three available options, but it might be critical when going beyond those low numbers. In that case, the allocation of resources might be governed by two‐stage processes (Hauser & Wernerfelt, 1990; Mehta et al., 2003; Roberts & Lattin, 1991; Shocker, Ben‐Akiva, Boccara, & Nedungadi, 1991), instead of purely sequential processes, where the first decision is about the subset of options that will be considered for further processing.

Here, we study whether narrowing attention to a few options results from optimally allocating finite resources in multialternative choices. To this end, we consider an infinitely divisible sampling resource (e.g., time or precision), such that there are no bounds in the number of alternatives that can be considered. In our model, an agent can first allocate finite sampling time over an arbitrarily large number of options, modeled as accumulators of noisy evidence, with the only restriction that the total sampling time is fixed. This is in stark contrast with previous work on the breadth–depth tradeoff, where the sampling process was simplified, and where the sampling outcomes and resources were discrete, thus obtaining qualitatively different predictions (Moreno‐Bote et al., 2020). This accumulation of evidence runs in parallel and independently for each accumulator, and only their final states are observed. Based on the observations, the agent picks up the one with the highest expected rate of evidence accumulation, which defines the utility of the choice. The goal of the agent is to optimize the allocation of sampling time such that expected utility is maximized. We identify a critical variable in the problem, that we simply call capacity, that increases with the actual size of the resources of the agent as well as with the discriminability between options, and we find that this capacity separates two distinct regimes of optimal allocation. When sampling capacity is small, the optimal policy is to sample exactly five options, regardless of the prior. In contrast, when capacity is large, the number of options to sample grows with capacity in a sublinear fashion that depends on the prior. We find a duality between allocated time and allocated precision to the options, such that all our results generalize to allocating precision while keeping fixed sampling time. Finally, we show that even allocations are optimal, and thus better than more complex asymmetric time allocations over the considered options. Overall, our results suggest that decisional bottlenecks can be a byproduct of optimal policies in the face of uncertainty.

2. Multiaccumulator model

We consider an environment that generates many options ($N\gg 1$) from which to choose (Fig. 1, top), each one characterized by a ‘drift’ parameter ${\mu }_i$ ($i=1,\text{…},N$), unknown to the agent. All drifts ${\mu }_i$ are drawn identically and independently from a prior probability distribution $p_{\theta }(\mu )$, known to the agent and assumed to have finite mean and variance. In order to choose between the options, the agent gathers information by sampling them. The critical aspect of our model is that sampling times $t_i\ge 0$ (Fig. 1, bottom) need to be allocated before the actual sampling occurs, and with the constraint that the total sampling time $T$ is limited,

$$\sum _{i=1}^N{t}_i=T.$$ (1)

In practice, the agent needs to decide on the number of options $M\le N$ to be sampled and their corresponding sampling times $t_i>0$ for $i\le M$, while the remaining options $i>M$ are ignored by giving them no sampling time, $t_i=0$ (Fig. 1, bottom). The ordering of the options is irrelevant, as they are initially indistinguishable, and thus we take the first $M$ as those that are sampled. We assume that nonsampled options cannot be chosen, although a ‘default’ option can be added to our framework with no change of our main results.

Fig. 1.

A multiaccumulator model with finite sampling resources. The environment produces a large number of options, each characterized by a drift ${\mu }_i$, unknown to the agent and drawn from a prior distribution characterized by hyperparameters $\theta$, which is known to the agent. The agent has a finite resource $T$, that they divide and allocate across options, ${\sum }_i{t}_i=T$, in order to sample them. In practice, the agent allocates finite sampling time to a finite number $M$ of accumulators to infer their unknown drifts. After allocation, evidence (red lines) is optimally integrated by the accumulators. The agent observes the integrated evidence $x_i$ at the end of the accumulation, after time $t_i$, infers the drifts for each of the accumulators and chooses the one that is deemed to have the highest drift (in this case, ${\mu }_M$; green box).

Once total sampling time is allocated, noisy evidence about the drift ${\mu }_i$ of each of the sampled options $i\le M$ is integrated by independent accumulators (Fig. 1, middle) according to the drift‐diffusion process

$$\frac{d{x}_i(t)}{dt}={\mu }_i+{\eta }_i\left(t\right),$$ (2)

where $x_i(t)$ is the accumulated evidence up to time $t$ with initial condition $x_i(0)=0$, and ${\eta }_i(t)$ is a Gaussian white noise with zero mean and fixed variance ${\sigma }^2$, independent and identical for all the accumulators.

The result of the accumulation is the total evidence $x_i$ at time $t_i$, both of which are observed by the agent and constitute the sufficient statistics for the unknown drift ${\mu }_i$ Moreno‐Bote (2010). With these observations, the agent builds the posterior distribution of the drifts by using Bayes rule as:

$$p({\mu }_i|x_i,t_i,\sigma ,\theta )=\frac{\mathcal{L}({\mu }_i|x_i,t_i,\sigma )p_{\theta }(\mu )}{p(x_i|t_i,\sigma ,\theta )},$$ (3)

where $\mathcal{L}({\mu }_i|x_i,t_i,\sigma )=\mathcal{N}(x_i|{\mu }_i{t}_i,{\sigma }^2{t}_i)$ is the likelihood function for the drift, $p_{\theta }(\mu )$ is the prior distribution, and $p(x_i|t_i,\sigma ,\theta )=\int d\mu \mathcal{N}(x_i|\mu t_i,{\sigma }^2{t}_i)p_{\theta }(\mu )$ is the marginal distribution of the evidence, which serves as a normalization constant.

After building these posterior distributions, the agent simply chooses the option with the highest expected drift (Fig. 1, middle, green box), which defines utility, $U(M,\mathbf{x},\mathbf{t},\sigma ,\theta )={\max }_{i\le M}{\widehat{\mu }}_i(x_i,t_i,\sigma ,\theta )$, where $\mathbf{x}=(x_1,\text{…},x_M)$ is the vector of observations for the $M$ accumulators with allocated times $\mathbf{t}=(t_1,\text{…},t_M)$. To avoid notation clutter, from now on, we will stop writing the dependence on $\sigma$ and $\theta$ of the various functions and leave it implicit.

The previous expression is the utility of the choice of accumulator, which depends on the observations and allocation times. However, before time is allocated, the observations $\mathbf{x}$ themselves will be unknown to the agent. Therefore, the expected utility of a given allocation $\mathbf{t}$ is given by taking the expectation of the above utility over all possible observations as:

$$\widehat{U}(M,\mathbf{t})\equiv \mathbb{E}\left[\underset{i\le M}{\max }{\widehat{\mu }}_i|\mathbf{t}\right]=\int d{x}_1\text{…}d{x}_{M}p(x_1,\text{…},x_M|\mathbf{t})\underset{i}{\max }{\widehat{\mu }}_i\left(x_i,t_i\right),$$ (4)

where, using the independence of the accumulators, $p(x_1,\text{…},x_M|\mathbf{t})={\prod }_{k\le M}p(x_k|t_k)$ is the product of the marginal distribution of the evidence.

Optimally inferring the drifts from observations is readily accessible through Bayesian inference as shown above. Thus, the main, and harder, objective of the agent is to optimize the allocation policy, that is, to select both the number of sampled accumulators $M\le N$ and the time $t_i$ allocated to each, in order to maximize expected reward, while satisfying the total sampling time constraint in Eq. 1. This is accomplished by optimizing the utility with respect to $M$ and $\mathbf{t}=(t_1,\text{…},t_M)$ as:

$$(M^{*},{\mathbf{t}}^{*})=arg\underset{M,\mathbf{t}}{\max }\widehat{U}(M,\mathbf{t}).$$ (5)

3. Capacity and time–precision duality

While time can be understood as the resource that the agent allocates, we found a dimensionless scale that expresses their actual sampling capacity, that is, their ability to sample and differentiate between drifts, which we call capacity $C$ (Fig. 2). As the agent integrates noisy evidence through Eq. 2, the likelihood of the drift ${\mu }_i$ for accumulator $i$ is proportional to a Gaussian (Fig. 2a, orange curve) with mean $x_i/t_i$ and variance ${\sigma }^2/t_i$, $\mathcal{L}({\mu }_i|x_i,t_i,\sigma )\propto \mathcal{N}({\mu }_i|\frac{x_i}{t_i}\frac{{\sigma }^2}{t_i})$ Moreno‐Bote (2010). Its variance ${\sigma }^2/t_i$ shows how the sampling time and the variance of the sampling noise are related when inferring the drift ${\mu }_i$: the likelihood gets broader with increasing $\sigma$ or decreasing time $t_i$, reflecting that the precision of the observations is decreased by having more noise or less time, respectively. In fact, the sampling capacity of the agent should capture this duality. Thus, having a fixed capacity could be interpreted as having a fixed noise variance ${\sigma }^2$ for all accumulators and allocating time $T$ between them (Fig. 2b, left) or as having a fixed sampling time $T$ for each of the accumulators and allocating precision $1/{\sigma }^2$ between them (Fig. 2b, right).

Fig. 2.

Time/precision duality and the notion of capacity. (a) The likelihood of the drift $\mu$ (in orange) given the evidence has variance ${\sigma }^2/t_i$ and the prior distribution of the drifts (in cyan) has variance ${\sigma }_0^2$. These quantities determine capacity as in Eq. 6. (b) Time and sampling noise are intricately related (see text). In this example, allocating time $T/3$ to each accumulator under fixed precision $1/{\sigma }^2$ (left) is equivalent to allocating precision $1/3{\sigma }^2$ to each accumulator under fixed sampling time $T$ (right). (c) Small capacity means that the variance of the observation is much larger than the variance of the prior, indicating that it is difficult to confidently identify the best drift from the observations. (d) In the large capacity limit, it is easier to differentiate the good drifts from the poor ones.

Moreover, the posterior in Eq. 3 depends on the prior as well (Fig. 2a, cyan curve). For fixed evidence, the broader the prior is, the easier it is to differentiate between sampled drifts, since the expected squared distance between two drifts drawn from the same distribution is twice its variance $\mathrm{Var}[p_{\theta }(\mu )]$ (see online Appendix A.1). Therefore, we define the capacity allocated to option $i$ as the ratio between the precision of the observation and the precision of the prior,

$$c_i=\frac{\mathrm{Var}[p_{\theta }(\mu )]}{\mathrm{Var}\left[\mathcal{N}\left({\mu }_i|\frac{x_i}{t_i},\frac{{\sigma }^2}{t_i}\right)\right]}=\frac{{\sigma }_0^2}{{\sigma }^2}{t}_i.$$ (6)

Adding the individual capacities results in the total sampling capacity of the agent,

$$C=\sum _i{c}_i=\frac{{\sigma }_0^2}{{\sigma }^2}T.$$ (7)

For the rest of this article, we stick to the interpretation of allocating capacity as dividing the total time $T$ while fixing the accumulation noise $\sigma$, such that the variable we can control is the sampling time allocated to each option, keeping in mind that all the results presented below can be readily reinterpreted as dividing precision while giving to all options the same sampling time.

4. Results

Optimally dividing sampling capacity $C$ into options is an a priori hard problem due to its high dimensionality. However, we show in Subsection 4.2 that the optimal allocation lies within the family of even allocations, where $M$ options receive equal sampling time $t_i=t\equiv T/M$, while the remaining others are given no time. Thus, finding the optimal policy reduces to finding the optimal number $M$ of accumulators to sample.

4.1. Even sampling

In this subsection, we exploit the structure of even sampling. First, the posterior mean of the drift ${\widehat{\mu }}_i(x_i,t)$, computed from Eq. 3, is a monotonously increasing function of the evidence $x_i$ for any prior (see proof in Section A.2 in online Appendix). Therefore, the option that maximizes the posterior mean ${\widehat{\mu }}_i$ is the one that has the highest evidence $x_i(t)$, as all $M$ sampled options are given the same sampling time $t$. This allows us to work by maximizing evidence instead of maximizing the posterior means of the drifts in Eq. 4. Second, by changing variables $y\equiv {\max }_i{x}_i$, and using the probability distribution of the maximum $y$, denoted by $p_{\max }(y|t,\sigma ,\theta )$, the expected utility in Eq. 4 can be recast in the one‐dimensional integral

$$\widehat{U}(M,t)=\int \mathrm{dy}{p}_{\max }(y|t)\widehat{\mu }(y,t).$$

Finally, given that the $M$ options are sampled evenly, the probability distribution of the maximum can be simplified by using the cumulative distribution of the evidence $x$ for an arbitrary accumulator, $F_x(y|t)={\int }_{-\infty }^{y}d{x}^{\prime }p(x^{\prime }|t)$, where $p(x|t)$ is the marginal of the evidence $x$ of the accumulator, as:

$$p_{\max }(y|t)=\frac{d}{dy}{\left[F_x(y|t)\right]}^M.$$ (8)

With all the above, the expected utility in Eq. 4 can thus be written as:

$$\widehat{U}(M,t)=M\int \mathrm{dy}{\left[F_x(y|t)\right]}^{M-1}p(y|t)\widehat{\mu }(y,t).$$ (9)

When the prior distribution is a Gaussian with mean ${\mu }_0$ and variance ${\sigma }_0^2$, it is possible to identify the total capacity $C=\frac{{\sigma }_0^2}{{\sigma }^2}T$ explicitly and Eq. 9 simplifies to

$$\widehat{U}(M,C)={\mu }_0+\frac{M{\sigma }_0}{\sqrt{1+\frac{M}{C}}}{\int }_{-\infty }^{\infty }\mathrm{dy}{\left[\mathrm{\Phi }(y)\right]}^{M-1}\mathcal{N}(y|0,1)y,$$ (10)

where $\mathrm{\Phi }(y)=\frac{1}{2}[1+erf(\frac{y}{\sqrt{2}})]$ is the cumulative distribution function of a normal distribution.

Plotting the utility in Eq. 10 as a function of the number of sampled accumulators $M$ reveals a clear breadth–depth tradeoff (Fig. 3a). At the depth limit, $M=1$, only one accumulator is sampled and it is given all sampling time $T$. In this case, the expected utility will simply be the expected value of the prior, ${\mu }_0=0.5$ (Fig. 3a, left point), since there is no choice to be made between accumulators. At the breadth extreme, $M/C\to \infty$, the evidence gathered for each accumulator is very noisy because each has been allocated a very short sampling time, and thus choosing any will amount to an expected utility again equal to the prior mean (rightmost points). Therefore, for all capacities, there is an intermediate optimal value for the number of accumulators to sample, $M^{\ast }$.

Fig. 3.

Expected utility as a function of sampled accumulators exhibits the breadth–depth tradeoff. Results for the Gaussian prior case (${\mu }_0=0.5,{\sigma }_0^2=1$), for various different capacities. Blue points denote the maxima. Note log horizontal scale (points, Monte Carlo simulations; lines, theoretical predictions, Eq. 10). (a) For large capacities, the optimal number of sampled accumulators changes with the capacity. (b) For small capacities, the optimum is independent of capacity and equal to five.

4.1.1. Sharp transition between the small and large capacity regimes

Our main result is that the optimal allocation policies are qualitatively different at small and large capacity, and that there is a abrupt transition between the two regimes. We provide useful asymptotic analytical expressions for the utility in Eq. 9 and the optimal $M^{\ast }$ in both limits and describe their characteristic features.

The limit $C\ll 1$ corresponds to the case where the uncertainty in the observation ${\sigma }^2/T$ is much larger than the variance of the prior ${\sigma }_0^2$, that is, the Gaussian likelihood is much wider than the prior (Fig. 2c). In this limit, we find that the utility in Eq. 9 can be expanded a series in powers of $\sqrt{C}$, which at first order is given by (see online Appendix, Section A.3)

$$\widehat{U}(M,C)={\mu }_0+{\sigma }_0\sqrt{\frac{C}{2\pi }}\left[\sqrt{M}{\int }_{-\infty }^{\infty }dzz\exp (-\frac{z^2}{2}){\left(\frac{1}{2}+\frac{1}{2}erf\left(\frac{z}{\sqrt{2}}\right)\right)}^{M-1}\right]+\mathcal{O}(C).$$ (11)

Remarkably, this expression holds for any prior distribution as long as capacity is small enough. Let us now note that the only dependence on $M$ appears in the quantity in square brackets, so we isolate it to look for the optimal $M$. Since capacity does not play a role here, we see that $M^{\ast }$ will be constant as a function of small capacity. Furthermore, using Extreme Value Theory (see Section A.5 in online Appendix), we find that this quantity decreases with $M$ for large $M$. This means that sampling many accumulators will not be optimal, following the intuition that there is no point in sampling many options when having scarce resources. On the other hand, and as noted before, it is easy to see that expected utility attains its lowest value when $M=1$, since in this case, there is no choice to be made. Thus, the optimal $M$ is attained at some intermediate value. Given these observations, the optimum can be thus found numerically by varying M, and it happens when

$$M^{\ast }(C\ll 1)=5,$$

which can be checked visually for various values of small capacity in Fig. 3b, which also validates the approximation in Eq. 11. It is important to highlight that when capacity is strictly zero, expected utility does not depend on $M$, and is equal to the prior mean, since choosing options to sample has no effect when there is no time to be allocated. However, as long as the small capacity is finite, an optimal number of options to sample equal to five emerges, regardless of the prior and the value of capacity. We have confirmed this strong prediction by direct numerical integration of Eq. 9 using different prior distributions, including Gaussian, uniform, and bimodal (Fig. 4), which also holds even when a nonsampled, default, option can be chosen (diamond markers).

Fig. 4.

The optimal number of sampled accumulators undergoes qualitatively different behaviors at small and large capacity values. Results come from searching the maximum expected utility via Monte Carlo simulations (points) and numerical integration (lines) for a Gaussian (blue line; Eq. 10), uniform (pink; Eq. A.6), and a bimodal (green; Eq. A.7) priors (illustrated in inset). We used ${\mu }_0=0.5$ for all priors and ${\sigma }_0^2=1/12$ for the Gaussian prior to match the uniform distribution. For the bimodal prior, the variance of each mode equals ${\sigma }_0^2$. Dashed red line corresponds to the asymptotic limit in the Gaussian prior case, Eq. 13. Dashed gray line (almost overlaid by the pink line) is the best power law fit for the uniform prior case ($M^{\ast }\propto C^a,a=0.33$). Diamonds (overlaying most of points) indicate simulations with a ‘default’ option.

The opposite limit $C\gg 1$ corresponds to the case where the precision of the observation is much greater than the one of the prior (Fig. 2d). Intuitively, this means that the quality of the observations is good enough to likely differentiate the drifts between two randomly chosen accumulators, and thus we expect the optimal number of accumulators to increase with increasing capacity, giving a qualitatively different behavior than at small capacity. Hence, we make this assumption to inspect the optimality of Eq. 9 for this large capacity limit, which we find to be consistent with the numerical results shown below. In particular, when the prior distribution is Gaussian, the expected utility in Eq. 10 has the following asymptotic behavior,

$$\widehat{U}(M\gg 1,C)\to {\mu }_0+{\sigma }_0\frac{b_M}{\sqrt{1+\frac{M}{C}}},$$ (12)

where $b_M={(2\log (M)-\log (\log (M))-\log (4\pi ))}^{1/2}$ (see Section A.5 in online Appendix). By relaxing $M$ to be continuous, we can maximize expected utility, and we find that the optimal number of sampled options for large capacity satisfies, up to leading order, the implicit equation $M^{\ast }\log (M^{\ast })=C$. After inverting it, the optimal number of sampled options is

$$M^{\ast }(C\gg 1)=\frac{C}{W(C)},$$ (13)

where $W(C)$ is the Lambert function. This asymptotic limit provides a very good approximation to the optimal $M^{\ast }$ at large $C$ obtained from direct numerical integration of Eq. 10 (Fig. 4; red dashed line, theory; blue points, simulations). For prior distributions other than the Gaussian, we rely on numerical integration of Eq. 9 (see Sections A.6 and A.7 in online Appendix for analytical expressions). For a uniform prior, the optimal number of sampled options increases as a power law with an exponent close to $1/3$ (Fig. 4, pink), while for a bimodal prior, the optimal number increases in a similar fashion to the Gaussian prior case (green). While differences of asymptotic limits are due to the presence of bounded or unbounded drifts in the priors, in all cases, the increase is sublinear, indicating that increasingly longer times are allocated to each of the sampled accumulators as capacity increases.

The above results show that there are two distinct regimes, one at small and another at large capacities, characterized by qualitatively different optimal allocations: while at small capacity, the optimal number of sampled accumulators should be five regardless of the prior, at large capacity, the optimal number of sampled accumulators grows sublinearly regardless of the tested prior. Further, we observe that there is an abrupt transition between the two regimes as capacity grows, with a bump being observed at intermediate capacity values.

4.2. Even allocation is optimal

Above, we have assumed that we could find the optimal time allocation within the subset of even allocations, such that, given finite total time $T$, an agent just needs to determine how many options will be sampled and split equal time to all of them. Conveniently, this set is discrete and thus amenable to effective search of the optimum. However, in general, the set of allocation policies is the infinite‐dimensional simplex ${\sum }_i{t}_i=T$, $t_i\ge 0$ for all $i$, as a priori the agent could unevenly split time to options in any arbitrary way. Despite its infinite‐dimensionality, we have seen in the case of even sampling that it is optimal to ignore (infinitely) many options, such that $t_i>0$ only for $i\in \{1,\text{…},M\}$, with finite $M$, to which we will refer as having $M$ active dimensions.

To address the most general case, using the above intuitions, we first generalize the expected utility, Eq. 9, to the case when allocated time is unevenly distributed among $M$ accumulators, as:

$$\widehat{U}(M,\mathbf{t})={\int }_{-\infty }^{\infty }dy\frac{d}{dy}\left[\prod _{i=1}^M{F}_x(y|t_i)\right]\widehat{\mu }(y,t_i),$$ (14)

where $F_x(y|t_i)$ is the cumulative distribution function of the posterior when using $t_i$ sampling time. Our goal is then, for every $M$, to find the allocation $\mathbf{t}$ that maximizes Eq. 14 under the capacity equality constraint and the inequalities $t_i\ge 0$ for all $i$, and then select the optimal $M$, the one that achieves the highest utility.

In this more general setup, an even allocation corresponds to the symmetrical point in $M$ active dimensions given by ${\mathbf{t}}_M^e$, where $t_{M,i}^e=T/M$ for $i=1,\text{…},M$ (superscript reflects ‘even’ allocation). As the expected utility in Eq. 14 is symmetric under any permutation $t_j\leftrightarrow t_k$ for any $j$ and $k$, all its partial derivatives have to be equal at ${\mathbf{t}}_M^e$. Therefore, every even allocation for each $M$ corresponds to a critical point of the constrained optimization problem (see online Appendix, Section A.8).

We still need to characterize these critical points in order to show that the global maximum is indeed an even allocation. We first remember that the optimal number of active dimensions $M$ needs to be found, and thus it is useful to see how expected utility varies as a function of $M$. To do this, we note that any $M$‐dimensional simplex is in fact the border of an ($M+1$)‐dimensional simplex. For example, for $M=2$, the constraints describe a line segment, or 1‐simplex, where we have the symmetric critical point ${\mathbf{t}}_2^e=(T/2,T/2)$ (Fig. 5b, black circle). We then notice that the line $t_1+t_2=T$ is one of the three edges of the triangle, or 2‐simplex (Fig. 5c: pink lines are the edges of triangle), where in fact we have another symmetric critical point in its interior (black triangle). With this, we can ‘visualize’ the infinite‐dimensional nature of this problem, since all critical points of the utility lie at the edges of a higher dimensional simplex.

Fig. 5.

Even allocations correspond to critical points of utility lying at the center of $M$‐simplices. (a) In one dimension, there is only one point that complies with the constraint. (b) For $M=2$ dimensions, constraints define a line segment or 1‐simplex. The circle depicts the symmetric critical point ${\mathbf{t}}_2^e$. (c) For $M=3$, constraints form a triangle or 2‐simplex. The black triangle is the symmetric critical point ${\mathbf{t}}_3^e$. The colors at the extremes reflect the minimum and maximum utility reached in this simplex, which was computed with Monte Carlo simulations of Eq. 4 for the Gaussian prior with $T=0.1$, $\sigma =1$, ${\sigma }_0=1$, ${\mu }_0=0.5$. (d) Expected utility computed along directions that go orthogonally from ${\mathbf{t}}_M^e$ to ${\mathbf{t}}_{M+1}^e$ (as illustrated with orange arrows in panel c, same parameters). The red dot shows the maximum occurring at ${\mathbf{t}}_5^e$. (e) Using the stochastic projected gradient ascent detailed in Section A.8, we initialized the algorithm at random points (10 shown here) in a high‐dimensional simplex and measured the coefficient of variation (CV) of the allocation vector at every step of the algorithm until convergence, for various values of capacity. Zero CV implies even allocation.

To assess the landscape of expected utility in high‐dimensional simplices, we can evaluate it at all symmetric critical points ${\mathbf{t}}_M^e$ and along directions that go orthogonally between them (Fig. 5c, orange arrows). Thus, we devised a one‐dimensional path that allows to continuously connect all symmetrical critical points, and applied it to the small capacity limit $C\ll 1$. As we move from the 1‐simplex to higher dimensional simplices (as in Fig. 5c), we find that first utility increases, reaching a maximum at the even allocation in $M=5$ dimensions, and then decreases (Fig. 5d). Therefore, critical points ${\mathbf{t}}_2^e$, ${\mathbf{t}}_3^e$, and ${\mathbf{t}}_4^e$ are ‘saddle'‐like points, as they are maxima in the interior of their corresponding simplex, and minima as one moves to the interior of the higher dimensional simplex.

Although the above analysis suggests that the optimum lies at an even allocation point, it is still unclear whether there are other critical points that are asymmetrical and have a larger utility. To argue that the presence of nonsymmetrical local optima is unlikely, we used a stochastic gradient projection method (Fletcher, 2013) that maximizes expected utility subject to the constraints, and applied it to the Gaussian prior case (see Section A.8 in online Appendix for details). Indeed, we find for various capacities that, regardless of the initial condition, that is, random initial allocations, a maximum utility is attained when time is evenly divided (Fig. 5e), and the global maxima coincide with the ones found in the previous sections.

5. Discussion

We have studied a model of multialternative decision making where an agent can allocate finite sampling resources to options and choose the best one among them. We found that the capacity of the agent depends on both the amount of sampling resources, that is, time or precision, as well as on the discriminability of the options in the environment. As a function of capacity, optimal policies undergo an abrupt transition: at small capacity, allocating time to a handful of options is optimal; at large capacity, the number of options grows sublinearly, well below the actual sampling capacity of the agent. Our results show that decision bottlenecks, such as option‐narrowing, can arise from optimal policies in the face of uncertainty, and provide so far untested predictions on choice behaviors in multialternative decision making as a function of capacity.

Seemingly strict limits pervade cognition, from the so‐called attentional bottleneck (Deutsch & Deutsch, 1963; Treisman, 1969; Yantis & Johnston, 1990), over working memory (Brady, Konkle, & Alvarez, 2011; Cowan et al., 2005; Luck & Vogel, 2013; Miller, 1956; Ma et al., 2014), to executive control (Norman & Shallice, 1986; Shenhav et al., 2017; Sleezer, Castagno, & Hayden, 2016). These limits might result from using scarce neuronal resources or from using them inefficiently. However, a likely alternative is that bottlenecks reflect strategies that make optimal use of limited but large resources. Indeed, past work has recognized that some apparent limits, most notably dual tasking bottlenecks (Fischer & Plessow, 2015; Meyer & Kieras, 1997), could be the result of optimal allocation of finite resources to avoid overlap and interference between the different representations needed to solve the two tasks (Feng, Schwemmer, Gershman, & Cohen, 2014; Meyer & Kieras, 1997; Zylberberg, Dehaene, Roelfsema, & Sigman, 2011). Further, it has been recognized that the narrow focus of attention could be at the heart of solution to the binding problem by integrating separate features into a coherent object (Treisman, 1998), and thus, its narrowness might reflect a function more than a limitation. Our work follows this line of argument and provides for the first time a quantitative account for why it is optimal for an agent to consider a handful of options in the face of uncertainty, well above two but well below 10. In addition, our results shed light on why people might ignore hundreds of accessible options and focus resources to a very small number of options (Hauser & Wernerfelt, 1990; Iyengar & Lepper, 2000; Scheibehenne et al., 2010). Thus, some of the seemingly strict limits in decision making can be the result of optimal policies that favor depth versus breadth processing of the options.

It has been long recognized that people often consider a small set of options while ignoring many others (Hauser & Wernerfelt, 1990; Mehta et al., 2003; Payne, 1976; Stigler, 1961). In the ‘consumer’ literature, this is explained by arguing that small consideration sets are favored because they optimally balance the probability of finding a good option in the set with the search and mental costs incurred in adding new options to that set. These models thus assume that resources are not limited, but are costly. In contrast, the assumptions in our work do not explicitly tune the cost of sampling, but rather an implicit cost arises naturally from the strict capacity constraint, which depends intrinsically on the agent as well as extrinsically on the environment. A more fundamental distinction is that previous work did not focus on allocating resources intensively into the options, such that the only decision was whether to include an option into the set or not, without considering the amount of resources allocated to it. This distinction makes that problem drastically different than the tradeoffs of the breadth–depth dilemma considered here. This can explain why transitions of optimal policies as a function of agent's parameters have not been reported before.

Most current theories of perceptual and value‐based decision making are based on accumulation of evidence that favors certain hypotheses over others (Drugowitsch et al., 2012; Gold & Shadlen, 2007; Moreno‐Bote, 2010; Ratcliff & Smith, 2004). When combined with sampling costs and rewards, a normative theory emerges where it is optimal to only accumulate evidence up to a bound (Drugowitsch et al., 2012; Gold & Shadlen, 2007). By letting the information between the competing options be processed parallelly, a tradeoff between speed and accuracy of the choice emerges, and it is then possible to derive optimal policies under various further assumptions (Callaway et al., 2021; Jang et al., 2021; Tajima et al., 2019; Vul et al., 2014). For instance, the work of Vul and colleagues assumes that only one option is correct, that is, that there are only two types of reward. Combined with an opportunity cost, taking zero to one sample in this setting becomes optimal under large parameter regions of the cost of time (Vul et al., 2014). However, multialternative decision making requires estimating the subjective value of offers, and none of them is correct in any absolute sense, which can favor not single‐sample, deep strategies (Moreno‐Bote et al., 2020). The work of Jang and colleagues aims to optimally allocate attentional resources by solving numerically the Bellman equation in binary choice (Jang et al., 2021), but extending this framework to (many) multialternative decisions is intractable. While the speed–accuracy tradeoff is a ubiquitous phenomenon in sequential decisions, there are certainly other features of decision making that need to be studied in isolation to advance our understanding of the various challenges that arise in multialternative decision making.

In our work, we have implemented the prevalent feature of evidence accumulation, but we have highlighted other characteristics of decision making, such as finite sampling resources, delayed feedback, and limited interaction with the environment, that are critical in many real‐world examples (see Introduction). These assumptions differentiate our results with those of previous work. Most importantly, we have not considered a sequential process where the evidence gathered during the accumulation is observable and thus, it cannot be used to stop the accumulation process. This is an important case when accumulation of evidence happens in a decentralized manner by, for example, different groups of neurons, and only the final result of the accumulation is conveyed to another set of neurons where the comparison and choice takes place. Indeed, parallel sampling of information is a ubiquitous ingredient in theories of decision making (Busemeyer & Townsend, 1993; Glöckner & Betsch, 2008). In this work, limited capacity in parallel sampling is understood as a limitation on available sampling precision, thus allowing for the allocation of attentional resources under parallel evidence accumulation with time pressure (Fig. 2). These ingredients force the decision maker, in a deliberate or automatic way, to allocate resources into the options in advance in a strategic fashion, prompting the need to trade sampling breadth over depth.

Although previous work has characterized optimal breadth–depth tradeoffs in multialternative choices like the ones studied here, it has been assumed that agents have a finite ‘discrete’ capacity (Moreno‐Bote et al., 2020). Our assumption of a continuous resource (e.g., time or precision) that can be infinitely divided has allowed us to uncover qualitatively novel optimal policies at small capacity. We have, therefore, been able to derive optimal policies that trade off breadth with depth search in (many) multialternative settings where using traditional sequential decision frameworks would be intractable. Integrating resource allocation with sequential decision making into a single theory of dynamic allocation will be most relevant to understand human decision making, but its study is deferred to the near future. In any event, any agent with finite capacity cannot avoid the problem of first deciding how many options to allocate capacity to, and thus breadth–depth tradeoffs as described above will be generally at play.

Bounded rationality accounts (Gershman et al., 2015; Griffiths et al., 2015; Russell & Wefald, 1991; Simon, 1972) propose that cognition results from the finite limits of the nervous system from where it emerges. Our work follows this line of research in two ways. First, we propose that agents indeed have a finite sampling capacity that can be arbitrarily allocated to the available options. However, an important assumption in our work is that while the intrinsic resources of an agent might seem large, the interaction of the agent with the environment might render their effective decision‐making capacity small. Therefore, capacity is not an absolute quantity that describes an agent, but a relative quantity that contextualizes the agents and characterizes how well they are suited to solve a given task in the world. An important contribution of our work is to show that optimal policies depend on effective capacity in a highly nonlinear way, such that small‐capacity agents would behave qualitatively different than large‐capacity agents (or even the behavior of the same agent operating in different environments could be qualitatively different). This is clearly a prediction that can be tested with humans where time or other resources are constrained and varied on a trial by trial basis. Second, agents perform the allocation before feedback is received, which relates to a bounded‐optimal agent that is optimized at ‘design'‐time, which eliminates the paradox of perfect rationality by not letting the agent optimize their decisions at run‐time (Russell & Subramanian, 1994), an argument that further supports the validity and relevance of two‐stage decisions.

Another important result of our work is that evenly dividing time to a small set of options is optimal when they are initially indistinguishable. This optimal division of resources coincides with the $1/N$ heuristic rule (Gigerenzer & Gaissmaier, 2011) or equality heuristic (Messick, 1993), which has proven to be implemented in human decision making and highly efficient as a portfolio strategy (DeMiguel, Garlappi, & Uppal, 2009). In our case, the fact that options are drawn from the same prior (known to the agent) contributes to the optimality of the even allocation. Although the optimal allocation of nonidentically distributed options is not addressed here, this heuristic can be efficient in such situations (Thorngate, 1980). It is important to realize that the optimal low numbers of considered options have been found in the case where their values are not known in advance and come from the same distribution. If agents have strong preferences or have additional information about the expected values of the options (e.g., by sampling them sequentially), then the number of considered alternatives will be further reduced. Of course, if the agents are allowed to sequentially sample options with which they are familiar, a noneven allocation might emerge to be optimal (Callaway et al., 2021; Sepulveda et al., 2020; Tajima et al., 2019). Nonetheless, for binary choice, reward is still maximized at even allocations in sequential sampling when options have not been unevenly sampled in the past (Fudenberg, Strack, & Strzalecki, 2018; Jang et al., 2021). On the other hand, when the number of alternatives is much higher than two, people choose to ignore many of the available options (Thomas, Molter, & Krajbich, 2021), consistent with our findings. Moreover, for fixed‐duration tasks, there is evidence that humans have a choice set of around five in sequential decisions (Reutskaja, Nagel, Camerer, & Rangel, 2011), even if their final allocation might be uneven. This shows once again that a low number of considered options can hardly be taken as evidence of a decisional bottleneck and is more in line with an optimal tradeoff between breadth and depth.

Finally, our results can have important implications for the optimal wiring of neural networks in the brain (Rushworth et al., 2011; Siegel et al., 2015; Vickery et al., 2011; Yoo & Hayden, 2018). First, as just few options should be considered at the same time, it is expected that only those would be encoded in different, albeit possibly overlapping, pools of neurons. Thus, although models consisting of two or three pools that compete for dominance through mutual inhibition can be a sensible idea for binary and ternary decision making (Cisek & Kalaska, 2010; Churchland, Kiani, & Shadlen, 2008; Gold & Shadlen, 2007; Moreno‐Bote, Rinzel, & Rubin, 2007; Roe, Busemeyer, & Townsend, 2001; Usher & McClelland, 2001; Wang, 2008), extrapolating this to many more options (e.g., larger than 10) by splitting neurons into corresponding pools of neurons would be hardly optimal. Our results are, in contrast, consistent with the opposite view that posits that a single pool of neurons is sufficient for decision making (Hayden & Moreno‐Bote, 2018). In this framework, a single pool encodes just one of the available options, the one that is under the focus of attention. Previously attended options produce a background activity against which the current option is compared to, and other options fall outside the representation of the neural network (Hayden & Moreno‐Bote, 2018; Krajbich, Armel, & Rangel, 2010; Lim, O'Doherty, & Rangel, 2011; Redish, 2016; Rich & Wallis, 2016). Thus, comparison and selection between options occurs through a temporal contrast, rather than through mutual inhibition between simultaneously encoded options. This model can be readily extrapolated to multiple many options, with the only dilemma of dividing time or precision into few or many options (like in Fig. 1), thus addressing the associated breadth–depth tradeoffs. The debate of the one‐pool versus several‐pools models remains open (Ballesta & Padoa‐Schioppa, 2019; Hayden & Moreno‐Bote, 2018), but electrophysiology experiments with many options should be able to arbitrate between the two hypotheses under the new computational constraints that we have identified here.

Conflicts of interest

The authors have no conflicts of interest to declare.

Data availability

All the numerical work performed to generate the various figures is available as documented Julia code along with a guided notebook at this public Github Repository: https://github.com/jorgeerrz/finite_time_allocation_paper.

Supporting information

Optimal allocation of finite sampling capacity in accumulator models of multi‐alternative decision making.