A Predictive Approach to Nonparametric Inference for Adaptive Sequential Sampling of Psychophysical Experiments

Abstract

We present a predictive account on adaptive sequential sampling of stimulus-response relations in psychophysical experiments. Our discussion applies to experimental situations with ordinal stimuli when there is only weak structural knowledge available such that parametric modeling is no option. By introducing a certain form of partial exchangeability, we successively develop a hierarchical Bayesian model based on a mixture of Pólya urn processes. Suitable utility measures permit us to optimize the overall experimental sampling process. We provide several measures that are either based on simple count statistics or more elaborate information theoretic quantities. The actual computation of information theoretic utilities often turns out to be infeasible. This is not the case with our sampling method, which relies on an efficient algorithm to compute exact solutions of our posterior predictions and utility measures. Finally, we demonstrate the advantages of our framework on a hypothetical sampling problem.

1. Introduction and Motivation

The application of adaptive measurement methods have a long tradition in psychophysics. The need for such methods is mainly due to the limited number of measurements that can be taken during experiments. Most of the classical methods are motivated by their simplicity, both conceptually and computationally. With the advent of modern computers and the continuing progress in statistical theory, the development of more sophisticated adaptive sampling procedures has recently seen much progress.

Especially the consideration of Bayesian experimental designs based on the information theoretic description of experimental objectives and their numerical approximation (cf. MacKay (1992); Chaloner and Verdinelli (1995)) has moved into the focus of contemporary research. Recent developments are, for instance, the Ψ-method introduced in Kontsevich and Tyler (1999), the consideration of multidimensional stimulus spaces in Kujala and Lukka (2006), and the framework of adaptive design optimization (ADO) for model discrimination proposed in Cavagnaro et al. (2010). Another interesting example is given by Kujala (2010) who considers random cost as a further constraint for experimental observations.

Our contribution to the field is twofold. First, we address the case where no particular statistical model in form of parametric curves can be assumed. We present a complete formal description of a suitable framework for such a nonparametric setting. Second, we do not rely on any numerical approximation and the quantities of interest can be computed efficiently and exactly.

Our method applies to the following experimental setting: In a psychophysical experiment, the causal relation X → Y between a physical stimulus X and the psychological response Y of an observer is investigated. Before the experiment, a discrete set of L stimuli = {x₁, …, x_L} and K possible responses = {y₁, …, y_K } is determined. The stimuli is considered to be ordinal, i.e. the set of stimuli is assumed to be linearly ordered, for instance by strength or any other property associated with the physical parameters. The actual experiment is then performed in a sequential manner. That is, at the n-th stage of the experiment, a particular stimulus X_n ∈ is set and the participant’s response Y_n to that stimulus is recorded.

Figure 1 outlines such a experiment for illustrative purposes, taken from Elze et al. (2011), Experiment 2. Figure 1A shows the experimental setup that involves a two-alternative-forced-choice (2AFC) discrimination task: An observer has to report which of two possible target stimuli has been presented on a computer monitor. The discrimination performance was impaired by the presentation of a second stimulus, the so-called mask stimulus, at a position close to the target location. Within this experimental setting, the actual stimulus of interest X is the time interval between the offset of the target and the onset of the mask, the so-called interstimulus interval (ISI) that takes values in = {20 ms, 40 ms, …, 200 ms}, whereas the response Y takes values in = {correct, incorrect}, depending on whether or not the observer reported the correct target stimulus.

Figure 1.

Illustrative example of a simple psychophysical experiment. A: The observer has to report which of two possible target stimuli were presented on a computer display. The detection of the target stimulus is impaired by a mask stimulus which is presented in close proximity to the position of the target after a variable interstimulus interval (ISI). B: The observed relative frequency of correct responses of a single observer for 20 different discrete ISIs ≤ 200 ms, which have been uniformly sampled. Each ISI occured 16 times.

One of the most often considered approaches to model such an experimental situation is to assume a multinomial sampling law. More specifically, given any stimulus {X = x} the generation of the response Y is thought to be described by multinomial parameters p_x, = (p_x,y)_y∈ ∈ Δ, where

$${\mathrm{\Delta }}_{\mathcal{Y}}=\left\{{\mathit{p}}_{\mathcal{Y}}={(p_y)}_{y\in \mathcal{Y}}\in {\mathbb{R}}^{\mathcal{Y}}\mid p_y\ge 0\text{for}\text{all}y\in \mathcal{Y}\text{and}\sum _{y\in \mathcal{Y}}{p}_y=1\right\}$$ (1)

is the probability simplex, such that the conditional probability of the event {Y = y} given {X = x} is

$$\mathbf{P}[Y=y\mid X=x,{\mathit{p}}_{\mathcal{X},\mathcal{Y}}]=p_{x,y}.$$

Within this setting, the statistical task is then entirely focused on the estimation of the psychometric rates ${\mathit{p}}_{\mathcal{X},\mathcal{Y}}={({\mathit{p}}_{x,\mathcal{Y}})}_{x\in \mathcal{X}}\in {\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}$. A low-dimensional parametric family of functions = {f_θ| θ ∈ Θ} can be used to introduce dependencies among the psychometric rates, such that

$$\mathbf{P}\left[Y=y\mid X=x,\theta \right]=f_{\theta }(x,y).$$

This facilitates the inference task by exploiting structural knowledge about the interlink between the stimuli. The function f_θ is commonly termed psychometric function in the particular case of a binary response, e.g. a 2AFC experiment as outlined above. Especially sigmoid curves are a common choice if the stimulus X can be considered to be real-valued. Their geometric parameters such as slope and threshold can serve to actually define relevant psychophysical quantities of interest. There exists a vast literature on the statistical inference of psychometric rates and functions, see for instance Wichmann and Hill (2001); Kuss et al. (2005), which provide a good entry point into the literature.

From a mere statistical point of view, this parametric approach seems to be a reasonable strategy as it allows sharing statistical strength across stimuli. By learning the psychometric rate for a particular stimulus we also learn about all other rates because dependencies are introduced by the parametric family. Hence, the parametric approach allows seemingly good estimates even with few experimental data. Nevertheless, this modeling approach is unsuitable and can even bear the risk of a severe bias if there is no or only vague knowledge about the potential shapes of the functions f_θ and no member of the proposed parametric family does match with the actual psychometric rates. For instance, in the above example it seems hard to motivate any plausible regular functional form (see Figure 1B).

This form of bias is of course avoided by allowing the psychometric rates to freely range over ${\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}$, which we refer to as the nonparametric approach¹. Clearly, much more data is then needed to draw informative inferences.

In this paper, we use an intermediate approach fairly balancing the advantages and disadvantages of both approaches by exploiting the fact that is of ordinal structure. Loosely speaking, we allow neighboring stimuli to share statistical strength by joining their respective psychometric rates. Each possible way of joining neighboring stimuli imposes a particular partition on the stimulus space , which is then assessed by a suitable Bayesian inference scheme. The resulting model is a variant of the product partition model proposed by Hartigan (1990) and the inhomogeneous Bernoulli process with piecewise constant probabilities described in Endres et al. (2008).

In our description of the model and the respective adaptive sampling procedures we follow the predictive paradigm as pioneered, for instance, in Roberts (1965); de Finetti (1974); Geisser (1993), by putting special emphasis on the prediction for the observables, which are the stimulus-response outcomes of the sequential experiment. By imposing a particular epistemic condition of partial exchangeability, the psychometric rates naturally emerge as a particular limiting statistic of the data rather than an external quantity. The corresponding Bayesian model matches extensionally with a multinomial sampling model with unknown parameters.

2. A Predictive Perspective on Sequential Experiments

2.1. Sequential Construction of Adaptive Sampling Processes

A reasonable adaptive experimental design requires that we have at least partial control of the sampling process concerning the presentation of stimuli. This experimental controllability can be subject to uncertainty, e.g. the experimental setup might be prone to errors in generating the required stimulus. For simplicity, we shall nonetheless assume that we can adjust the stimulus in the way we want. Our personal action policy, which is the subjective assessment of which stimulus might be best to choose, is then expressed by a probability measure P [X]. By setting a stimulus X w.r.t. P [X], we subsequently observe a respective response Y, where our prediction is described by a conditional measure P [Y |X].

In the following, we shall extend this scenario to sequential sampling schemes. We consider that at any sampling step n ∈ ℕ we can freely choose a stimulus X_n that results in the observation of an instance (X_n, Y_n). Let E_n = (X_n, Y_n) denote the n-th experiment, such that we refer to the first n experiments of the overall experimental process E = (E_n)_n∈ℕ by E_n = (E₁, …, E_n) = (X_n, Y_n), where X_n = (X₁, …, X_n) and Y_n = (Y₁, …, Y_n). Two important statistics that summarize the data from the experiments E_n are given by the total count statistic n = (n_x, )_x∈ = (n_x,y )_{x∈ ,y∈} , where n_x,y is the (absolute) frequency of the event {X = x, Y = y} in E_n, and n = (n_x)_x∈ is the related stimulus count statistic, i.e. n_x = Σ_y∈ n_x,y. We illustrate this notation by the following short example.

Example 1

Suppose we run an experiment with a set of stimuli = {a, b, c, d} and dichotomous responses = {0, 1}. If in eight trials we observe that

$$\begin{array}{l}{\mathit{E}}_8=\left(\begin{array}{c}{\mathit{X}}_8\\ {\mathit{Y}}_8\end{array}\right)=\left(\begin{array}{cccccccc}{X}_1& X_2& X_3& X_4& X_5& X_6& X_7& X_8\\ Y_1& Y_2& Y_3& Y_4& Y_5& Y_6& Y_7& Y_8\end{array}\right)\\ =\left(\begin{array}{cccccccc}a& b& a& c& d& b& b& c\\ 1& 1& 0& 1& 0& 1& 0& 1\end{array}\right),\end{array}$$ (2)

then the two statistics just defined are

$$\begin{array}{l}{\mathit{n}}_{\mathcal{X},\mathcal{Y}}=\left(\begin{array}{cccc}{\mathit{n}}_{a,\mathcal{Y}}& {\mathit{n}}_{b,\mathcal{Y}}& {\mathit{n}}_{c,\mathcal{Y}}& {\mathit{n}}_{d,\mathcal{Y}}\end{array}\right)\\ =\left(\begin{array}{cccc}{n}_{a,0}& n_{b,0}& n_{c,0}& n_{d,0}\\ n_{a,1}& n_{b,1}& n_{c,1}& n_{d,1}\end{array}\right)=\left(\begin{array}{cccc}1& 1& 0& 1\\ 1& 2& 2& 0\end{array}\right),\\ {\mathit{n}}_{\mathcal{X}}=(n_a,n_b,n_c,n_d)=(2,3,2,1).\end{array}$$

We now take a sequential and prediction oriented perspective. We fix our expectations about the experimental course E by specifying a sequence of conditional measures in form of kernels π_n(E_n+1 | E_n), n ∈ ℕ₀, where each π_n describes our uncertainty about the outcome of the n + 1-th experiment given the experimental data from the previous n sampling steps. Each such sequence of kernels π_n(· | ·), n ∈ ℕ₀, defines a unique measure π on the space of experimental courses, such that marginally

$$\pi ({\mathit{E}}_n)=\prod _{i=0}^{n-1}{\pi }_i(E_{i+1}\mid {\mathit{E}}_i),n\in \mathbb{N}.$$

Thus, we can formally describe the experimental course E as a random process, which we call the adaptive sampling process E distributed with respect to π (i.e E ~ π). This process describes our belief dynamic since it is constructed from our experimental predictions (π_n)_n∈ℕ0. Each experimental prediction π_n can in turn be constructed from two separate kernels

$${\pi }_n(E_{n+1}\mid {\mathit{E}}_n)={\pi }_n^X(X_{n+1}\mid {\mathit{E}}_n){\pi }_n^Y(Y_{n+1}\mid ,X_{n+1},{\mathit{E}}_n).$$

There exists a very natural interpretation for each of these kernels in terms of experimental design and prediction:

• The Stimulus Placement Rule: ${\pi }_n^X(X_{n+1}\mid {\mathit{E}}_n)$ specifies our action policy in the n-th sampling step. It determines which stimulus X_n+1 we select given the n previous experiments E_n.

• The Response Prediction Rule: ${\pi }_n^Y(Y_{n+1}\mid X_{n+1},{\mathit{E}}_n)$ is our prediction of the response knowing the outcome of the last n experiments E_n, i.e. which response Y_n+1 we expect given the actual stimulus X_n+1.

Given a particular assignment for the prediction rule ${\pi }_n^Y$, we want to learn the stimulus-response relation X → Y in an optimal manner with regard to the inference scheme and external constraints. Thus, we want to derive a placement rule ${\pi }_n^X$ that allows us to adapt the experimental course to our objectives.

In order to understand the logic of adaptive sampling strategies it is worthwhile to first consider the case of a non-adaptive design. Such a conventional design consists of a fixed sequence of stimuli ${\mathbf{x}}^{\ast }={(x_n^{\ast })}_{n\in \mathbb{N}}$, such that

$${\pi }_n^X(X_{n+1}=x\mid {\mathit{E}}_n)=\{\begin{array}{ll}1\hfill & \text{if}x=x_{n+1}^{\ast }\hfill \\ 0\hfill & \text{otherwise}\hfill \end{array}.$$

Clearly, such an action policy does not take any information about the already collected data into account. A more reasonable strategy allows the decision for a stimulus $x_{n+1}^{\ast }$ to depend on the outcomes of the n foregoing experiments E_n, i.e. $x_{n+1}^{\ast }=x_{n+1}^{\ast }({\mathit{E}}_n)$. More precisely, instead of describing one fixed sequence of stimuli we rather specify a decision rule that determines a stimulus $x_{n+1}^{\ast }$ on the basis of the previous experiments E_n. Many of the classical adaptive procedures for testing psychometric functions, such as PEST (Taylor and Creelman (1967)), QUEST (Watson and Pelli (1983)) and the up-down procedures (Levitt (1971)) can be described that way. A comprehensive review of these methods can be found in Leek (2001).

One principled way to obtain a decision rule is to specify a utility measure U_n+1(x, E_n), which quantifies the utility of a stimulus x based on the outcome of the previous experiments E_n. Given such a measure, an optimal stimulus is determined by

$$x_{n+1}^{\ast }({\mathit{E}}_n)=\underset{x\in \mathcal{X}}{argmax}{U}_{n+1}(x,{\mathit{E}}_n).$$ (3)

A proper placement rule in the case of multiple optimal stimuli (E_N ) ⊆ is given by

$${\pi }_n^X(X_{n+1}\mid {\mathit{E}}_n)=\{\begin{array}{ll}\frac{1}{\mid {\mathcal{O}}_{n+1}({\mathit{E}}_n)\mid }\hfill & \text{if}{X}_{n+1}\in {\mathcal{O}}_{n+1}({\mathit{E}}_n)\hfill \\ 0\hfill & \text{otherwise}\hfill \end{array}.$$

Hence, by taking on the utility-oriented perspective, the problem of determining a placement rule becomes a problem of choosing suitable utility measures U_n+1, n ∈ ℕ₀. Many possible utility measures exist and which one we choose depends solely on our objectives. For instance, if we want to place the stimuli in a random but balanced manner, then we could choose the utility measure

$$U_{n+1}(x,{\mathit{E}}_n)=-n_x({\mathit{E}}_n),$$

where n_x is the stimulus count statistic of x. Clearly, the respective placement rule selects stimuli that have seen the least trials. This random uniform sampling scheme has also been called method of constant stimuli (cf. McKee et al. (1985)) within the context of psychophysical experiments. It is usually considered as a non-adaptive strategy (Watson and Fitzhugh (1990)).

Another common and more sensible strategy to obtain a placement rule is suggested by the theory of optimal sequential decisions under uncertainty (cf. DeGroot (2004); Berger (1993)). In principle we should consider that only a finite number of experiments is performed, say N ∈ ℕ, such that we should formulate our objectives in form of a global utility measure u(E_N ) for the outcome of the overall experimental course E_N. As a matter of rationality, one should choose a sequence of descision rules ${\mathit{x}}_N^{\ast }({\mathit{E}}_{N-1})=(x_1^{\ast },x_2^{\ast }({\mathit{E}}_1),\dots ,x_N^{\ast }({\mathit{E}}_{N-1}))$, such that the expected global utility

$$\overline{u}=\sum _{{\mathit{y}}_N\in {\mathcal{Y}}^N}u({\mathit{x}}_N^{\ast },{\mathit{y}}_N)\prod _{n=0}^{N-1}{\pi }_{n+1}^Y(Y_{n+1}=y_{n+1}\mid X_{n+1}=x_{n+1}^{\ast },{\mathit{X}}_n={\mathit{x}}_n^{\ast },{\mathit{Y}}_n={\mathit{y}}_n)$$

is maximized. This optimization problem is highly non-trivial, but can be solved, at least in principle, with backward induction (Berger (1993); Bernardo and Smith (1995); DeGroot (2004)). For a concise description of the backward induction method see in particular Müller et al. (2007). This procedure leads to a sequence of local utility measures u_n+1(x, y, E_n) that are induced from both the global utility u and the predictions π^Y. The optimal stimulus for the n + 1-th experiment is determined by maximizing the expected local utility, i.e.

$$x_{n+1}^{\ast }({\mathit{E}}_n)=\underset{x\in \mathcal{X}}{argmax}\sum _{y\in \mathcal{Y}}{u}_{n+1}(x,y,{\mathit{E}}_n){\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n).$$ (4)

The local utility measure u_n+1(x, y, E_n) for a particular stimulus-response (x, y) is the expected global utility ū as of the n + 1-th sampling step given that all subsequent decisions are made in the same optimal manner. Although this scheme leads in principle to an optimal design, in most cases, except for trivial settings, the actual computation turns out to be infeasible.

It is primarily for this reason that various approximations to such an optimal design have been developed. One of them is to resort to a myopic adaptive optimal design by directly specifying local utility measures u_n+1(x, y, E_n), n = 0, …, N − 1, in an attempt to optimize the global utility. Likewise, the optimal stimulus is determined by (4), such that we optimize the next step, but with the hope that this also maximizes the global utility. We shall discuss particular choices for such measures based on information theoretical considerations in section 3.7.

2.2. Partial Exchangeable Response Processes

Concerning the choice of a proper prediction rule ${\pi }_n^Y$, we already mentioned in the introduction that the response generation X → Y is usually thought to be governed by a multinomial sample law described by some unknown psychometric rates p ∈ Δ. From a predictive perspective this amounts to the assumption of a specific form of partial exchangeability, which has been introduced in de Finetti (1980) as a generalization of the concept of exchangeability (de Finetti (1937)). Roughly speaking, we have to assume that particular temporal orderings within any finite sequence of experiments E_n do not provide relevant information for our predictions.

In order to make this more precise, we need to introduce the following statistics and respective variables. We define the response statistic ${\mathit{y}}_{n_{\mathcal{X}}}^{\mathcal{X}}={({\mathit{y}}_{n_x}^x)}_{x\in \mathcal{X}}$, where ${\mathit{y}}_{n_x}^x=(y_1^x,\dots ,y_{n_x}^x)$. Here $y_i^x=y$ indicates that in the i-th trial in which {X = x} occurred the response outcome was the event {Y = y}. It is crucial to notice that the count statistic n can also be computed from the response statistic. The random variables related to the response statistic are denoted by Y = (Y^x)_x∈ with ${\mathit{Y}}^x={({\mathit{Y}}_i^x)}_{i\in \mathbb{N}}$, such that $Y_i^x=y_i^x$. Since we often need to refer to a particular subset of Y, we also define ${\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={({\mathit{Y}}_{n_x}^x)}_{x\in \mathcal{X}}$, where ${\mathit{Y}}_{n_x}^x=(Y_1^x,\dots ,Y_{n_x}^x)$, x ∈ , i.e. ${\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}$.

Example 2

In our previous example the response statistic ${\mathit{y}}_{n_x}^x$ is given by

$${\mathit{y}}_2^a=(1,0),{\mathit{y}}_3^b=(1,1,0),{\mathit{y}}_2^c=(1,1),{\mathit{y}}_1^d=(0),$$

such that we observed that

$$({\mathit{Y}}_2^a,{\mathit{Y}}_3^b,{\mathit{Y}}_2^c,{\mathit{Y}}_1^d)=\left(\begin{array}{cccc}{Y}_1^a& Y_1^b& Y_1^c& Y_1^d\\ Y_2^a& Y_2^b& Y_2^c& \\ & Y_3^b& Y_3^c& \end{array}\right)=\left(\begin{array}{cccc}1& 1& 1& 0\\ 0& 1& 1& \\ & 0& & \end{array}\right),$$

We now proceed as follows. Instead of specifying the prediction rule ${\pi }_n^Y$ directly, we rather fix a probability measure P for Y, i.e Y ~ P, such that we treat Y as a random process called the response process. This response process is meant to describe our personal beliefs about the observational part of the experiments irrespective of the actual underlying sequence of stimuli X_N, which clearly depends on our action policy. A proper prediction rule is then given by

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n)=\frac{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y\right]}{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]}.$$ (5)

Furthermore, and more importantly, the response process Y can be utilized to derive an intrinsic statistical model for X → Y by requiring the following form of partial exchangeability (cf. Link (1980); de Finetti (1980)):

The response process Y is said to be partial exchangeable iff

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]$$

for every two responses ${\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}$ and ${\stackrel{\sim }{\mathbf{y}}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}$, which share the same count statistic n. This is equivalent to require that the response process Y_n is summarized by the count statistics n (cf. Lauritzen (1974)), i.e. every sequence ${\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}$ with the same count statistic is predicted to be equally likely. We illustrate this notion of exchangeability by the following example.

Example 3

Consider we observed the data set in (2), see example 1. The alternative observations

$$({\mathit{Y}}_2^a,{\mathit{Y}}_3^b,{\mathit{Y}}_2^c,{\mathit{Y}}_1^d)=\left(\begin{array}{cccc}0& 1& 1& 0\\ 1& 1& 1& \\ & 0& & \end{array}\right)\text{or}\left(\begin{array}{cccc}0& 1& 1& 0\\ 1& 0& 1& \\ & 1& & \end{array}\right),$$

do all preserve the count statistics n, such that we would judge them all to be equally likely under the condition of partial exchangeability. Note that all of these equivalent observations are related by particular permutations, i.e. by exchanging positions within each of the columns ${\mathit{Y}}_{n_x}^x$, x ∈ , but not across them.

An import subclass of partially exchangeable response processes is described by the multinomial sampling laws², where

$${\mathit{Y}}^x~\text{Mult}[{\mathit{p}}_{\mathcal{X},\mathcal{Y}}],$$

such that the marginal probability mass function is given by

$$f_{\text{Mult}[{\mathit{p}}_{\mathcal{X},\mathcal{Y}}]}[{\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}]=\prod _{x\in \mathcal{X},y\in \mathcal{Y}}{p}_{x,y}^{n_{x,y}}.$$

This class is especially important because for each partial exchangeable random process there exists a mixing measure μ on ${\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}$ (Link (1980); Bernardo and Smith (1995)), also called a de Finetti measure, such that

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]={\int }_{{\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}}{f}_{\text{Mult}[{\mathit{p}}_{\mathcal{X},\mathcal{Y}}]}\left[{\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]\mu (\text{d}{\mathit{p}}_{\mathcal{X},\mathcal{Y}}).$$

Therefore each partial exchangeable random process is a mixture of the multinomial sampling laws. Within this context, each ${\mathit{p}}_{\mathcal{X},\mathcal{Y}}\in {\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}$ can be interpreted as a possible limit of the relative frequencies, i.e.

$$\underset{n_x\to \infty }{lim}\frac{{\mathit{n}}_{x,\mathcal{Y}}}{n_x}={\mathit{p}}_{x,\mathcal{Y}},x\in \mathcal{X},$$

such that the measure μ expresses our prediction for these limits. Hence, under the condition of partial exchangeability we can formally identify these limits as the psychometric rates we are uncertain about, but with the important distinction that the rates are not external quantities but asymptotic statistics of the response process itself. The corresponding Bayesian model can thus be described by

$${\mathit{Y}}^{\mathcal{X}}~\text{Mult}[{\mathit{p}}_{\mathcal{X},\mathcal{Y}}],$$

whereas

$${\mathit{p}}_{\mathcal{X},\mathcal{Y}}~\mu .$$

By conditioning on finite data ${\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}$, the resulting response process is still partial exchangeable, such that the respective de Finetti measure $\mu (·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}})$ describes our posterior belief about the psychometric rates and can be seen to be a Bayesian posterior of p.

3. Response Processes with Proximally Related Stimuli

Here, we introduce a particularly useful instance of a partial exchangeable response process, namely the multibin Pólya mixture urn process, which allows a flexible modeling of similarities between stimuli. We first consider the case of only one stimulus and continue with the easiest non-trivial case of two stimuli before we develop the full process with multiple stimuli. Afterwards we shall consider the construction of respective adaptive sampling procedures.

3.1. One stimulus

Let us consider only one stimulus, i.e. = {x}, with the following probability assignment for the response process Y ^x: We say that Y ^x is a Pólya urn process with parameters α_x, = (α_x,y )_y∈ , where α_x,y > 0, if

$$\mathbf{P}[{\mathit{Y}}_{n_x}^x]=\frac{\text{Beta}({\mathit{n}}_{x,\mathcal{Y}}+{\mathit{\alpha }}_{x,\mathcal{Y}})}{\text{Beta}({\mathit{\alpha }}_{\mathcal{Y}})},$$

where Beta is the multinomial beta function. The corresponding prediction rule is

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{X}}_n,{\mathit{Y}}_n)=\frac{n_{x,y}+{\alpha }_{x,y}}{n_x+{\alpha }_x},$$

where α_x = Σ_y∈ α_x,y, which is commonly known as the (generalized) Bayes-Laplace rule. The parameters α_x, have a natural interpretation as pseudo counts added to the actual counts n_x, . The term Pólya urn stems from the original introduction as an urn model in Eggenberger and Pólya (1923). In this picture, balls are successively drawn from an urn that is initially filled with K balls of different colors. Each color indicates a particular y ∈ and the respective ball is assigned an initial weight of α_x,y. At each sampling step, a ball is thought to be drawn with chance given by its individual mass and put back to the urn with another ball of the same color and mass one. The so generated sequence of colors Y ^x is then described by the above Pólya urn process, which is partial exchangeable.

The respective de Finetti measure is given by a Dirichlet distribution with parameters α_x, , i.e.

$${\mathit{p}}_{x,\mathcal{Y}}~\text{Dir}\left[{\mathit{\alpha }}_{x,\mathcal{Y}}\right],$$

where the respective density is

$$f({\mathit{p}}_{x,\mathcal{Y}})=\frac{1}{\text{Beta}({\mathit{\alpha }}_{x,\mathcal{Y}})}\prod _{y\in \mathcal{Y}}{p}_y^{{\alpha }_{x,y}-1}.$$

Conditional on finite data $Y_{n_x}^x$, the resulting response process is again a Pólya urn process with parameters ${\mathit{\alpha }}_{x,\mathcal{Y}}^{\prime }$ given by the update rule

$${\alpha }_{x,y}^{\prime }={\alpha }_{x,y}+n_{x,y},y\in \mathcal{Y},$$

i.e. the observed counts n_x, are just added to the pseudo counts α_x, . Hence, assuming a Pólya urn process for describing our personal predictions justifies the formal adoption of the standard Bayesian model of multinomial sampling and a subjective prior in form of the Dirichlet distribution.

3.2. Two stimuli

We now consider a stimulus space with two elements, say = {x₁, x₂}. Based on the previously described Pólya urn process we can think of two extremal urn models for the response process Y. First, we consider the case where the underlying mechanisms of (X = x₁) → Y and (X = x₂) → Y are identical. More precisely, we say that x₁ and x₂ are similar, denoted x₁ ~ x₂, if we expect that there is no difference in the generation of Y given either x₁ or x₂. This similarity allows us to define a bin b = {x₁, x₂}, such that we can exploit the similarity by joining the observations from both stimuli³. We call this binning scheme B₁ and conditional on it the response process Y is characterized by

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B_1\right]=\frac{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}{\text{Beta}({{\mathit{\alpha }}_{b,}}_{\mathcal{Y}})},$$

where n_b, = (n_b,y )_y∈ is the total count n of observations joined in bin b, i.e. n_b,y = n_x1,y + n_x2,y, y ∈ , and α_b, describes the pseudo counts. The induced prediction rule is

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n,B_1)=\frac{n_{b,y}+{\alpha }_{b,y}}{n_b+{\alpha }_b},$$

where x ∈ b. The so described response process is partial exchangeable and the respective de Finetti measure μ(·|B₁) can informally be described by the density

$$f({\mathit{p}}_{\mathcal{X},\mathcal{Y}})=\frac{1}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}\prod _{y\in \mathcal{Y}}{p}_{b,y}^{{\alpha }_{b,y}-1}\prod _{x\in b}\delta ({\mathit{p}}_{b,\mathcal{Y}}-{\mathit{p}}_{x,\mathcal{Y}}),$$

where δ is the Dirac delta function. That is, the measure μ(·|B₁) concentrates on the diagonal of the product simplex ${\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}={\mathrm{\Delta }}_{\mathcal{Y}}^{x_1}\times {\mathrm{\Delta }}_{\mathcal{Y}}^{x_2}$, which is the set

$$\{{\mathit{p}}_{\mathcal{X},\mathcal{Y}}\in {\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}\mid p_{x_1,y}=p_{x_2,y},y\in \mathcal{Y}\},$$

such that if we identify both psychometric rates p_x1, and p_x2, by just one rate p_b, ∈ Δ, then this rate is described by a Dirichlet distribution with parameters α_b, . Similar to the case of a single stimulus, the update rule for the parameters is given by

$${\alpha }_{b,y}^{\prime }={\alpha }_{b,y}+n_{b,y},y\in \mathcal{Y}.$$

The second case we have to consider is that both mechanism (X = x₁) → Y and (X = x₂) → Y are independent. In order to describe a respective response process, consider two independent Pólya urns both filled with the same kind of colored balls, where we mark the two urns x₁ respectively x₂. In each sampling step we first decide from which urn to sample and then proceed as before. To be consistent with our notation we introduce a binning scheme B₂ with two separate bins b₁ = {x₁} and b₂ = {x₂}. Instead of assigning weights to the urns directly, we assign them to our bins, i.e. α_b1, for the first bin and α_b2, for the second one. The respective response is given by the product measure

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B_2\right]=\frac{\text{Beta}({\mathit{n}}_{b_1,\mathcal{Y}}+{\mathit{\alpha }}_{b_1,\mathcal{Y}})}{\text{Beta}({{\mathit{\alpha }}_{b_1,}}_{\mathcal{Y}})}\frac{\text{Beta}({\mathit{n}}_{b_2,\mathcal{Y}}+{\mathit{\alpha }}_{b_2,\mathcal{Y}})}{\text{Beta}({{\mathit{\alpha }}_{b_2,}}_{\mathcal{Y}})}.$$

The so induced prediction rule is

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n,B_2)=\frac{n_{I_{B_2}(x),y}+{\alpha }_{I_{B_2}(x),y}}{n_{I_{B_2}(x)}+{\alpha }_{I_{B_2}(x)}},$$ (6)

where x ∈ and I_B2(x) tells us the bin to which x is assigned. This response process is partially exchangeable and the respective de Finetti measure μ(·|B₂) is a product measure on ${\mathrm{\Delta }}_{\mathcal{Y}}^{\mathcal{X}}$, where p_x1, ~ Dir[α_b1, ] and p_x2, ~ Dir[α_b2, ] are independently distributed, i.e. the respective density is given by the product density

$$f({\mathit{p}}_{\mathcal{X},\mathcal{Y}})=\frac{1}{\text{Beta}({\mathit{\alpha }}_{b_1,\mathcal{Y}})}\prod _{y\in \mathcal{Y}}{p}_{x_1,y}^{{\alpha }_{b_1,y}-1}\times \frac{1}{\text{Beta}({\mathit{\alpha }}_{b_2,\mathcal{Y}})}\prod _{y\in \mathcal{Y}}{p}_{x_2,y}^{{\alpha }_{b_2,y}-1}.$$

Conditional on finite data ${\mathit{Y}}_{{\mathit{n}}_{\mathcal{X},\mathcal{Y}}}^{\mathcal{X}}$, the respectively updated parameters ${\mathit{\alpha }}_{B,\mathcal{Y}}^{\prime }$ are

$${\alpha }_{b_i,y}^{\prime }={\alpha }_{b_i,y}+n_{b_i,y},y\in \mathcal{Y},i\in \{1,2\}.$$

We can utilize both schemes B₁ and B₂ to model our beliefs about the similarity and dissimilarity between the stimuli. If B₁ represents the hypothesis that x₁ ~ x₂, then B₂ is the alternative hypothesis that x₁ ≁ x₂. If we assess our a priori belief in B₁ with a probability of P [B₁], then our a priori belief for model B₂ is P [B₂] = 1 − P [B₁]. Our overall prediction for the response process is then given by the mixture measure

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B_1\right]\mathbf{P}[B_1]+\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B_2\right]\mathbf{P}[B_2].$$ (7)

The induced prediction rule is

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n)=\frac{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y\right]}{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]},$$

which can be rewritten as

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n)=\sum _{i=1}^2{\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n,B_i)\mathbf{P}\left[B_i\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right],$$ (8)

where

$$\mathbf{P}\left[B_i\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\frac{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B_i\right]}{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]}\mathbf{P}[B_i],i\in \{1,2\}.$$ (9)

The latter probability measure $\mathbf{P}\left[B_i\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]$ can be interpreted as the Bayesian posterior assessment for each binning scheme B_i. The respective de Finetti measure for p is given by

$$\mu (·)=\mathbf{P}[B_1]\mu (·\mid B_1)+\mathbf{P}[B_2]\mu (·\mid B_2),$$

which can be seen to be a model average over B₁ and B₂. One of our major goals is to generalize this mixture to multiple stimuli and responses, for which we want to develop adaptive sequential sampling strategies.

3.3. Multiple stimuli

Here, we briefly discuss the multibin Pólya process on discrete finite stimulus spaces = {x₁, …, x_L}, where we introduce an equivalence relation ~_B that describes similarities in . This relation induces a partition B of into |B| bins, called a multibin. We then bin the data with respect to the resulting multibin B = {b₁, …, b_|B|}, such that we get the binned count statistic n_B, = (n_b, )_b∈B. In full analogy to the scenario of two stimuli, we set pseudo counts α_B, = (α_b, )_b∈B and fix the multibin Pólya urn process Y with

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B\right]=\prod _{b\in B}\frac{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}.$$

The induced prediction rule is

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n,B)=\frac{n_{I_B(x),y}+{\alpha }_{I_B(x),y}}{n_{I_B(x)}+{\alpha }_{I_B(x)}},$$

where I_B (x) denotes the bin of stimulus x given the multibin B, i.e. if x ∈ b then I_B (x) = b. Likewise the above case of two separate stimuli, the respective de Finetti measure μ(·|B) can be informally described by the density

$$f({\mathit{p}}_{\mathcal{X},\mathcal{Y}})=\prod _{b\in B}\frac{1}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}\prod _{y\in \mathcal{Y}}{p}_{b,y}^{{\alpha }_{b,y}-1}\prod _{x\in b}\delta ({\mathit{p}}_{b,\mathcal{Y}}-{\mathit{p}}_{x,\mathcal{Y}}),$$

such that conditional on some finite data ${\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}$ the relevant parameters α_B, are simply updated according to

$${\alpha }_{b,y}^{\prime }={\alpha }_{b,y}+n_{b,y},b\in B,y\in \mathcal{Y}.$$

3.4. A Hierachical Bayesian Model for Proximally Related Stimuli

We assumed that the stimulus space = {x₁, x₂, …, x_L} exhibits some well-ordering, as indicated by the indexing. We need to introduce some suitable terminology. Let ( ) = {b_i,j = {x_i, x_i+1 …, x_j }⊆ | i ≤ j} denote the class of consecutive bins, where we call each partition B of a proximal multibin if it consists only of consecutive bins. The class of all proximal multibins with m bins is denoted ( ), such that $\mathcal{P}(\mathcal{X})={\cup }_{m=1}^L{\mathcal{P}}_m(\mathcal{X})$ constitutes the class of all proximal multibins. These definitions are illustrated by the following example.

Example 4

Consider a set of stimuli = {1, 2, 3}, such that

$$\begin{array}{l}\mathcal{C}(\mathcal{X})=\{b_{1,1},b_{2,2},b_{3,3},b_{1,2},b_{2,3},b_{1,3}\}\\ =\{\{1\},\{2\},\{3\},\{1,2\},\{2,3\},\{1,2,3\}\}.\end{array}$$

Furthermore, ( ) = {B₁, B₂, B₃, B₄}, where

$$\begin{array}{ll}{B}_1=\{b_{1,1},b_{2,2},b_{3,3}\},\hfill & B_2=\{b_{1,2},b_{3,3}\},\hfill \\ B_3=\{b_{1,1},b_{2,3}\},\hfill & B_4=\{b_{1,3}\}.\hfill \end{array}$$

For each consecutive bin b ∈ we choose pseudo counts α_b, and construct the following response process: By fixing a priori beliefs P [B], B ∈ ( ), we can describe the full response process Y as the mixture

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\sum _{B\in \mathcal{P}(\mathcal{X})}\mathbf{P}[B]\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B\right]=\sum _{B\in \mathcal{P}(\mathcal{X})}\mathbf{P}[B]\prod _{b\in B}\frac{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}.$$ (10)

We shall refer to this process as the multibin Pólya mixture process. The induced prediction rule is

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n)=\frac{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y\right]}{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]},$$ (11)

which can be rewritten as

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n)=\sum _{B\in \mathcal{P}(\mathcal{X})}{\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n,B)\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right],$$ (12)

where

$$\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\frac{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\mid B\right]}{\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]}\mathbf{P}[B],B\in \mathcal{P}(\mathcal{X}),$$ (13)

is the posterior for the multibins B ∈ ( ). The Bayesian model that corresponds to the multibin Pólya mixture process is described by the hierarchical model

$$\begin{array}{l}B~\mathbf{P}[·]\\ {\mathit{p}}_{\mathcal{X},\mathcal{Y}}~\mu (·\mid B)\\ {\mathit{Y}}^{\mathcal{X}}~\text{Mult}[{\mathit{p}}_{\mathcal{X},\mathcal{Y}}].\end{array}$$ (14)

There are many ways how to look at the so described hierarchical model. For instance, the model can be seen as a Bayesian regression model for the psychometric rates, where a non-trivial prior in form of

$$\mu (·)=\sum _{B\in \mathcal{P}(\mathcal{X})}\mu (·\mid B)\mathbf{P}[B]$$

is chosen. This prior assigns probability mass on particular diagonals of the product simplex, such that the psychometric rates become piecewise constant w.r.t. to a particular multibin, which are in turn assumed to be random quantities. From that point of view, the described model can be seen to be a generalization of the inhomogenous Bernoulli process described in Endres et al. (2008). Likewise, the model can be interpreted as a particular clustering model. Observations are clustered with respect to an unknown partition of the data space. From this point of view, the model is related to the so called product partition model as introduced by Hartigan (1990), which requires a particular prior structure for P [B] that will be discussed next.

3.5. Efficient Model Evaluation

The actual computation of the sum-product Σ_{B∈ ( )}Π_b∈B in equation (10) can become computationally very demanding and infeasible as it may take up to (2^L−1 ) steps. This is especially problematic for adaptive sampling where the relevant quantities have to be computed as quickly as possible. However, based on the computational approaches taken by Yao (1984); Barry and Hartigan (1992); Endres and Földiák (2005); Fernhead (2006); Hutter (2007) it can be shown that particular prior structures of P [B] lead to a drastic reduction of the computational effort. In fact, it can be reduced to (L³) if

$$\mathbf{P}[B]=\frac{1}{c(\mathit{\beta },\mathit{\gamma })}{\beta }_{\mid B\mid }\prod _{b\in B}{\gamma }_b,$$ (15)

where |B| is the number of bins in B. The respective computational algorithm is given in lemma 1 (see appendix) to which we refer as Proximal Multi-Bin Summation (ProMBS). It is an abstracted and generalized version of the algorithm presented in Endres and Földiák (2005). The parameters γ = (γ_b)_{b∈ ( )}, with γ_b ≥ 0, assess the a priori importance of each consecutive bin b ∈ ( ) and have been also called cohesions in Barry and Hartigan (1992), whereas β = (β₁, …, β_L), with β_l ≥ 0, determine a relative weight for each class ( ), m = 1, …, L, in ( ).

Given a set of parameters (α, β, γ) a particular multibin Pólya mixture process is fixed that describes our a priori belief about the response process Y. All relevant posterior quantities are determined by the updated parameters

$$\begin{array}{l}{\alpha }_{b,y}^{\prime }={\alpha }_{b,y}+n_{b,y}\\ {\beta }_m^{\prime }=\frac{{\beta }_m}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}\\ {\gamma }_b^{\prime }={\gamma }_b\frac{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}\end{array}$$

where

$$d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}}):=\sum _{B\in \mathcal{P}(\mathcal{X})}{\beta }_{\mid B\mid }\prod _{b\in B}{\gamma }_b\frac{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}.$$

The ProMBS algorithm can be used to efficiently compute d(α, β, γ, n), which will reappear in many other relevant expressions. For instance, we can rewrite equation (10) as

$$\mathbf{P}\left[{\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\frac{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}{c(\mathit{\beta },\mathit{\gamma })},$$

whereas the normalization constant in (15) is given by

$$c(\mathit{\beta },\mathit{\gamma })=d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },0),$$

such that the prediction rule in equation (11) becomes

$${\pi }_n^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n)=\frac{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}}^{+(x,y)})}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})},$$

where ${\mathit{n}}_{\mathcal{X},\mathcal{Y}}^{+(x,y)}$ is the count statistic n incremented by one count for the event {X = x, Y = y}. Likewise, given a stimulus x ∈ , the marginal posterior density of the respective psychometric rate p_x, is given by

$$f({\mathit{p}}_{x,\mathcal{Y}}\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}})=\frac{d(\mathit{\alpha },\mathit{\beta },\stackrel{\sim }{\mathit{\gamma }}({\mathit{p}}_{x,\mathcal{Y}}),{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})},$$

where

$${\stackrel{\sim }{\gamma }}_b({\mathit{p}}_{x,\mathcal{Y}})=\{\begin{array}{ll}{\gamma }_b\frac{{\displaystyle \prod _{y\in \mathcal{Y}}}{p}_{x,y}^{n_{b,y}+{\alpha }_{b,y}-1}}{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}\hfill & \text{if}x\in b\hfill \\ {\gamma }_b\hfill & \text{otherwise}\hfill \end{array}.$$

The pointwise evaluation of this density can get computationally very expensive, but is nevertheless possible without Monte Carlo methods. Alternatively, the density $f({\mathit{p}}_{x,\mathcal{Y}}\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}})$ can be described by its moments. Here, the k-th raw moment is

$$\mathbb{E}\left[{(p_{x,y})}^k\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\frac{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}}^{+k(x,y)})}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})},$$

where we add k events {X = x, Y = y} to the count statistic n. We can also compute the posterior for the multibins (13)

$$\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\frac{{\beta }_{\mid B\mid }{\displaystyle \prod _{b\in B}}{\gamma }_b\frac{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})},$$

and by introducing a variable M ∈ {1, …, L} that restricts the model to multibins from ( ), we can compute

$$\mathbf{P}\left[M=m\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]=\frac{{\sum }_{B\in {\mathcal{P}}_m(\mathcal{X})}{\beta }_m{\displaystyle \prod _{b\in B}}{\gamma }_b\frac{\text{Beta}({\mathit{n}}_{b,\mathcal{Y}}+{\mathit{\alpha }}_{b,\mathcal{Y}})}{\text{Beta}({\mathit{\alpha }}_{b,\mathcal{Y}})}}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}$$

which is the posterior assessment that X → Y is described by a multibin model with m bins. Since neighboring stimuli potentially share strength, it is of interest to quantify to which extent this is happening. Such an informative statistic is the effective count n̄_x, which we define as the expectation value

$${\overline{n}}_x({\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}})=\sum _{B\in \mathcal{P}(\mathcal{X})}{n}_{I_B(x)}\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right],$$ (16)

where I_B (x) = b if x ∈ b and b ∈ B. An efficient evaluation is possible because

$${\overline{n}}_x({\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}})=\frac{d(\mathit{\alpha },\mathit{\beta },\stackrel{\sim }{\mathit{\gamma }},{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})},$$

where

$${\stackrel{\sim }{\gamma }}_b=\{\begin{array}{ll}{\gamma }_b{n}_b\hfill & \text{if}x\in b\hfill \\ {\gamma }_b\hfill & \text{otherwise}\hfill \end{array}.$$

3.6. Prior Selection

For the multibin Pólya mixture process we need to select parameters (α, β, γ). With β_m we specify the importance of models consisting of exactly m bins. For instance, if we want to fall back to a simpler problem with at most l < L bins we can set β_l = 1 and β_j = 0 for j ≠ l. A reasonable choice is often given by

$${\beta }_m={\left(\begin{array}{l}L-1\hfill \\ m-1\hfill \end{array}\right)}^{-1},\text{for}m=1,2,\dots ,L.$$

The binomial coefficient gives the number of multibin models that constist of m bins. Hence, if for some constant c all γ_b = c, b ∈ ( ), then the above prior assigns a uniform distribution to P [M = m], the a priori probability of multibins with m bins. In any case, there are only L values that we have to specify for β. However, for α_b, and γ_b we need to select values for each b ∈ ( ). If we are dealing with many stimuli the assignment of parameters can get a very extensive task. A first and very convenient choice is to set all α_b, and γ_b to one, which means that we use an uninformative prior for the pseudo counts and we do not have any preference for specific bins. A more elaborate and natural way is to compute α_b, in a hierarchical manner, i.e. if we require that

$${\mathit{\alpha }}_{b,\mathcal{Y}}=\frac{1}{\mid b\mid }\sum _{x\in b}{\mathit{\alpha }}_{x,\mathcal{Y}},$$

then we only need to specify α_x, for each x ∈ .

3.7. Placement Rules for Adaptive Sampling

We discussed in section 2.1 a utility based approach concerning the choice of a suitable action policy for choosing the placement rule ${\pi }_n^X$. We shall assume that our objective is to become most informed about the stimulus-response relation modeled by the multibin Pólya mixture process. We present here several proposals for how this might be achieved.

As already mention earlier, a most trivial sampling scheme is to distribute measurements uniformly, where the respective utility is

$$U_{n+1}(x,{\mathit{E}}_n)=-n_x,$$ (17)

which guarantees at least some homogeneity in the data, but does not take any properties of the underlying model into account. A more sensible utility is given by

$$U_{n+1}(x,{\mathit{E}}_n)=-{\overline{n}}_x,$$

where n̄_x is the effective count. The resulting placement rule essentially follows the logic of random uniform sampling, but takes the sharing of strength between neighboring stimuli into account. The attractive feature of this adaptive scheme is of course its conceptual and computational simplicity.

More elaborate adaptive strategies can be obtained by considering utilities based on information-theoretical quantities. The idea of using information theoretic utility measures for experimental designs was probably first considered by Cronbach (1953). A more detailed study of their application in experimental designs was later given by Lindley (1956), whereas explications of Lindley’s ideas within the context of Bayesian experimental design can be found in Bernardo (1979); Chaloner and Verdinelli (1995), but see also the application for optimizing sequential experimental designs in DeGroot (1962).

Consider that we want to learn as much as possible about which multibin model B ∈ ( ) describes the data best. Our a priori expectation is given by the probability P [B], whereas our a posteriori assessment is expressed by $\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]$. The information we would gain about B if we learn that { ${\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}$} is then quantified by the Kullback-Leiber divergence⁴

$${\text{D}}_{\text{KL}}\left(\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]\left|\right|\mathbf{P}[B]\right)=-\sum _{B\in \mathcal{P}(\mathcal{X})}\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]log\left(\frac{\mathbf{P}[B]}{\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}={\mathit{y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]}\right),$$

which measures the deviation between both measures. Hence, if we want to learn as much as possible about B then it seems reasonable to adopt the Kullback-Leibler divergence as a global utility measure. This, however, leads to almost intractable computations, as mentioned already in section 2.1. Nonetheless, it motivates the following myopic adaptive sampling strategy with local utility measure

$$u_{n+1}^{\text{MB}}(x,y,{\mathit{E}}_n)={\text{D}}_{\text{KL}}(\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y\right]||\mathbf{P}\left[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}\right]),$$

i.e. we try to improve the incremental information in every sampling step. This divergence can be computed with

$$u_{n+1}^{\text{MB}}(x,y,{\mathit{E}}_n)=\frac{d(\mathit{\alpha },\mathit{\beta },\stackrel{\sim }{\mathit{\gamma }},{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})},$$

where

$${\stackrel{\sim }{\gamma }}_b=\{\begin{array}{ll}{\gamma }_{b}ln\left(\frac{n_{b,y}+{\alpha }_{b,y}}{n_b+{\alpha }_b}\frac{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}}^{+(x,y)})}\right)\hfill & \text{if}x\in b\hfill \\ {\gamma }_b,\hfill & \text{otherwise}\text{.}\hfill \end{array}$$

The respective expected local utility is

$$U_{n+1}^{\text{MB}}(x,{\mathit{E}}_n)=\sum _{y\in \mathcal{Y}}{u}_{n+1}^{\text{MB}}(x,y,{\mathit{E}}_n){\pi }_{n+1}^Y(Y_{n+1}=y\mid X_{n+1}=x,{\mathit{E}}_n),$$

such that the optimal stimulus is determined by

$$x_{n+1}^{\ast }({\mathit{E}}_n)=\underset{x\in \mathcal{X}}{argmax}{U}_{n+1}^{\text{MB}}(x,{\mathit{E}}_n).$$

It is noteworthy that the expected gain can be rewritten in terms of a mutual information⁵, namely

$$U_{n+1}^{\mathbf{MB}}(x,{\mathit{E}}_n)=\text{MI}(B:Y_{n_x+1}^x\mid Y_{n_{\mathcal{X}}}^{\mathcal{X}}),$$

such that by maximizing the expected local utility we do in fact maximize the mutual information between B and $Y_{n_x+1}^x$ conditional on ${\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}$. The so obtained adaptive sampling scheme, shortly denoted u^MB, is fully equivalent to the Bayesian framework of adaptive design optimization (ADO) for model discrimination (cf. Cavagnaro et al. (2010)).

The very same line of reasoning applies to the case where we do have strong evidence for a particular multibin model, say B ∈ ( ), but want to optimally learn the psychometric rates p. The respective local utility measures are given by

$$u_{n+1}^{\mathrm{\Psi }\mid \text{B}}(x,y,{\mathit{E}}_n)={\text{D}}_{\text{KL}}(\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y,B]||\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},B]),$$

where $\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},B\mid ]$ and $\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y,B]$ are the respective successive posterior measures for the psychometric rates p conditional on B. These utility measures can be computed with

$$u_{n+1}^{\mathrm{\Psi }\mid \text{B}}(x,y,{\mathit{E}}_n)=\frac{d(\mathit{\alpha },\mathit{\beta },\stackrel{\sim }{\mathit{\gamma }},{\mathit{n}}_{\mathcal{X},\mathcal{Y}})}{d(\mathit{\alpha },\mathit{\beta },\mathit{\gamma },{\mathit{n}}_{\mathcal{X},\mathcal{Y}})},$$

where

$${\stackrel{\sim }{\gamma }}_b=\{\begin{array}{ll}{\gamma }_b\left(-ln\left(\frac{n_{b,y}+{\alpha }_{b,y}}{n_b+{\alpha }_b}\right)+\psi (n_{b,x}+{\alpha }_{b,x}+1)-\psi (n_b+{\alpha }_b+1)\right)\hfill & \text{if}x\in b\hfill \\ {\gamma }_b\hfill & \text{otherwise}\hfill \end{array},$$

and ψ is the psigamma function. We expect the resulting adaptive sampling scheme, denoted u^Ψ|B, to optimize the inferential task for the psychometric rate p given that B is the ‘true’ underlying model.

In practice, we usually have only vague knowledge about the underlying multi-bin model. Henceforth, it seems reasonable to consider the local utility measures

$$u_{n+1}^{\text{Total}}(x,y,{\mathit{E}}_n)={\text{D}}_{\text{KL}}(\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y]||\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}]),$$

where $\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}]$ and $\mu [·\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y]$ are the respective successive posterior measures for the psychometric rates p. This measure decomposes as

$$u_{n+1}^{\text{Total}}(x,y,{\mathit{E}}_n)=u_{n+1}^{\text{MB}}(x,y,{\mathit{E}}_n)+u_{n+1}^{\mathrm{\Psi }}(x,y,{\mathit{E}}_n),$$

where

$$u_{n+1}^{\mathrm{\Psi }}(x,y,{\mathit{E}}_n):=\sum _{E\in \mathcal{P}(\mathcal{X})}P[B\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}},Y_{n_x+1}^x=y]u_{n+1}^{\mathrm{\Psi }\mid \text{B}}(x,y,{\mathit{E}}_n)$$

is the model-averaged local utility for optimizing the inference of the psychometric rates given a multibin model. Hence, if we want to learn about p given that B is uncertain, then we also have to make inference about B. That is the uncertainty about B also influences our uncertainty about p. We thus expect this strategy, called u^Total, to optimize the inference of B and p. It might also be worthwile to base the sampling process solely on u^Ψ because it allows us to optimize the inference of the psychometric rates regardless of the underlying multibin model.

The actual effect of the proposed adaptive sampling strategies are difficult to describe and we shall proceed by discussing a simple example. In general, it can be said that there is no such thing as a universal criterium for optimal adaptive sampling as long as we do not clearly formulate what we want to achieve. Whether or not a chosen strategy is appropriate for the experiment at hand must be carefully assessed from case to case, for example with simulation studies or experimental pre-studies.

4. A Practical Demonstration

In order to demonstrate our framework we consider a hypothetical 2AFC experiment with 35 stimuli = {x₁, x₂, …, x₃₅} and a dichotomous outcome = {s, f }. The stimulus-response relation (ground truth) is given by an asymmetric U-shaped function with a small irregularity on the right side (see Figure 2). We assume that we have no a priori knowledge about the curve, except that proximal stimuli are likely to cause similar responses. In a naïve approach we would simply distribute N measurements random uniformly over , see (17), and infer the psychometric rates for each x ∈ independently. Figure 3 shows the result of such an experiment with 200 and 400 samples, where we chose pseudo counts α_x, = (1, 1), y ∈ . The variance of the estimate is large because there are only very few samples for each stimulus x. A highly irregular curve as an estimate for the stimulus-response relation is therefore obtained and many measurements are taken at regions that are quite uninformative. This motivates two advantages of our proposed framework: First, we can use samples from neighboring stimuli to share statistical strength. Second, we want to distribute the samples such that we maximize the information that we gain with each measurement.

Figure 2.

Hypothetical stimulus-response curve used as ground truth for the practical demonstration. The curve shows the probability of the response s (success) for a given stimulus x.

Figure 3.

A hypothetical experiment with (a) 200 and (b) 400 samples uniformly distributed over the stimulus space . The stimulus-response relation (ground truth), shown as dashed line, is inferred without taking information from proximal x into account. The thick continuous line shows the first moment $\mathbb{E}[p_{x,s}\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}}]$ of the stimulus-response function given all outcomes of previous measurements. The standard deviation is shown as a thin continuous line around the expectation. The marginal posterior density $f(p_{x,s}\mid {\mathit{Y}}_{{\mathit{n}}_{\mathcal{X}}}^{\mathcal{X}})$ is plotted as shadings in the back of the figure. The number of measurements at each x ∈ is shown as a bar plot in the lower plot with utility U (x) (thin continuous line), which is the negative number of counts, see (17).

With our framework we can optimize the experimental process. We choose the following parameters:

$${\mathit{\alpha }}_{x,\mathcal{Y}}=(1,1),{\beta }_m{\left(\begin{array}{l}L-1\hfill \\ m-1\hfill \end{array}\right)}^{-1},{\gamma }_b=1\text{for}b\in \mathcal{C}(\mathcal{X})\text{and}m=1,2,\dots ,L$$

With α_x, = (1, 1) we assign equal pseudo counts to all stimuli and responses, since we have no a priori knowledge about the shape of the curve. The particular choice of β_m and γ_b assigns a uniform distributions to P [M = m], which is the a priori probability of multibins with m bins (see section 3.6). Finally, we set all γ_b = 1 since all bins should receive an equal weight.

With this prior setting we observe a smoothening of the inferred stimulus-response curve (see Figure 4). However, in contrast to common kernel smoothing approaches, our method does not smear sharp transitions but represents a higher degree of uncertainty whenever necessary.

Figure 4.

A hypothetical experiment with (a) 200 and (b) 400 samples uniformly distributed. The ground truth is inferred by taking information from neighboring x into account. This sharing of statistical strength allows a much more accurate inference of the stimulus-response function as compared to Figure 2.

In order to distribute samples more efficiently, we can utilize the adaptive sampling strategies described in section 3.7. Unfortunately, it is difficult to compare the performance of different strategies since there is no general criterion for optimality. A good intuition can however be gained from the distribution of measurements.

The Figures 5, 6, 7, and 8 show the time course of the experiment for the adaptive sampling schemes u^Ψ, u^MB, u^Total, and the strategy based on the effective count n̄_x. All simulations were initialized with the same random seeds for each stimulus x ∈ , such that results can be better compared. The uninformative parameter setting lead to a uniformly distributed expected local utility for all four strategies before the first experiment. Therefore, the first measurement was always taken at x = 20.

Figure 5.

Adaptive sampling in a hypothetical experiment with scheme u^Ψ. The figure shows the experiment after 1, 2, 10, 50, 100, and 200 samples. At first, samples are uniformly distributed. In (a) only one measurement was taken and the expected utility is largest at the left boundary. (d) shows the experiment after 50 samples where the algorithm starts to locate measurements at sloped regions. The general shape of the stimulus-response function is already well established after 100 samples (e). Many measurements are also taken at x = 1 and x = 35 since those x have only one neighbor.

Figure 6.

Adaptive sampling in a hypothetical experiment with scheme u^MB. The figure shows the experiment after 1, 2, 10, 50, 100, and 200 samples. After the first sample (a) two maxima of the expected utility arise next to the measurement. The first 10 samples (c) are allocated near the initial measurement, whereas afterwards the algorithm starts move further right (d) until almost the full stimulus space has been explored (e–f).

Figure 7.

Adaptive sampling in a hypothetical experiment with scheme u^Total. The figure shows the experiment after 1, 2, 10, 50, 100, and 200 samples. The sampling behavior clearly shows a mixture of both schemes u^MB and u^Ψ.

Figure 8.

Adaptive sampling in a hypothetical experiment with the strategy based on the effective count n̄_x. The figure shows the experiment after 1, 2, 10, 50, 100, and 200 samples. In (a) no measurements are taken and we can see our prior expectation. The respective utility is uniformly distributed on . The general shape of the stimulus-response function is already well established after 100 samples. Measurements are mostly allocated at regions where the slope of the stimulus-response function is high. Many measurements are also taken at x = 1 and x = 35 since those x have only one neighbor.

Already after the first sample one can observe a striking difference between u^Ψ and u^MB. Whereas the expected utility for u^Ψ is maximal at the very left stimulus x = 1, the expected utility for u^MB shows two maxima directly next to the previous measurement at x = 20. This reveals the very distinct properties of both schemes, which are better seen after 10 and 50 samples. u^Ψ causes a uniform distribution of the first samples on . On the other hand, the scheme u^MB places all measurements around the initial sample and only gradually moves further away from x = 20. The U-shape of the stimulus-response function is already very well established after 100 samples and after 200 samples one can see that both measures place most samples at positions where the psychometric function is highly sloped. This behavior is expected since those regions allow less sharing of statistical strength. For the scheme u^Total we observe the same initial behavior as for u^Ψ, but the count statistic after 200 samples shows characteristics of both u^Ψ and u^MB, which is expected from its definition. It is however quite noteworthy that the effective counts show a very similar behavior as u^Ψ. Figure 9 summarizes the differences between the four adaptive sampling strategies by showing the experiment after 200 samples.

Figure 9.

Direct comparison of the four adaptive sampling strategies: (a) u^Ψ, (b) u^MB, (c) u^Total, and (d) effective counts. A relatively balanced distributions of measurements is observed with u^Ψ and the effective counts. On the other hand, u^MB and therefore also u^Total lead to a more peaked distribution.

Note also that many measurements are taken at the boundaries. This is because these stimuli have only one neighbor which limits the extent to which statistical strength can be shared. We now assume that we have prior knowledge about the ground truth, i.e. we have a strong belief about its value at x₁ and x₃₅. We incorporate this information into our model and thereby optimize the sampling process further. That is, we set α_1, = α_35, = (100, 1). This parameter setting alters our prior expectation about the stimulus-response relation, i.e. p_x,s is expected to be close to one at x₁ and x₃₅, see Figure 10(a). The expected utility to sample at these points is substantially reduced, since we have a strong belief about the response. Figure 10(b) shows the experiment after 200 samples. One can see that all measurements that were previously allocated at the boundaries are now distributed elsewhere. Further optimizations of the sampling process are possible if more prior knowledge is available.

Figure 10.

Demonstration of using prior knowledge to limit sampling at the boundaries with scheme u^Ψ. (a) shows the prior belief before any measurements are made and (b) shows the posterior after 200 samples. The prior setting leads to a strong belief about the response at the boundaries which substantially reduces the expected utility at x₁ and x₃₅. The result is that all measurements that were previously allocated at the boundaries are now placed elsewhere.

5. Conclusion

We have introduced a framework for adaptive sequential sampling which helps to optimize the measurement process in a wide range of psychophysical experiments, especially when there is only vague prior knowledge about the relation X → Y. Our framework consists of two major components. The first is a response process that we use to make predictions based on a finite number of observations. We termed it the multibin Pólya mixture process as it consists of a mixture of binned Pólya urns. On top of the response process we defined various adaptive sampling process, which are equipped with utility measures to actively guide the course of an experiment. We also demonstrate the effect of several sampling strategies on a hypothetical experiment and how prior knowledge can be used to further optimize the allocation of measurements. During experiments it is of great importance that decisions for the next stimulus are computed fast. Although our model is computationally demanding, we provide an algorithm that makes it applicable in typical psychophysics experiments.

Highlight.

• We present a predictive account on adaptive sampling in psychophysical experiments.

• Our method applies to situations where there is only weak knowledge available.

• We demonstrate the advantages of our framework on a hypothetical sampling problem.

Appendix

In Yao (1984); Barry and Hartigan (1992); Endres and Földiák (2005); Fernhead (2006); Hutter (2007) various algorithms are presented, which in their given context address all the very same computational problem. Within our terminology of proximal multibins the general problem can be described as follows:

Let f: C( ) → ℝ be any function from consecutive proximal bins to real numbers and g = (g₁, …, g_L) ∈ ℝ^L be any real valued vector. Define the sum-products

$$S_m[f]:=\sum _{B\in {\mathcal{P}}_m(\mathcal{X})}\prod _{b\in B}f(b),m=1,\dots ,L,$$

and

$$S[f,\mathit{g}]:=\sum _{B\in \mathcal{P}(\mathcal{X})}{g}_{\mid B\mid }\prod _{b\in B}f(b).$$

Each S_m[f ] consists of $\left(\begin{array}{l}L-1\hfill \\ m-1\hfill \end{array}\right)$ terms, whereas the weighted total sum-product S[f, g] consists of 2^L−1 terms. For large L a computationally intractable effort of (2^L−1) is expected. However, the following lemma provides a simple and efficient algorithm, which computes the sum-product in (L³).

Lemma 1 (The Proximal Multibin Summation (ProMBS) Algorithm)

Define the upper triangular matrices $A_l={(a_{i,j}^l)}_{L\times L}$, l = 1, …, L, recursively by

$$a_{i,j}^1=\{\begin{array}{ll}f(b_{i,j})\hfill & \text{if}i\le j;\hfill \\ 0\hfill & \text{otherwise};\hfill \end{array}\text{and}{a}_{i,j}^l=\{\begin{array}{ll}{a}_{i-1,j-1}^{l-1}\hfill & i,j>1;\hfill \\ 1\hfill & i=j=1;\hfill \\ 0\hfill & \text{else};\hfill \end{array}$$

and define recursively the matrix-vector product

$${\mathit{v}}^{l+1}={(v_1^{l+1},\dots ,v_L^{l+1})}^T=A_l\times {\mathit{v}}^l,l=1,\dots L,$$

where the initial vector is v¹ = (0, …, 0, 1)^T. For every m, l ∈ ℕ with m ≤ l ≤ L it holds that

$$v_m^{l+1}=S_m[f],$$

such that

$$S[f,\mathit{g}]=\sum _{m=1}^L{g}_m{S}_m[f].$$

A simple example might help to illustrate the abstract formulation of the ProMBS algorithm. Consider the case with | | = 3 and let us abbreviate f (b_i,j ) simply by f_ij. In the first step we compute

$${\mathit{v}}^2=A_1{\mathit{v}}^1=\left(\begin{array}{ccc}{f}_{11}& f_{12}& f_{13}\\ 0& f_{22}& f_{23}\\ 0& 0& f_{33}\end{array}\right)\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right)=\left(\begin{array}{c}{f}_{13}\\ f_{23}\\ f_{33}\end{array}\right).$$

Note that $v_1^2=f_{13}=S_1[f]$. In the next step we compute

$${\mathit{v}}^3=A_2{\mathit{v}}^2=\left(\begin{array}{ccc}1& 0& 0\\ 0& f_{11}& f_{12}\\ 0& 0& f_{22}\end{array}\right)\left(\begin{array}{c}{f}_{13}\\ f_{23}\\ f_{33}\end{array}\right)=\left(\begin{array}{c}{f}_{13}\\ f_{11}{f}_{23}+f_{12}{f}_{33}\\ f_{22}{f}_{33},\end{array}\right)$$

where $v_1^3=f_{13}=S_1[f]$ and $v_2^3=f_{11}{f}_{23}+f_{12}{f}_{33}=S_2[f]$. In the last step we find that

$${\mathit{v}}^4=A_3{\mathit{v}}^3=\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& f_{11}\end{array}\right)\left(\begin{array}{c}{f}_{13}\\ f_{11}{f}_{23}+f_{12}{f}_{33}\\ f_{22}{f}_{33}\end{array}\right)=\left(\begin{array}{c}{f}_{13}\\ f_{11}{f}_{23}+f_{12}{f}_{33}\\ f_{11}{f}_{22}{f}_{33}\end{array}\right),$$

such that (v⁴)^T = (S₁[f ], S₂[f ], S₃[f ]) and S[f, g] = g₁S₁[f ]+ g₂S₂[f ]+ g₃S₃[f ].

Of course, the real computational power of the ProMBS algorithm becomes only evident when the set is large, but the case | | = 3 shows the general logic behind the ProMBS algorithm sufficiently enough. We should finally mention that numerical imprecisions can occur if the values of f become small. In such cases it is advisable to implement the ProMBS algorithm on a logarithmic number scale.