A martingale analysis of first passage times of time-dependent Wiener diffusion models

Abstract

Research in psychology and neuroscience has successfully modeled decision making as a process of noisy evidence accumulation to a decision bound. While there are several variants and implementations of this idea, the majority of these models make use of a noisy accumulation between two absorbing boundaries. A common assumption of these models is that decision parameters, e.g., the rate of accumulation (drift rate), remain fixed over the course of a decision, allowing the derivation of analytic formulas for the probabilities of hitting the upper or lower decision threshold, and the mean decision time. There is reason to believe, however, that many types of behavior would be better described by a model in which the parameters were allowed to vary over the course of the decision process.

In this paper, we use martingale theory to derive formulas for the mean decision time, hitting probabilities, and first passage time (FPT) densities of a Wiener process with time-varying drift between two time-varying absorbing boundaries. This model was first studied by Ratcliff (1980) in the two-stage form, and here we consider the same model for an arbitrary number of stages (i.e. intervals of time during which parameters are constant). Our calculations enable direct computation of mean decision times and hitting probabilities for the associated multistage process. We also provide a review of how martingale theory may be used to analyze similar models employing Wiener processes by re-deriving some classical results. In concert with a variety of numerical tools already available, the current derivations should encourage mathematical analysis of more complex models of decision making with time-varying evidence.

1. Introduction

Continuous time stochastic processes modeling a particle's diffusion (with drift) towards one of two absorbing boundaries have been used in a wide variety of applications including statistical physics (Farkas and Fulop, 2001), finance (Lin, 1998), economics (Webb, 2015), and health science (Horrocks and Thompson, 2004). Varieties of such models have also been applied extensively within psychology and neuroscience to describe both the behavior and neural activity associated with decision processes involved in perception, memory, attention, and cognitive control (Heath, 1992; Ratcliff and Rouder, 1998; Ratcliff and McKoon, 2008; Simen et al., 2009; Gold and Shadlen, 2001, 2007; Brunton et al., 2013; Feng et al., 2009; Shadlen and Newsome, 2001; Diederich and Oswald, 2014, 2016); for reviews see (Ratcliff and Smith, 2004; Bogacz et al., 2006; Busemeyer and Diederich, 2010).

In these stochastic accumulation decision models, the state variable x(t) is thought to represent the amount of accumulated noisy evidence at time t for decisions represented by the two absorbing boundaries, that we refer to as the upper (+) and lower (–) thresholds (boundaries). The evidence x(t) evolves in time according to a biased random walk with Gaussian increments, which may be written as dx(t) ~ Normal(μ dt, σ² dt), and a decision is said to be made at the random time τ, the smallest time t for which x(t) hits either the upper threshold (x(τ) = +ζ) or the lower threshold (x(τ) = −ζ), also known as the first passage time (FPT). The resulting decision dynamics are thus described by the FPT of the underlying model. In studying these processes one is often interested in relating the mean decision time and the probability of hitting a certain threshold (e.g. the probability of making a certain decision) to empirical data. For example, these metrics can offer valuable insight into how actions and cognitive processes might maximize reward rate, which is a simple function of the FPT properties (Bogacz et al., 2006).

However, not all decisions can be properly modeled if parameters are fixed throughout the duration of the decision process. Certain contexts can be better described by a model whose parameters change with time. In this article we analyze the time-dependent version of the Wiener process with drift between two absorbing boundaries, building on recent work that is focused on similar time-varying random walk models (Hubner et al., 2010; Diederich and Oswald, 2014). After reviewing how martingale theory can be used to analyze and re-derive the classical FPT results for the time independent case, we calculate results for the time-dependent case. The main theoretical results are presented in §5.2, where we provide closed form expressions for threshold-hitting probabilities and expected decision times. In Appendix D, we also describe how our methods can be applied to the more general Ornstein-Uhlenbeck (O-U) processes, which are similar to the Wiener diffusion processes albeit with an additional “leak” term. We conclude with a summary of the results and a discussion of how the present work interfaces with other similar analyses of time-varying random walk models.

2. Notation and terminology

Here we introduce the notation and terminology for describing the model we analyze, which is a Wiener process with (time-dependent) piecewise constant parameters. This simple stochastic model, and others close to it, have been studied before (Ratcliff, 1980; Heath, 1992; Smith, 2000; Diederich and Busemeyer, 2003, 2006; Diederich and Oswald, 2014; Bogacz et al., 2006; Wagenmakers et al., 2007), although the reader should note that our parameterization differs from that of some previous studies. Before describing the model, we first review the case where parameters are unchanging with time. In order to easily discuss this simpler model alongside the main time-dependent model analyzed in §5, we use the terms single-stage model and multistage model, respectively. Readers familiar with the popular Diffusion Decision Model of Ratcliff and McKoon (2008) should be aware that parameters in our model do not vary randomly trial-to-trial. Readers familiar with Bogacz et al. (2006) should be aware that the single-stage model (1) is equivalent to what Bogacz et al. (2006) call the “pure” Drift Diffusion Model.

2.1. The single-stage model/process with constant parameters

Consider the stochastic differential equation (SDE):

$$dx\left(t\right)=\mu dt+\sigma dW\left(t\right),x\left(t_0\right)=x_0,$$ (1)

where parameters μ and σ are constants referred to as the drift and diffusion rates, respectively; x₀ is the initial condition (the initial evidence or starting point) of the decision process; and σdW(t) are independent Wiener increments with variance σ²dt. This simple stochastic model has successfully modeled the evolution of evidence between two decisions during a two-alternative forced choice task (Ratcliff and Rouder, 1998; Bogacz et al., 2006; Ratcliff et al., 2016), so that (1) can be interpreted as modeling a decision process in which an agent is integrating noisy evidence until sufficient evidence is gathered in favor of one of the two alternatives.

A decision is made when the evidence x(t) crosses one of the two symmetric decision thresholds ±ζ for the first time, also referred to as its first passage time (FPT). In other words, a decision occurs the instance x(t) crosses and is absorbed by one of the two boundaries. The two boundaries each correspond to one of the two possible decisions for the task. We will refer to the absorbing thresholds at +ζ and −ζ as the upper and lower decision boundaries/thresholds. To contrast with the next section, we will sometimes refer to this model with time-invariant parameters as the single-stage model or single-stage process. We again note that the parameterization used here differs from that employed by others (Smith, 2000; Ratcliff and Smith, 2004; Navarro and Fuss, 2009), although the underlying model is equivalent. Our formulation, compared to some others, does not include a parameter for “non-decision time” or “timeout” for a given trial. Such a term could be incorporated by shifting the entire reaction time distribution – it has no effect on any of our analyses.

2.2. Time-dependent, piecewise constant parameters

The assumption that model parameters remain constant throughout the decision process is unlikely to hold in many situations. For example, the quality of evidence may not be stationary (i.e., the drift rate and diffusion rate are not a constant with respect to time) or decision urgency may require thresholds to collapse in order to force a decision by some deadline (Cisek et al., 2009; Mormann et al., 2010; Zhang et al., 2014; Drugowitsch et al., 2012).

In order to analyze such situations, we focus our present study on a two-stage model originally analyzed by Ratcliff (1980), which we generalize to an arbitrary number of stages. We refer to this slightly generalized model as a multistage model or multistage process, to distinguish it from (1) above. The multistage model allows for the drift rate, diffusion rate, and thresholds to be piecewise constant functions of time.

To fully describe the multistage model, we first partition the set of non-negative real numbers (i.e. time axis) into n intervals (or stages) [t_i–1, t_i], i ∈ {1, . . . , n} with t₀ = 0 and t_n = +∞. We then assume that the drift rate, the diffusion rate, and the decision thresholds are constant within each interval, but that their values change between intervals. Evidence integration is thus modeled by

$$dx\left(t\right)=\mu \left(t\right)dt+\sigma \left(t\right)dW\left(t\right),x\left(t_0\right)=x_0,$$ (2)

where

$$\begin{array}{cc}\mu \left(t\right)={\mu }_i,\hfill & \text{for}{t}_{i-1}\le t<t_i,\hfill \\ \sigma \left(t\right)={\sigma }_i,\hfill & \text{for}{t}_{i-1}\le t<t_i,\hfill \end{array}$$

for each i ∈ {1, . . . , n}. The above assumptions are identical to the assumptions in Diederich and Oswald (2014, 2016). If n = 1, the multistage model reduces to the single-stage model (1). For expository clarity, we begin by assuming the decision thresholds are fixed at ±ζ, and in §6 we generalize to time-varying (piecewise constant) thresholds. Let τ be the first passage (decision) time for the multistage model.

We will frequently refer to the i-th stage of (2), which for t > t_i–1, is written as

$$dx\left(t\right)={\mu }_{i}dt+{\sigma }_{i}dW\left(t\right),x\left(t_{i-1}\right)=X_{i-1},\text{decision thresholds}\pm \zeta ,$$ (3)

where the initial condition X_i–1 is a random variable defined as x(t_i–1) conditioned on there being no decision (threshold-crossing) before time t_i–1. More precisely, the density of X_i–1 is the conditional distribution of x(t_i–1) given that τ > t_i–1. Thus, the random variable X_i–1 corresponds to realizations of the multistage model that remain within the thresholds ±ζ until time t_i–1. For the first stage, X₀ could be a point mass centered at x₀, or it may be a random variable capturing the variability in starting points (Ratcliff and Rouder, 1998). The key difference between (3) and the single-stage process (1) is that the initial condition X_i–1 is a random variable whose distribution is determined by previous stages.

3. Martingale theory applied to the single-stage model

In this section, we give an introduction to the basic properties of martingales and the optional stopping theorem, which are the key mathematical tools used in calculating our main results in §5. For readers who are less familiar with martingale methods, we first derive the mean decision time, hitting probabilities, and FPT densities for the single- and two-stage models. These analyses provide an alternate approach to deriving these classical results as compared to other non-martingale based approaches (Ratcliff, 1980; Diederich and Oswald, 2014). We discuss the differences between these various approaches in §8.

3.1. Continuous time martingales

Consider a continuous time stochastic process η(t), t > 0. Let η(t₁ : t₂) denote the portion of η(t) between times t₁ and t₂. A stochastic process M(t) is said to be a martingale with respect to η(t) if the following three conditions hold:

1. M(t) is a function of η(0 : t) and does not depend on future values of η(t)¹

2. $\mathbb{E}\mid M\left(t\right)\mid <\infty$

3. M(t) satisfies the stationarity condition in expected value

   $$\mathbb{E}[M\left(t_2\right)\mid \eta (0:t_1)]=M\left(t_1\right),\text{for all}0\le t_1<t_2<\infty .$$ (4)

The first condition means that given the realized values η(0 : t), we should be able to compute M(t) deterministically, so that M(t) does not depend on the future. The second condition is a regularity condition that ensures that M(t) is sufficiently² light-tailed and holds under several decision-making scenarios. The third condition is the most crucial – it enforces stationarity in expected value. This last condition can be interpreted as a “fair play” condition ensuring that chances of gaining and losing starting with value M(t₁) at time t₁ are the same, as in the classic example of a sequence of flips of a fair coin. When introducing a martingale, one often does not explicitly specify the process η(t), and in this case η(t) is assumed equal to M(t).

Martingale theory is very broad and there are many different choices for M(t) and η which are interesting. All of the calculations and results of this paper are constructed from two fundamental stochastic processes, which we now introduce. Let W(t) be the standard Wiener process and X(t) = μt + σW(t) be a single-stage Wiener process with drift rate μ and diffusion rate σ (without boundaries). We now consider some martingales associated with these two stochastic processes:

1. W(t) is a martingale. It is easy to verify that for 0 ≤ t₁ < t₂, $\mathbb{E}[W\left(t_2\right)\mid W\left(t_1\right)]=W\left(t_1\right)$. Similarly, X(t) – μt is a martingale.

2. W(t)² – t is a martingale. Note that for 0 ≤ t₁ < t₂, conditioned on W(t₁), W(t₂) has a Gaussian distribution with mean W(t₁) and variance (t₂ – t₁). Therefore,

   $$\mathbb{E}[W{\left(t_2\right)}^2\mid W\left(t_1\right)]=W{\left(t_1\right)}^2+(t_2-t_1)\Rightarrow \mathbb{E}[W{\left(t_2\right)}^2-t_2\mid W\left(t_1\right)]=W{\left(t_1\right)}^2-t_1.$$
3. For any $\lambda \in \mathbb{R}$, exp(λX(t) – λμt – λ²σ²t/2) is a martingale. Note that for 0 ≤ t₁ < t₂, conditioned on X(t₁), X(t₂) has a Gaussian distribution with mean X(t₁) + μ(t₂ – t₁) and variance σ²(t₂ – t₁). Thus,

   $$\begin{array}{cc}\hfill \mathbb{E}[\exp \left(\lambda X\left(t_2\right)\right)\mid X\left(t_1\right)]& =\exp (\lambda (X\left(t_1\right)+\mu (t_2-t_1))+{\lambda }^2{\sigma }^2(t_2-t_1)∕2)\hfill \\ \hfill \mathbb{E}[\exp (\lambda X\left(t_2\right)-\lambda \mu t_2-{\lambda }^2{\sigma }^2{t}_2∕2)\mid X\left(t_1\right)]& =\exp (\lambda X\left(t_1\right)-\lambda \mu t_1-{\lambda }^2{\sigma }^2{t}_1∕2).\hfill \end{array}$$

For λ = −2μ/σ², this martingale reduces to exp(−2μX(t)/σ²) which is referred to as the exponential martingale.

3.2. Stopping times and the optional sampling theorem

The first passage time τ is a random variable defined by τ = inf{t > 0|x(t) ∉ (−ζ, ζ)}. We are interested in computing conditional expectations and probability densities of τ, which correspond to expected decision times and the corresponding distributions of response times. The key tool we borrow from the theory of martingales is a classic result known as Doob's optional sampling theorem (also known as the optional stopping theorem), which we motivate and introduce here. To understand the optional sampling theorem, one must first recall that the expected value of a martingale M(t) computed over all realizations starting from M(0) is equal to the initial expectation of M(0). That is, martingales by definition must satisfy the following:

$$\mathbb{E}\left[M\left(t\right)\right]=\mathbb{E}\left[M\left(0\right)\right],\text{for each}t>0.$$

One then wonders: Does a similar property extend to the random time τ? More specifically, if we consider different realizations of τ and compute averages of M(τ) at these realized values, does this average, as the number of realizations grow large, converge to M(0)? The answer is affirmative if τ is well behaved and is independent of the process M(t). Indeed, in this case

$$\mathbb{E}[M\left(\tau \right)\mid M\left(0\right)]=\mathbb{E}\left[\mathbb{E}[M\left(t\right)\mid M\left(0\right),\tau =t]\right]=\mathbb{E}\left[M\left(0\right)\right],$$ (5)

where the outer expectation is with respect to τ and the inner expectation is with respect to M(t).

But what if τ is not independent of M(t)? In these cases the situation is more subtle. Suppose τ is bounded from above by t̄. Then, we can write

$$M\left(\tau \right)=M\left(0\right)+{\int }_{u=0}^{\tau }dM\left(u\right)=M\left(0\right)+{\int }_{u=0}^{\stackrel{‒}{t}}1(\tau \ge u)dM\left(u\right),$$

where 1(·) is the indicator function that takes value 0 if its argument is false and 1 otherwise, and dM(u) is the (random) increment in M(u) at time u. If we assume that the value 1(τ ≥ u) can be deterministically computed based on the knowledge of M(0 : u), we can then write

$$\mathbb{E}\left[M\left(\tau \right)\right]=\mathbb{E}\left[M\left(0\right)\right]+{\int }_{u=0}^{\stackrel{‒}{t}}\mathbb{E}\left[1(\tau \ge u)dM\left(u\right)\right]=\mathbb{E}\left[M\left(0\right)\right]+{\int }_{u=0}^{\stackrel{‒}{t}}\mathbb{E}\left[\mathbb{E}[1(\tau \ge s)dM\left(u\right)\mid M(0:u)]\right],$$

where the second equality follows from the law of total expectation³. The first equality requires swapping of integral and expectation operators, which is allowed because t̄ is finite. Furthermore, 1(τ ≥ u) is a deterministic function of M(0 : u) and thus, $\mathbb{E}[1(\tau \ge u)dM\left(s\right)\mid M(0:u)]=\mathbb{E}[dM\left(u\right)\mid M(0:u)]1(\tau \ge u)=0$, where the last equality follows by definition of martingale. Consequently, for a random variable τ and martingale M(t), $\mathbb{E}\left[M\left(\tau \right)\right]=\mathbb{E}\left[M\left(0\right)\right]$ if (i) the event τ ≥ u is determined by M(0 : u), and (ii) τ is bounded from above with probability one. A random variable satisfying the first condition is called a stopping time, and the above discussion is the optional sampling theorem which we formally state:

The optional sampling theorem

Suppose M(t), t ≥ 0 is a martingale with respect to η(t) and τ is a bounded (with probability one) stopping time with respect to η(t), then $\mathbb{E}\left[M\left(\tau \right)\right]=\mathbb{E}\left[M\left(0\right)\right]$.

Heuristically, the optional sampling theorem states that different realizations of a martingale M(t) stopped at random times average out to constitute a fair game. The crucial aspect is that the stationarity of the expected value holds even for random (stopping) times, including our first passage time τ. As we will see in §3.3, this stationarity enables us to calculate analytic expressions for first passage time properties by finding appropriate martingales.

A helpful example is to consider the standard Wiener process with initial position at X(0) = x₀, and absorbing thresholds at ±ζ, with x₀ between ±ζ. In this case X(t) is itself a martingale, the first passage time τ is a stopping time, and the optional sampling theorem says that $\mathbb{E}\left[X\left(0\right)\right]=\mathbb{E}\left[X\left(\tau \right)\right]$. The expectation on the left hand side is simply the average of the initial distribution of X(0) which is the number x₀. The right hand side is more interesting: X(τ) is the random value of X(t) at the random decision time τ, the instant X(t) crosses +ζ or −ζ. Thus X(τ) attains one of two possible values, +ζ or −ζ, and $\mathbb{E}\left[X\left(\tau \right)\right]$ heuristically resembles an average over all +ζ's and −ζ's corresponding to sample paths starting at x₀ and diffusing until they hit either +ζ or −ζ at τ. The optional sampling theorem says that this average of +ζ's and −ζ's, over all such sample paths, ends up being equal to the number x₀.

3.3. Applications to the single-stage model

The optional sampling theorem is a powerful mathematical tool for decision-making models that associate decisions and decision times with a diffusion processes crossing a threshold. Ratcliff's Diffusion Decision Model (Ratcliff, 1978; Ratcliff and McKoon, 2008), the leaky competing accumulator model (Usher and McClelland, 2001), and the EZ diffusion model (Wagenmakers et al., 2007), are popular examples of such models. The optional sampling theorem reduces the problem of computing analytic expressions for the statistics of the first passage times to identifying appropriate martingales. In this section we illustrate the flavor of such calculations for the single-stage model from (1) in §2.1. Recall that the decision time τ is defined by τ = inf{t > 0|x(t) ∉ (−ζ, ζ)}. Throughout this this section, we introduce θ = (μ, σ, ζ) to slightly condense the notation when desired.

We first compute $\mathbb{P}[x\left(\tau \right)=-\zeta ]$, the probability of hitting the lower threshold. First, we let s = μ/σ², the ratio of the drift parameter to the squared diffusion parameter (i.e. signal to noise). Recall from §3.1 that for μ ≠ 0, exp(−2sX) is a martingale. Applying the optional sampling theorem, we get

$$\exp (-2s{x}_0)=\mathbb{E}\left[\exp (-2sx\left(\tau \right))\right]=\mathbb{P}[x\left(\tau \right)=\zeta ]\mathbb{E}\left[\exp (-2s\zeta )\right]+\mathbb{P}[x\left(\tau \right)=-\zeta ]\mathbb{E}\left[\exp \left(2s\zeta \right)\right].$$

Substituting $\mathbb{P}[x\left(\tau \right)=\zeta ]=1-\mathbb{P}[x\left(\tau \right)=-\zeta ]$ and solving for $\mathbb{P}[x\left(\tau \right)=-\zeta ]$, we obtain a closed form expression

$$\mathbb{P}[x\left(\tau \right)=-\zeta ]=\frac{\exp (-2s{x}_0)-\exp (-2s\zeta )}{\exp \left(2s\zeta \right)-\exp (-2s\zeta )}.$$

Similarly, for μ = 0, we note that X(t) is a martingale. Applying optional sampling theorem, we get

$$x_0=\mathbb{E}\left[X\left(\tau \right)\right]=\zeta \mathbb{P}[x\left(\tau \right)=\zeta ]-\zeta \mathbb{P}[x\left(\tau \right)=-\zeta ],$$

and following the same argument we obtain $\mathbb{P}[x\left(\tau \right)=-\zeta ]=(\zeta -x_0)∕2\zeta$. In summary, we get

$$P^{-}(x_0,\theta )≔1-P^{+}(x_0,\theta )≔\mathbb{P}[x\left(\tau \right)=-\zeta ]=\{\begin{array}{cc}\frac{\exp (-2s{x}_0)-\exp (-2s\zeta )}{\exp \left(2s\zeta \right)-\exp (-2s\zeta )},\hfill & \text{if}\mu \ne 0,\hfill \\ \frac{\zeta -x_0}{2\zeta },\hfill & \text{if}\mu =0,\hfill \end{array}\phantom{\}}$$ (6)

where P^±(x₀, θ) is the probability of hitting the upper and the lower threshold, respectively.

To compute the expected decision time $\mathbb{E}\left[\tau \right]$, recall from §3.1 that X(t) – μt is a martingale. For μ ≠ 0, applying the optional sampling theorem yields

$$x_0=\mathbb{E}\left[(\zeta -\mu \tau )\right]\mathbb{P}[x\left(\tau \right)=\zeta ]+\mathbb{E}\left[(-\zeta -\mu \tau )\right]\mathbb{P}[x\left(\tau \right)=-\zeta ].$$

Solving for $\mathbb{E}\left[\tau \right]$, we get $\mathbb{E}\left[\tau \right]=(1-2\mathbb{P}[x\left(\tau \right)=-\zeta ])\zeta ∕\mu$. When μ = 0, recall from §3.1 that $X\left(t\right)-{\sigma }_1^{2}t$ is a martingale, and the same argument as above yields $\mathbb{E}\left[\tau \right]=({\zeta }^2-x_0^2)∕{\sigma }_1^2$. In summary, the mean decision time mDT(x₀, θ) is given by

$$\mathrm{mDT}(x_0,\theta )=\mathbb{E}\left[\tau \right]=\{\begin{array}{cc}\frac{(1-2P^{-}(x_0,\theta ))\zeta -x_0}{\mu },\hfill & \text{if}\mu \ne 0,\hfill \\ \frac{{\zeta }^2-x_0^2}{{\sigma }^2},\hfill & \text{if}\mu =0.\hfill \end{array}\phantom{\}}$$ (7)

We also wish to find τ's Laplace transform, or moment generating function⁴, $\mathbb{E}\left[\exp \left(\alpha \tau \right)\right]$. We remember from §3.1 that exp(λX(t) – λμt – λ²σ²t/2) is a martingale, and choose λ so that the coefficient of t becomes −α, i.e., λ solves the equation $\frac{1}{2}{\sigma }^2{\lambda }^2+\mu \lambda -\alpha =0$. The two solutions to this equation are

$${\lambda }_1=\frac{-\mu -\sqrt{\mu +2\alpha {\sigma }^2}}{{\sigma }^2},\text{and}{\lambda }_2=\frac{-\mu +\sqrt{\mu +2\alpha {\sigma }^2}}{{\sigma }^2}.$$

Applying the optional sampling theorem, we obtain a pair of equations,

$$\begin{array}{c}\hfill \exp \left({\lambda }_1{x}_0\right)=\exp \left({\lambda }_1\zeta \right)\mathbb{E}\left[\exp (-\alpha \tau )1(x\left(\tau \right)=\zeta )\right]+\exp (-{\lambda }_1\zeta )\mathbb{E}\left[\exp (-\alpha \tau )1(x\left(\tau \right)=-\zeta )\right]\hfill \\ \hfill \exp \left({\lambda }_2{x}_0\right)=\exp \left({\lambda }_2\zeta \right)\mathbb{E}\left[\exp (-\alpha \tau )1(x\left(\tau \right)=\zeta )\right]+\exp (-{\lambda }_2\zeta )\mathbb{E}\left[\exp (-\alpha \tau )1(x\left(\tau \right)=-\zeta )\right],\hfill \end{array}$$

which we can solve simultaneously for $\mathbb{E}\left[\exp (-\alpha \tau )1(x\left(\tau \right)=\pm \zeta )\right]$ to obtain

$$\mathbb{E}\left[\exp (-\alpha \tau )1(x\left(\tau \right)=\pm \zeta )\right]=\exp \left(\frac{\mu (\pm \zeta -x_0)}{{\sigma }^2}\right)\frac{\sinh \left(\frac{(\zeta \pm x_0)\sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}{\sinh \left(\frac{2\zeta \sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}.$$

Thus, the moment generating function of the decision time is

$$\mathbb{E}\left[\exp (-\alpha \tau )\right]=\exp \left(\frac{\mu (\zeta -x_0)}{{\sigma }^2}\right)\frac{\sinh \left(\frac{(\zeta +x_0)\sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}{\sinh \left(\frac{2\zeta \sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}+\exp \left(-\frac{\mu (\zeta +x_0)}{{\sigma }^2}\right)\frac{\sinh \left(\frac{(\zeta -x_0)\sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}{\sinh \left(\frac{2\zeta \sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}.$$ (8)

As a byproduct, we also get the Laplace transform of conditional decision times:

$$\mathbb{E}[\exp (-\alpha \tau )\mid x\left(\tau \right)=\zeta ]=\frac{\exp \left(\frac{\mu (\zeta -x_0)}{{\sigma }^2}\right)}{P^{+}(x_0,\theta )}\frac{\sinh \left(\frac{(\zeta +x_0)\sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}{\sinh \left(\frac{2\zeta \sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}$$ (9)

$$\mathbb{E}[\exp (-\alpha \tau )\mid x\left(\tau \right)=-\zeta ]=\frac{\exp (-\frac{\mu (\zeta +x_0)}{{\sigma }^2})}{P^{-}(x_0,\theta )}\frac{\sinh \left(\frac{(\zeta -x_0)\sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}{\sinh \left(\frac{2\zeta \sqrt{2\alpha {\sigma }^2+{\mu }^2}}{{\sigma }^2}\right)}.$$ (10)

The derivatives of the Laplace transform yield moments of decision time (see Srivastava et al. (2016) for detailed derivation of conditional and unconditional moments of decision time using Laplace transforms). Here, we focus on expressions for conditional expected (mean) decision times that are the derivative of the Laplace transform with respect to −α computed at α = 0. The expected decision time conditioned on hitting the upper and lower boundaries are denoted by mDT⁺ and mDT⁻, and may be computed by differentiating (9) and (10):

$${\mathrm{mDT}}^{+}(x_0,\theta )=\frac{{\widehat{\mathrm{mDT}}}^{+}(x_0,\theta )}{P^{+}(x_0,\theta )}=\{\begin{array}{cc}\frac{2\zeta }{\mu }\coth \left(2s\zeta \right)-\frac{\zeta +x_0}{\mu }\coth \left(s(\zeta +x_0)\right),\hfill & \text{if}\mu \ne 0,\hfill \\ \left(\frac{4{\zeta }^2}{3{\sigma }^2}-\frac{{(\zeta +x_0)}^2}{3{\sigma }^2}\right),\hfill & \text{if}\mu =0;\hfill \end{array}\phantom{\}}$$ (11)

$${\mathrm{mDT}}^{-}(x_0,\theta )=\frac{{\widehat{\mathrm{mDT}}}^{-}(x_0,\theta )}{P^{-}(x_0,\theta )}=\{\begin{array}{cc}\frac{2\zeta }{\mu }\coth \left(2s\zeta \right)-\frac{\zeta -x_0}{\mu }\coth \left(s(\zeta -x_0)\right),\hfill & \text{if}\mu \ne 0,\hfill \\ \left(\frac{4{\zeta }^2}{3{\sigma }^2}-\frac{{(\zeta -x_0)}^2}{3{\sigma }^2}\right),\hfill & \text{if}\mu =0,\hfill \end{array}\phantom{\}}$$ (12)

where ${\widehat{\mathrm{mDT}}}^{\pm }=\mathbb{E}\left[\tau 1(x\left(\tau \right)=\pm \zeta )\right]$ and 1(·) is the indicator function. We again note, just as with $\mathbb{P}[x\left(\tau \right)=-\zeta ]$, mDT^± also depend on the underlying parameters μ, x₀, ζ, and σ.

We now compute τ's first passage time density f(t; x₀, μ, σ, ζ), i.e., the probability density function of the decision time. This amounts to calculating the inverse Laplace transform of (8). In this case, the inverse Laplace transform needs to be expressed as an infinite series (see Lin (1998) for a detailed derivation):

$$f(t;x_0,\theta )=\frac{d}{dt}\mathbb{P}[\tau \le t]=\exp \left(-\frac{a^{2}t}{2{\sigma }^2}\right)\left(\exp (-s(\zeta +x_0))\vartheta \left(t;\frac{\zeta -x_0}{\sigma },\frac{2\zeta }{\sigma }\right)+\exp \left(s(\zeta -x_0)\right)\vartheta \left(t;\frac{\zeta +x_0}{\sigma },\frac{2\zeta }{\sigma }\right)\right),$$ (13)

where θ = (a, σ, ζ), and ϑ(t; u, v) is a function (Borodin and Salminen, 2002, pp. 451) defined by

$$\vartheta (t;u,v)=\sum _{k=-\infty }^{+\infty }\frac{v-u+2kv}{\sqrt{2\pi }{t}^{3∕2}}\exp \left(-\frac{{(v-u+2kv)}^2}{2t}\right),u<v.$$

Similarly, the first passage time density conditioned on a particular decision is given by

$$\frac{d}{dt}\mathbb{P}[\tau \le t\mid x\left(\tau \right)=\zeta ]=\frac{f^{+}(t;x_0,\theta )}{P^{+}(x_0,\theta )}=\frac{\exp (-\frac{{\mu }^{2}t}{2{\sigma }^2}+s(\zeta -x_0))}{P^{+}(x_0,\theta )}\vartheta \left(t;\frac{\zeta +x_0}{\sigma },\frac{2\zeta }{\sigma }\right)$$ (14)

$$\frac{d}{dt}\mathbb{P}[\tau \le t\mid x\left(\tau \right)=-\zeta ]=\frac{f^{-}(t;x_0,\theta )}{P^{-}(x_0,\theta )}=\frac{\exp (-\frac{{\mu }^{2}t}{2{\sigma }^2}-s(\zeta +x_0))}{P^{-}(x_0,\theta )}\vartheta \left(t;\frac{\zeta -x_0}{\sigma },\frac{2\zeta }{\sigma }\right),$$ (15)

where $f^{\pm }(t;x_0,\theta )=\frac{d}{dt}\mathbb{P}[\tau \le t\&x\left(\tau \right)=\pm \zeta ]$, i.e., f^±(t; x₀, θ)dt is the probability of the event τ ∈ [t, t + dt) and x(τ) = ±ζ. Note that f defined in (13) is the sum of f⁺ and f⁻.

Alternate derivations for the hitting probabilities, mean decision times, and FPT densities may be found in the decision making literature (Ratcliff and Smith, 2004; Bogacz et al., 2006; Navarro and Fuss, 2009). It is worth noting that the infinite series solution for the FPT density given in (13) is equivalent to the small-time representations for the FPT analyzed in (Navarro and Fuss (2009) and Blurton et al. (2012)). For completeness, we provide the alternative expression for density in Appendix A.

4. Analysis of the two-stage model

In this section, we use the tools developed in §3 in order to analyze the two-stage process originally presented and analyzed in Ratcliff (1980). While our calculations lead to equivalent formulas for the first passage time densities, a martingale argument provides us with additional closed form expressions for the probability of hitting a particular threshold and expected decision times. Computations of these FPT statistics using the results of Ratcliff (1980) requires numerical integration of the FPT density, which our formulas now avoid.

We may explicitly write the two-stage model as

$$dx\left(t\right)=\mu \left(t\right)dt+\sigma \left(t\right)dW\left(t\right),x\left(t_0\right)=x_0,$$ (16)

where

$$\mu \left(t\right)=\{\begin{array}{cc}{\mu }_1,\hfill & \text{for}0\le t<t_1,\hfill \\ {\mu }_2,\hfill & \text{for}t\ge t_1,\hfill \end{array}\phantom{\}}\text{and}\sigma \left(t\right)=\{\begin{array}{cc}{\sigma }_1,\hfill & \text{for}0\le t<t_1,\hfill \\ {\sigma }_2,\hfill & \text{for}t\ge t_1.\hfill \end{array}\phantom{\}}$$

As before, the decision time τ = inf{t > 0|x(t) ∉ (−ζ, ζ)} is the first passage time with respect to boundaries at ±ζ. Let θ_i = (μ_i, σ_i, t_i, ζ) and $s_i={\mu }_i∕{\sigma }_i^2$, for i ∈ {1, 2}.

We interpret the first stage as a single-stage model with a deadline at t₁. For a single-stage model with thresholds ±ζ and a deadline t₁, the joint density g^ddln(x, τ; x₀, t₁, θ₁) of the evidence x(t₁) and the event τ ≥ t₁ is given by (Douady, 1999; Durrett, 2010):

$$\begin{array}{cc}\hfill g^{\mathrm{ddln}}(x,\tau ;x_0,t_1,{\theta }_1)& =\frac{d}{dx}\mathbb{P}[x\left(t_1\right)\le x\&\tau \ge t_1]\hfill \\ \hfill & =\frac{1[x\in (-\zeta ,\zeta )]}{\sqrt{2\pi t{\sigma }_1^2}}\exp \left(\frac{-{\mu }_1^{2}t+2{\mu }_1(x-x_0)}{2{\sigma }_1^2}\right)\times \sum _{n=-\infty }^{\infty }\left[\exp \left(\frac{-{(x-x_0+4n\zeta )}^2}{2{\sigma }_1^{2}t}\right)-\exp \left(\frac{-{(2\zeta -x-x_0+4n\zeta )}^2}{2{\sigma }_1^{2}t}\right)\right].\hfill \end{array}$$ (17)

Here superscript “ddln” refers to the deadline. g^ddln may then be used to determine the FPT distribution by integrating it over the range of x. More importantly, dividing g^ddln by $\mathbb{E}[\tau \ge t_1]$ yields the conditional density on the evidence x(t₁) conditioned on no decision until time t₁.

4.1. Probability of hitting the lower threshold

In trying to compute $\mathbb{P}[x\left(\tau \right)=-\zeta ]$, we view the two-stage process as two single-stage processes in sequence. Let τ₁ be the first passage time for the first stage by itself (without any deadline at t₁) and define the random time ${\widehat{\tau }}_1=\min \{{\tau }_1,t_1\}$ which is a stopping time. Applying the optional sampling theorem to the exponential martingale for the first stage gives us

$$\begin{array}{cc}\hfill & \exp (-2s_1{x}_0)=\mathbb{E}\left[\exp (-2s_{1}x\left({\widehat{\tau }}_1\right))\right]\hfill \\ \hfill =& \mathbb{E}[\exp (-2s_{1}x\left({\tau }_1\right))\mid {\tau }_1\le t_1]\mathbb{P}[{\tau }_1\le t_1]+\mathbb{E}[\exp (-2s_{1}x\left(t_1\right))\mid {\tau }_1>t_1]\mathbb{P}[{\tau }_1>t_1]\hfill \\ \hfill =& (\exp (-2s_1\zeta )\mathbb{P}[x\left({\tau }_1\right)=\zeta \mid {\tau }_1\le t_1]+\exp \left(2s_1\zeta \right)\mathbb{P}[x\left({\tau }_1\right)=-\zeta \mid {\tau }_1\le t_1])\mathbb{P}[{\tau }_1\le t_1]+\mathbb{E}\left(\exp (-2s_{1}x\left(t_1\right))1({\tau }_1>{\tau }_1)\right]\hfill \\ \hfill =& (\exp (-2s_1\zeta )+(\exp \left(2s_1\zeta \right)-\exp (-2s_1\zeta ))\mathbb{P}[x\left({\tau }_1\right)=-\zeta \mid {\tau }_1\le t_1])\mathbb{P}[{\tau }_1\le t_1]+\mathbb{E}\left[\exp (-2s_{1}x\left(t_1\right))1({\tau }_1>t_1)\right],\hfill \end{array}$$

which yields

$$\mathbb{P}[x\left({\tau }_1\right)=-\zeta \&{\tau }_1\le t_1]=\frac{\exp (-2s_1{x}_0)-\mathbb{E}\left[\exp (-2s_{1}x\left(t_1\right))1({\tau }_1>t_1)\right]-\exp (-2s_1\zeta )\mathbb{P}[{\tau }_1\le t_1]}{\exp \left(2s_1\zeta \right)-\exp (-2s_1\zeta )}.$$

In this expression, both $\mathbb{E}\left[\exp (-2s_{1}x\left(t_1\right))1({\tau }_1>t_1)\right]$ and $\mathbb{P}[{\tau }_1\le t_1]$ may be obtained from (17).

The second stage is another single-stage process, this time starting at time t₁ with a random initial condition x(t₁) with distribution given by (17). Computing the expected value of the standard lower threshold hitting probability (6) with respect to the random initial condition X₁, i.e., x(t₁) conditioned on τ₁ > t₁ we obtain

$$\mathbb{P}[x\left(\tau \right)=-\zeta \mid {\tau }_1>t_1]=\frac{\mathbb{E}[\exp (-2s_{2}x\left(t_1\right))\mid {\tau }_1>t_1]-\mathbb{E}(-2s_2\zeta )}{\exp \left(2s_2\zeta \right)-\exp (-2s_2\zeta )}.$$

The $\mathbb{E}[\exp (-2s_{2}x\left(t_1\right))\mid {\tau }_1>t_1]$ term can be readily computed from (17). Combining the previous two conditional expressions allows us to obtain

$$\mathbb{P}[x\left(\tau \right)=-\zeta ]=\mathbb{P}[x\left({\tau }_1\right)=-\zeta \&{\tau }_1\le t_1]+\mathbb{P}[x\left(\tau \right)=-\zeta \mid {\tau }_1>t_1]\mathbb{P}[{\tau }_1>t_1].$$

This probability depends on all of the parameters of the two-stage model: x₀, μ₁, σ₁, t₁, μ₂, σ₂, ζ.

4.2. Expected decision time

To compute the expected decision time, we apply the optional sampling theorem to the martingale x(t) – μ₁t with the stopping time ${\widehat{\tau }}_1$ from the previous section to obtain

$$x_0=\mathbb{E}[x\left({\widehat{\tau }}_1\right)-{\mu }_1{\widehat{\tau }}_1]=\mathbb{E}[x\left({\tau }_1\right)-{\mu }_1{\tau }_1\mid {\tau }_1\le t_1]\mathbb{P}[{\tau }_1\le t_1]+\mathbb{E}[x\left(t_1\right)-{\mu }_1{t}_1\mid {\tau }_1>t_1]\mathbb{P}[{\tau }_1>t_1]=(\zeta (1-2\mathbb{P}[x\left({\tau }_1\right)=-\zeta \mid {\tau }_1\le t_1])-{\mu }_1\mathbb{E}[{\tau }_1\mid {\tau }_1\le t_1])\mathbb{P}[{\tau }_1\le t_1]+(\mathbb{E}[x\left(t_1\right)\mid {\tau }_1>t_1]-{\mu }_1{t}_1)\mathbb{P}[{\tau }_t>t_1].$$

Solving for $\mathbb{E}\left[\tau 1({\tau }_1\le t_1)\right]$ we obtain

$$\mathbb{E}\left[\tau 1({\tau }_1\le t_1)\right]=\frac{(\mathbb{P}[{\tau }_1\le t_1]-2\mathbb{P}[x\left({\tau }_1\right)=-\zeta \&{\tau }_1\le t_1])\zeta -x_0+(\mathbb{E}\left[x\left(t_1\right)1({\tau }_1>t_1)\right]-{\mu }_1{t}_1\mathbb{P}[{\tau }_1>t_1])}{{\mu }_1}.$$

Much like in the previous section, we observe that the second stage is similar to a single-stage process starting at time t₁ with a random initial condition determined by (17). Thus, the associated expected first passage time is

$$\mathbb{E}[\tau \mid {\tau }_1>t_1]=t_1+\frac{(1-2\mathbb{P}[x\left(\tau \right)=-\zeta \mid \tau >t_1])\zeta -\mathbb{E}[x\left(t_1\right)\mid {\tau }_1>t_1]}{{\mu }_2}.$$

Combining the above expressions, we obtain

$$\mathbb{E}\left[\tau \right]=\frac{(\mathbb{P}[{\tau }_1\le t_1]-2\mathbb{P}[x\left({\tau }_1\right)=-\zeta \&{\tau }_1\le t_1])\zeta -x_0+(\mathbb{E}\left[x\left(t_1\right)1({\tau }_1>t_1)\right]}{{\mu }_1}+\frac{(1-2\mathbb{P}[x\left(\tau \right)=-\zeta \mid \tau >t_1])\mathbb{P}[{\tau }_1>t_1]\zeta -\mathbb{E}\left[x\left(t_1\right)1({\tau }_1>t_1)\right]}{{\mu }_2}.$$

4.3. First passage time density

We now compute the first passage time probability density function. Let F(t; θ, x₀) be the cumulative distribution function of the decision time for the single-stage process (1) obtained by integrating (13). For t ≤ t₁, the two-stage model is identical to the first-stage model and the first passage time distribution is F(t; θ₁, x₀). For t > t₁, the first passage time distribution is

$$F(t;{\theta }_1,{\theta }_2,x_0)=F(t_1;{\theta }_1,x_0)+\mathbb{E}\left[F(t-t_1;{\theta }_2,x\left(t_1\right))1({\tau }_1>t_1)\right],$$

i.e., the distribution function corresponds to trajectories that reach threshold before t₁ and trajectories that reach threshold between t₁ and t. The latter trajectories can be modeled as trajectories of a single stage process starting at time t₁ with stochastic initial condition. The stochastic initial condition leads to the expectation operator on the second term. The distribution of decision time conditioned on particular decisions can be computed analogously.

5. Analysis of the multistage model

In this section we derive first passage time (FPT) properties of the multistage process defined in §2.2 using an approach similar to that employed throughout §4. The model is viewed as n modified processes in sequence in which for each stage, the initial condition is a random variable and only the decisions made before a deadline are considered. For the i-th stage process with a known distribution of initial condition X_i–1, we derive properties of the FPT conditioned on a decision before the deadline t_i, along with the distribution of X_i for i ∈ {1, . . . , n}. This latter distribution, more precisely, is the distribution of x(t_i) conditioned on the FPT for the i-th stage being greater than t_i. We then use these properties sequentially for i ∈ {1, . . . , n} to determine the FPT properties during each stage. Finally, we aggregate FPT properties at each stage to compute FPT properties for the whole multistage process. Our calculations have features similar to an idea of Diederich and Busemeyer (2006), who first proposed that the bias (i.e. initial condition) of a stage may have a time dimension. In some sense our formulas below elucidate how previous temporal stages of processing affect the bias of future stages.

The extension of the FPT distribution computation from the two-stage model (Ratcliff , 1980) to the multistage model requires careful computation of expressions similar to (17) at the end of each stage. Also, in contrast to Ratcliff (1980), as in §4, our martingale based approach allows direct computation of probability of hitting a particular threshold and expected decision times. As in the previous section, this avoids integration of the first passage time density to compute these quantities.

Throughout this section we use the following notations:

• τ = inf{t > 0|x(t) ∉ (−ζ, ζ)}, the first passage time through either threshold for the entire multistage process;

• τ_i = τ|τ > t_i–1, the first passage time for the i-th stage (3) without any deadline;

• θ_i = (μ_i, σ_i, t_i, ζ), and ${\theta }_{1:\ell }=({\mu }_1,\dots ,{\mu }_{\ell },{\sigma }_1,\dots ,{\sigma }_{\ell },t_1,\dots ,t_{\ell },\zeta )$ representing the parameters for the i-th stage and stages 1, . . . , $\ell$, respectively;

• $s_i={\mu }_i∕{\sigma }_i^2$, the i-th stage ratio of signal to squared noise.

Here we are concerned with computation of FPT properties and allow θ_1:n to be free parameters. For scenarios such as estimation of parameters, to ensure identifiability of the parameters, all diffusion rates may be set equal to unity. However, such cases are beyond the scope of this manuscript and we do not discuss these issues here.

5.1. FPT properties of the i-th stage

For the i-th stage, the initial condition X_i–1 is a random variable and only decisions made before the deadline t_i are relevant. The analysis thus focuses on the random variable X_i–1 and the random time τ_i. Conditioned on a realization of X_i–1, the density of X_i can be computed using (17). If the density of X_i–1 is known, then the unconditional density of X_i can be obtained by computing the expected value of the conditional density of X_i with respect to X_i–1. Since the density of X₀ is known, this procedure can be recursively applied to obtain densities of X_i–1, for each i ∈ {1, . . . , n}. Formally, the joint density $g_i^{\mathrm{ddln}}(x,\tau ;x_0,{\theta }_{1:i})$ of the evidence x(t_i) and the event τ ≥ t_i is

$$g_i^{\mathrm{ddln}}(x,{\tau }_i;x_0,t_i,{\theta }_{1:i})=\frac{d}{dx}\mathbb{P}[x\left(t_i\right)\le x\&{\tau }_i>t_i]={\mathbb{E}}_{X_{i-1}}\left[g^{\mathrm{ddln}}(x,{\tau }_i-t_{i-1};X_{i-1},t_i-t_{i-1},{\theta }_i)\right],$$ (18)

where ${\mathbb{E}}_{X_{i-1}}[\cdot ]$ denotes the expected value with respect to X_i–1. Note that

$$\mathbb{P}[x\left(t_i\right)\le x\&{\tau }_i>t_i]=\mathbb{P}[x\left(t_i\right)\le x\mid {\tau }_i>t_i]\mathbb{P}[{\tau }_i>t_i]=\mathbb{P}[X_i\le x]\mathbb{P}[{\tau }_i>t_i].$$

Thus, the density of X_i, i.e., x(t_i) conditioned on τ_i > t_i is determined by dividing $g_i^{\mathrm{ddln}}$ by $\mathbb{P}[{\tau }_i>t_i]$ which can be computed by integrating $g_i^{\mathrm{ddln}}$ over the range of x(t_i). Note that the parameters in $g_i^{\mathrm{ddln}}$ are x₀ and θ_1:i; this highlights the fact that the distribution of X_i depends on all previous stages.

Similarly, the FPT density for the i-th stage conditioned on a realization of X_i–1 can be computed using (13), and the unconditional density can be obtained by computing the expected value of the conditional density with respect to X_i–1:

$$f_i(t;x_0,{\theta }_{1:i})=\frac{d}{dt}\mathbb{P}[{\tau }_i\le t]={\mathbb{E}}_{X_{i-1}}\left[f(t-t_{i-1};X_{i-1},{\theta }_i)\right],$$ (19)

where t > t_i–1. The cumulative distribution function $F_i(t;x_0,{\theta }_{1:i})=\mathbb{P}[{\tau }_i\le t]$ is obtained by integrating f_i(t; x₀, θ_1:i). Note that every trajectory crossing the decision threshold before t_i does so irrespective of the deadline at t_i. Thus, the expression for density f_i does not depend on t_i.

To conclude this section, we state equations for computing hitting times, mean decision times, and first passage time densities, conditional on a response during the i-th stage. The derivation of these expressions are found in Appendix B.

• (i)
  The probability of hitting the lower threshold given that a response is made during the i-th stage, denoted by $P_i^{-}(x_0,{\theta }_{1:i})≔1-P_i^{+}(x_0,{\theta }_{1:i})≔\mathbb{P}[x\left(\tau \right)=-\zeta \mid t_{i-1}<\tau \le t_i]$, is given by

  $$P_i^{-}=\{\begin{array}{cc}\frac{{\mathbb{E}}_{X_{i-1}}\left[\exp (-2s_i{X}_{i-1})\right]-{\mathbb{E}}_{X_i}\left[\exp (-2s_i{X}_i)\right]\mathbb{P}[{\tau }_i>t_i]-\exp (-2s_i\zeta )\mathbb{P}[{\tau }_i\le t_i]}{(\exp \left(2s_i\zeta \right)-\exp (-2s_i\zeta ))\mathbb{P}[{\tau }_i\le t_i]},\hfill & \text{if}{\mu }_i\ne 0,\hfill \\ \frac{1}{2}-\frac{({\mathbb{E}}_{X_{i-1}}\left[X_{i-1}\right]-{\mathbb{E}}_{X_i}\left[X_i\right]\mathbb{P}[{\tau }_i>t_i])}{2\zeta \mathbb{P}[{\tau }_i\le t_i]},\hfill & \text{if}{\mu }_i=0.\hfill \end{array}\phantom{\}}$$ (20)

  $P_i^{+}(x_0,{\theta }_{1:i})$ is the probability of hitting the upper threshold during the i-th stage. These expression depends on all of the model parameters up to and including the i-th stage (i.e. $P_i^{-}≔P_i^{-}(x_0,{\theta }_{1:i})$ and $P_i^{+}≔P_i^{+}(x_0,{\theta }_{1:i})$ both depend on x₀ and θ_1:i) and will be used in subsequent calculations.
• (ii)
  The joint FPT density for the i-th stage process and a given upper/lower response, denoted by $f_i^{\pm }(t;x_0,{\theta }_{1:i})=\frac{d}{dt}\mathbb{P}[{\tau }_i\le t\&x\left({\tau }_i\right)=\pm \zeta ]$, is given by

  $$f_i^{+}(t;x_0,{\theta }_{1:i})=\frac{d}{dt}\mathbb{P}[{\tau }_i\le t\&x\left({\tau }_i\right)=\zeta ]={\mathbb{E}}_{X_{i-1}}\left[f^{+}(t-t_{i-1};X_{i-1},{\theta }_i)\right]$$ (21)

  $$f_i^{-}(t;x_0,{\theta }_{1:i})=\frac{d}{dt}\mathbb{P}[{\tau }_i\le t\&x\left({\tau }_i\right)=-\zeta ]={\mathbb{E}}_{X_{i-1}}\left[f^{-}(t-t_{i-1};X_{i-1},{\theta }_i)\right],$$ (22)

  where the functions f^±(t; x₀, θ_i) are taken from (14) and (15). Again, these expressions depend on all of the multistage model parameters up to and including the i-th stage (i.e. both $f_i^{\pm }(t;x_0,{\theta }_{1:i})$ depend on x₀ and θ_1:i).
• (iii)
  The mean decision time given a response during stage i, denoted by mDT_i(x₀, θ_1:i), is given by

  $${\mathrm{mDT}}_i(x_0,{\theta }_{1:i})=\mathbb{E}[{\tau }_i\mid {\tau }_i\le t_i]=\{\begin{array}{cc}{t}_{i-1}+\frac{(1-2P_i^{-})\zeta \mathbb{P}[{\tau }_i\le t_i]-{\mathbb{E}}_{X_{i-1}}\left[X_{i-1}\right]+{\mathbb{E}}_{X_i}\left[X_i\right]\mathbb{P}[{\tau }_i>t_i]-{\mu }_i\left(t_{i-1}\right)\mathbb{P}[{\tau }_i>t_i]}{{\mu }_i\mathbb{P}[{\tau }_i\le t_i]},\hfill & \text{if}{\mu }_i\ne 0,\hfill \\ t_{i-1}+\frac{{\zeta }^2\mathbb{P}[{\tau }_i\le t_i]-{\mathbb{E}}_{X_{i-1}}\left[X_{i-1}^2\right]+{\mathbb{E}}_{X_i}\left[X_i^2\right]\mathbb{P}[{\tau }_i>t_i]-{\sigma }_i^2(t_i-t_{i-1})\mathbb{P}[{\tau }_i>t_i]}{{\sigma }_i^2\mathbb{P}[{\tau }_i\le t_i]},\hfill & \text{if}{\mu }_i=0.\hfill \end{array}\phantom{\}}$$ (23)
• (iv)
  The mean decision time conditioned on a given upper/lower response made during the i-th stage, denoted by ${\mathrm{mDT}}_i^{\pm }(x_0,{\theta }_{1:i})$, is given by

  $$\begin{array}{cc}\hfill {\mathrm{mDT}}_i^{+}(x_0,{\theta }_{1:i})& =\mathbb{E}[{\tau }_i\mid x\left({\tau }_i\right)=\zeta \&{\tau }_i\le t_i]\hfill \\ \hfill & =\frac{{\widehat{\mathrm{mDT}}}_i^{+}(x_0,{\theta }_{1:i})}{P^{+}(x_0,{\theta }_{1:i})\mathbb{P}[{\tau }_i\le t_i]}\hfill \\ \hfill & =\frac{1}{P^{+}(x_0,{\theta }_{1:i})\mathbb{P}[{\tau }_i\le t_i]}\left(t_{i-1}\mathbb{P}[x\left({\tau }_i\right)=\zeta ]+{\mathbb{E}}_{X_{i-1}}\left[{\widehat{\mathrm{mDT}}}^{+}(X_{i-1},{\theta }_i)\right]-\left({\mathbb{E}}_{X_i}\left[{\widehat{\mathrm{mDT}}}^{+}(X_i,{\theta }_i)\right]-t_i\mathbb{P}[x\left({\tau }_i\right)=\zeta \mid {\tau }_i>t_i]\right)\mathbb{P}[{\tau }_i>t_i]\right)\hfill \end{array}$$ (24)

  $$\begin{array}{cc}\hfill {\mathrm{mDT}}_i^{-}(x_0,{\theta }_{1:i})& =\mathbb{E}[{\tau }_i\mid x\left({\tau }_i\right)=-\zeta \&{\tau }_i\le t_i]\hfill \\ \hfill & =\frac{{\widehat{\mathrm{mDT}}}_i^{-}(x_0,{\theta }_{1:i})}{P_i^{-}(x_0,{\theta }_{1:i})\mathbb{P}[{\tau }_i\le t_i]}\hfill \\ \hfill & =\frac{1}{P_i^{-}(x_0,{\theta }_{1:i})\mathbb{P}[{\tau }_i\le t_i]}\left(t_{i-1}\mathbb{P}[x\left({\tau }_i\right)=-\zeta ]+{\mathbb{E}}_{X_{i-1}}\left[{\widehat{\mathrm{mDT}}}^{-}(X_{i-1},{\theta }_i)\right]-\left({\mathbb{E}}_{X_i}\left[{\widehat{\mathrm{mDT}}}^{-}(X_i,{\theta }_i)\right]-t_i\mathbb{P}[x\left({\tau }_i\right)=-\zeta \mid {\tau }_i>t_i]\right)\mathbb{P}[{\tau }_i>t_i]\right),\hfill \end{array}$$ (25)

  where ${\widehat{\mathrm{mDT}}}_i^{\pm }(x_0,{\theta }_{1:i})=\mathbb{E}\left[{\tau }_{i}1(x\left({\tau }_i\right)=\pm \zeta \&{\tau }_i\le t_i)\right]$ and is calculated using (11) and (12), $\mathbb{P}[x\left({\tau }_i\right)=\pm \zeta ]={\mathbb{E}}_{X_{i-1}}\left[P^{\pm }(X_{i-1},{\theta }_i)\right]$, and $\mathbb{P}[x\left({\tau }_i\right)=\pm \zeta \mid {\tau }_i>t_i]={\mathbb{E}}_{X_i}\left[P^{\pm }(X_i,{\theta }_i)\right]$ is calculated using (6) and (18).

5.2. FPT properties of the multistage model

For a given multistage process (2) with initial condition x₀, we sequentially compute all of the distributions of the initial conditions X_i for each i ∈ {1, . . . , n} using (18). Then, we compute the properties of the FPT associated with the i-th stage. Finally, the total probability formula aggregates these results into FPT properties of the entire multistage model. The calculations are contained in Appendix C and are given in terms of earlier formulas. In the following, we omit the arguments of functions whenever it is clear from the context.

• (i)
  Let t ≥ 0 be given such that t ∈ (t_k–1, t_k] for some k ∈ {1, . . . , n}. The FPT distribution for the multistage process is

  $$\mathbb{P}[\tau \le t]=1-\prod _{i=1}^{k-1}\mathbb{P}[{\tau }_i>t_i]+\mathbb{P}[{\tau }_k\le t]\prod _{i=1}^{k-1}\mathbb{P}[{\tau }_i>t_i].$$ (26)

  Note that ${\prod }_{i=1}^{k-1}\mathbb{P}[{\tau }_i>t_i]=\mathbb{P}[\tau >t_{k-1}]$ and $\mathbb{P}[{\tau }_k\le t]=\mathbb{P}[\tau \le t\mid \tau >t_{k-1}]$.
• (ii)
  The mean decision time, denoted by mDT_ms(x₀, θ_1:n), for the multistage process is

  $$\begin{array}{cc}\hfill {\mathrm{mDT}}_{\mathrm{ms}}(x_0,{\theta }_{1:n})=\mathbb{E}\left[\tau \right]& =\sum _{i=1}^n\left(\mathbb{E}[{\tau }_i\mid {\tau }_i\le t_i]\mathbb{P}[{\tau }_i\le t_i]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_i]\right)\hfill \\ \hfill & =\sum _{i=1}^n\left({\mathrm{mDT}}_i(x_0,{\theta }_{1:i})\mathbb{P}[{\tau }_i\le t_i]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_i]\right).\hfill \end{array}$$ (27)

  Put simply, the expected decision time is the sum of the expected decision times for the individual stages (mDT_i) weighted by the probability of the decision in each stage $\left(\mathbb{P}[{\tau }_i\le t_i]{\prod }_{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_i]\right)$.
• (iii)
  The probability of hitting the lower threshold, denoted by $P_{\mathrm{ms}}^{-}(x_0,{\theta }_{1:n})$, is

  $$P_{\mathrm{ms}}^{-}(x_0,{\theta }_{1:n})=1-P_{\mathrm{ms}}^{+}(x_0,{\theta }_{1:n})=\sum _{i=1}^n\left(P_i^{-}\mathbb{P}[{\tau }_i<t_i]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_j]\right).$$ (28)

  This expression is similar to (27), with the probability of hitting the lower threshold being the sum of the hitting probabilities for each stage $\left(P_i^{-}\right)$ weighted by the probability of the decision in each stage $\left(\mathbb{P}[{\tau }_i\le t_i]{\prod }_{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_i]\right)$.
• (iv)
  The mean decision time conditioned on hitting the upper/lower threshold is

  $$\mathbb{E}[\tau \mid x\left(\tau \right)=\zeta ]=\frac{1}{P_{\mathrm{ms}}^{+}}\sum _{i=1}^n\left(\mathbb{E}[{\tau }_i\mid x\left({\tau }_i\right)=\zeta \&{\tau }_i\le t_i]P_i^{+}\mathbb{P}[{\tau }_i\le t_i]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_i]\right)$$ (29)

  $$\mathbb{E}[\tau \mid x\left(\tau \right)=-\zeta ]=\frac{1}{P_{\mathrm{ms}}^{-}}\sum _{i=1}^n\left(\mathbb{E}[{\tau }_i\mid x\left({\tau }_i\right)=\zeta \&{\tau }_i\le t_i]P_i^{-}\mathbb{P}[{\tau }_i\le t_i]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_i]\right).$$ (30)

  Note that $\mathbb{E}[{\tau }_i\mid x\left({\tau }_i\right)=\zeta \&{\tau }_i\le t_i]P_i^{-}=\mathbb{E}[{\tau }_{i}1(x\left({\tau }_i\right)=\zeta )\mid {\tau }_i\le t_i]$ and $\mathbb{P}[{\tau }_i\le t_i]{\prod }_{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_i]=\mathbb{P}[t_{i-1}<\tau \le t_i]$. Similar to (27), these equations show that the conditional decision times are weighted sums of the expected conditional decision times for each stage, with the weights being the conditional probability of the decision in each stage.
• (v)
  The FPT cumulative distribution functions conditioned on hitting upper/lower threshold are

  $$\begin{gathered} \mathbb{P}[\tau \le t\mid x(\tau )=\zeta ]=\frac{1}{P_{\mathrm{ms}}^{+}}\left(\mathbb{P}[{\tau }_k\le t\&x\left({\tau }_k\right)=\zeta ]\prod _{j=1}^{k-1}\mathbb{P}[{\tau }_j>t_j] \\ +\sum _{i=1}^{k-1}\mathbb{P}[{\tau }_i\le t_i\&x\left({\tau }_i\right)=\zeta ]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_j]\right) \end{gathered}$$ (31)

  $$\begin{gathered} \mathbb{P}[\tau \le t\mid x\left(\tau \right)=-\zeta ]=\frac{1}{P_{\mathrm{ms}}^{-}}\left(\mathbb{P}[{\tau }_k\le t\&x\left({\tau }_k\right)=-\zeta ]\prod _{j=1}^{k-1}\mathbb{P}[{\tau }_j>t_j] \\ +\sum _{i=1}^{k-1}\mathbb{P}[{\tau }_i\le t_i\&x\left({\tau }_i\right)=-\zeta ]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_j]\right). \end{gathered}$$ (32)

  Note that ${\prod }_{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_j]=\mathbb{P}[\tau >t_{i-1}]$ and $\mathbb{P}[{\tau }_i\le t_i\&x\left({\tau }_i\right)=\zeta ]\mathbb{P}[\tau >t_{i-1}]=\mathbb{P}[t_{i-1}<\tau \le t_i\&x\left(\tau \right)=\zeta ]$.

6. Time-varying thresholds for the multistage process

The results in §5 were obtained under the assumption that the thresholds are constant throughout each stage. Now suppose that the thresholds for the i-th stage are ±ζ_i, i.e., piecewise constant thresholds. If the upper thresholds decrease at time t_i (i.e. ζ_i+1 < ζ_i) and x(t_i) is in the interval (ζ_i+1, ζ_i), then the path is absorbed by the upper boundary, and the probability of this instantaneous absorption is calculated by integrating (18) from ζ_i+1 to ζ_i. Likewise, the probability of instantaneous absorption into the lower threshold at t_i is determined by integrating (18) from −ζ_i to −ζ_i+1. The density of X_i is then found by truncating the support of the density in (18) to (−ζ_i+1, ζ_i+1) and normalizing the truncated density. In the cases where the upper threshold in the (i + 1)-th stage is larger than the upper threshold in the i-th stage, i.e., ζ_i+1 > ζ_i, there is no instantaneous absorption, and the density of X_i is found by extending the density in (18), assigning zero density to the previously undefined support (see Figure 1). In all cases, the new, updated X_i may be used for computations dealing with the (i + 1)-th stage of the multistage model. Codes implementing all of the formulas through §5 with time-varying thresholds may be found at https://github.com/sffeng/multistage. In Appendix D, we describe how these ideas extend to a time varying Ornstein-Uhlenbeck (O-U) model.

Figure 1.

Illustration of the key ideas for computation of FPT properties of the multistage model with piecewise constant drift and thresholds. The distribution of evidence x(t_i) conditioned on no decision until t_i serves as the distribution of initial condition for the i-th stage process. If threshold ζ_i+1 < ζ_i, then probability of instantaneous decision at time t_i is computed as the probability of x(t_i) conditioned on no decision until t_i not belonging to the set (−ζ_i, ζ_i). If threshold ζ_i+1 > ζ_i, then there is no instantaneous decision and only the support of x(t_i) conditioned on no decision until t_i is increased to (−ζ_i+1, ζ_i+1).

7. Numerical examples

In this section we apply our calculations from §5 and §6 to a variety of numerical experiments. In doing so, we compare the theoretical predictions obtained from the analysis in this paper with the numerical values obtained through Monte-Carlo simulations, thereby numerically verifying our derivations above. We also provide examples illustrating time pressure or changes in attention over the course of a decision process, and demonstrate how our work can help to find the optimal speed-accuracy trade-off by maximizing reward rate (or any other function of mean first passage time, threshold-hitting probability, and reward). Unless otherwise noted, Monte Carlo simulations were obtained using 1000 runs; relatively few runs are used so that curves are visually distinguishable. Stochastic simulations use the Euler-Maruyama method with time step size 10⁻³. All of the above calculations have been implemented in MATLAB, and all codes used to produce the figures in this section may be found at https://github.com/sffeng/multistage.

7.1. A Four-stage process

Consider a four stage process with drift and diffusion rates given by (μ₁, μ₂, μ₃, μ₄) = (0.1, 0.2, 0.05, 0.3) and (σ₁, σ₂, σ₃, σ₄) = (1, 1.5, 1.25, 2), respectively, with (t₀, t₁, t₂, t₃) = (0, 1, 2, 3) and initial condition x₀ = −0.2. The cumulative distribution function (CDF) of the unconditional and conditional decision time for ζ = 2 obtained using the above analytic expressions (solid lines) and Monte-Carlo simulations (dotted lines) is shown in Figure 2(a). Similarly, the unconditional mean decision time, the lower hitting probability, and the mean conditional decision times (for upper/lower responses) are shown in Figure 2(b) as a function of threshold ζ. Note that the analytic expressions match closely with quantities computed using Monte-Carlo simulations. Also, notice that the CDF almost looks like a double-sigmoidal function (it starts to saturate around 0.6 before picking up and eventually saturating at 1) due to the drop in drift rate from 0.2 to 0.05.

Figure 2.

FPT calculations for a four-stage process with drift rates (μ₁, μ₂, μ₃, μ₄) = (0.1, 0.2, 0.05, 0.3), diffusion rates (σ₁, σ₂, σ₃, σ₄) = (1, 1.5, 1.25, 2), stage initiation times (t₀, t₁, t₂, t₃) = (0, 1, 2, 3), and initial condition x₀ = −0.2. The FPT distribution is computed for threshold ζ = 2.

7.2. FPT Distribution for a process with Alternating Drift

In this example, the sign of the drift rate changes from stage to stage. This may be used to describe situations in which evidence accumulation changes dynamically with the decision-maker's focus of attention. For instance, Krajbich et al. (2010) have shown that the process of weighing two value-based options (e.g., foods) can be modeled with a process in which drift rates vary based on the option being attended at any given moment. We consider such a case using a 30-stage model in which the drift rates 1 and −0.75 alternate (i.e., μ₁ = 1, μ₂ = −0.75, μ₃ = 1, . . .) to capture a situation in which the decision maker's attention alternates between two options, one of which has greater perceived value (higher drift rate) than the other. Let t₀ = 0 and the remaining 29 stage initiation times be a fixed realization of 29 uniformly sampled points between 0 and 10. Assume x₀ = 0, ζ = 2, and let the diffusion rate be stationary and equal to unity (σ_i = 1). The unconditional and conditional FPT distributions in this scenario obtained using both the analytic expressions (solid lines) and Monte-Carlo simulations (dotted lines) are shown in Figure 3. Note that the analytic expressions match closely with quantities computed using Monte-Carlo simulations.

Figure 3.

Unconditional and conditional FPT distributions for a 30-stage model with alternating drift rate. The drift rates are (μ₁, μ₂, μ₃, . . .) = (1, −0.75, 1, . . .), diffusion rate at each stage is unity, the threshold ζ = 2, and stage initiation times are equally spaced throughout the interval (0, 10).

7.3. FPT Distribution for a model with gradually time-varying drift

Changes in evidence accumulation may occur gradually over time. For instance, White et al. (2011) proposed a “shrinking spotlight” model of the Eriksen Flanker Task, a task in which participants responding to the direction of a central arrow are influenced by the direction of arrows in the periphery (see also Servan-Schreiber et al., 1990; Liu et al., 2009; Servant et al., 2015). According to these models, evidence accumulation is initially influenced by all of the arrows (central as well as flankers, which may drive an incorrect response, modeled here as as a lower threshold response) but as the attentional spotlight narrows the drift rate is gradually more influenced by the central arrow alone. The multistage model, in spite of having discontinuous changes in parameters, can still be used to approximate a model with gradually time-varying parameters.

As a demonstration, we use a 20-stage model as an approximation to a model with continuously time-varying drift rate. Assume σ_i = 1, x₀ = 0, ζ = 2, and let the stage initiation times t₀, t₁, . . . , t₁₉ be equally spaced throughout the interval [0, 5]. Furthermore, suppose the drift rate during the i-th stage is −0.2 + 0.0263(i – 1). The unconditional and conditional FPT distributions for such a 20-stage process obtained using the analytic expressions (solid lines) and using Monte-Carlo simulations (dotted lines) are shown in Figure 4. Note that the analytic expressions match closely with quantities computed using Monte-Carlo simulations. We also show the error due to piecewise constant approximation of the drift rate as a function of number of stages (Figure 4, right). It should be noted that even for 5 stages the approximation error is very small.

Figure 4.

Unconditional (left) and conditional (middle) FPT distributions for a 20-stage process with gradually increasing drift rate. The drift rate for the i-th stage is μ_i = −0.2 + 0.0263(i – 1), diffusion rate at each stage is unity, the threshold ζ = 2, and stage initiation times are equally spaced throughout the interval [0, 5]. Right: The total squared error between the simulated and analytic CDFs (left) decreases with increasing discrete stages of the model. 10,000 simulations were used for each approximation.

7.4. Collapsing Thresholds

One may be interested in modeling a decision process in which thresholds are dynamic rather than constant across stages. This can be used to describe discrete changes in choice strategy, or a continuous change in thresholds over time; the latter approach has been successful at describing behavior under conditions that either involve an explicit response deadline (e.g., Milosavljevic et al., 2010; Frazier and Yu, 2008) or where there is an implicit opportunity cost for longer time spent accumulating evidence (Drugowitsch et al., 2012). Recently, (Voskuilen et al., 2016) used analytic methods to model collapsing boundaries in order to compare fixed boundaries against collapsing boundaries in diffusion models.

Here, we model such a situation using a 20-stage process, as an approximation to a diffusion model with continuously collapsing thresholds, i.e., ζ ↓ 0 with time, the drift rate and the diffusion rate are constant and equal to 0.15 and 1, respectively, x₀ = 0, and stage initiation times t₀, t₁, . . . , t₁₉ are equally spaced throughout the interval [0, 5]. The threshold in the i-th stage is ${\zeta }_i=3-\frac{1}{19}(i-1)$. The unconditional and conditional FPT distributions for such a 20-stage process obtained using the analytic expressions (solid lines) and using Monte-Carlo simulations (dotted lines) are shown in Figure 5. Note that the analytic expressions match closely with quantities computed using Monte-Carlo simulations.

Figure 5.

Unconditional and conditional FPT distributions for a 20-stage process with collapsing thresholds. The drift rate and diffusion rate at each stage are 0.15 and 1, respectively. The stage initiation times are equally spaced throughout the interval [0, 5], and the threshold at i-th stage is ${\zeta }_i=3-\frac{i-1}{19}$, for i = {1, . . . , 19}.

7.5. Optimizing the Speed-Accuracy Trade-off in a Two-stage Model

Human decision-making in many two alternative forced choice signal detection tasks has been successfully captured by the single stage model. In such tasks, hitting the upper/lower boundary is interpreted as a correct/incorrect response. The accuracy of a decision can then be determined by the sign of the drift rate – if μ is positive, participants are said to be more accurate the more likely they are to hit the upper threshold and more error-prone the more likely they are to hit the lower threshold. One then assumes without loss of generality that the drift rate is positive, in which case the lower hitting probability is called the error rate and the upper hitting probability is called the accuracy. Here, the choice of threshold dictates the speed-accuracy trade-off, i.e., the trade-off between a fast decision and an accurate decision. In the previous examples, the thresholds have been known and we have characterized the associated error rate and first passage time properties. These can be used to define a joint function of speed and accuracy that may dictate how humans/animals choose to set and adjust their threshold. In particular, it has been proposed (Bogacz et al., 2006) that human subjects choose a threshold that maximizes reward rate (RR), defined as

$$\mathrm{RR}=\frac{1-P^{-}}{\mathbb{E}\left[\tau \right]+T_{\mathrm{nd}}},$$ (33)

where T_nd is the sensory and motor processing time (non-decision time) and P⁻ and $\mathbb{E}\left[\tau \right]$ are computed using the expressions derived in §5.

The reward rate for a two-stage process is shown in Figure 6. For the set of parameters in Figure 6, reward rate is maximal at approximately (ζ₁, ζ₂) = (0.06, 0.01). Thus, the maximizing reward rate in this setting interestingly requires that the threshold across stages be different.

Figure 6.

Reward rate as a function of the thresholds for the two-stage model. The parameters are μ₁ = 0.5, μ₂ = −0.1, σ₁ = σ₂ = 0.1, x₀ = ζ₁/2, and t₁ = 0.1. The non-decision time T_nd = 0.3. Note that the maximum reward rate is achieved for ζ₁ ≠ ζ₂.

Setting the threshold to be constant across stages (ζ₁ = ζ₂), we can compare how reward rate changes with this constant threshold in a single-stage (traditional) versus a two-stage mode. As shown in Figure 7(a), we find that this reward rate function is unimodal in a single-stage model (as previously observed) whereas it is bimodal in a two-stage model. Figure 7(b) explores this parameter space in greater depth and shows that the curvature of reward rate (and in particular the relative height of its first and second modes) vary as a function of the length and drift rate of the first stage (for example). As a result, this analysis reveals a discontinuous jump in optimal threshold as these parameters vary. Whether individuals are sensitive to these discontinuities when setting thresholds for a multistage decision-making task deserves further exploration.

Figure 7.

(a) For the single-stage model μ₁ = 0.2, σ₁ = 0.1 and x₀ = 0, while for the two-stage model μ₁ = 0.1, μ₂ = 0.5, σ₁ = σ₂ = 0.1, x₀ = 0 and t₁ = 0.15. The non-decision time T_nd = 0.3. The reward rate for the single-stage model is a unimodal function and achieves a unique local maximum, while the reward rate for the two-stage model has two local maxima. (b) Optimal threshold for two-stage model obtained by maximizing reward rate. The left panel shows the variation of the optimal threshold as a function of t₁ and μ₁. The other parameters are μ₂ = 0.5, σ₁ = σ₂ = 0.1, and x₀ = 0. The regions of the contour plot associated with t₁ = 0 and μ₁ = 0.5 correspond to the single-stage model.

8. Discussion

In this paper we analyze the first passage time properties of a Wiener process between two absorbing boundaries with piecewise constant (time-dependent) parameters, which we call a multistage model or multistage process. Our main theoretical results, collected in §5, add to previous work on analyzing time-dependent random walk models in psychology and neuroscience. Broadly speaking, these can be split into three approaches. One approach is the integral equation approach introduced and developed in Smith (1995, 2000); Smith and Ratcliff (2009). Another approach is the matrix based Markov Chain approximation which has been applied to a wide variety of multi-attribute choice settings (Diederich and Busemeyer, 2003; Diederich and Oswald, 2016, 2014). A third approach analyzes the backward partial differential equation associated with the multistage process (Ratcliff, 1980; Heath, 1992). The results of §5 align most closely with the work of the third approach, as we also directly analyze the multistage stochastic process, albeit with different techniques. Whereas Ratcliff (1980) and Heath (1992) analyzed a multistage process by solving the Kolmogorov partial differential equation, we employ martingale theory (e.g. the optional sampling theorem) in order to obtain analytic results which extend those of previous studies. In doing so, our work also builds on martingale-based analyses described by Smith (1990) for a single stage model.

The modeler utilizing time-dependent random walk models in decision making should be aware of all of the above approaches, as one approach may demand additional approximations compared to another, due to differences in how they discretize temporal dynamics. For example, when modeling experiments with continuously (gradually) changing stimuli (e.g., White et al. (2011); Zhang et al. (2014)) one may find the integral equation approach more natural, given the continuity and smoothness assumptions built in to the techniques. Diederich and Oswald's model also allows for continuous changes in boundaries. In contrast, our methods must approximate the underlying gradually changing drift with a piecewise constant function, thereby inducing some additional error in the calculations of first passage times. If, however, the application lends itself well to discrete changes in drift rate or threshold (e.g., Krajbich et al. (2010)), then both the matrix based methods and methods discussed in this paper may be more natural, as they explicitly consider such discrete changes in the underlying calculations. Ornstein-Uhlenbeck processes lend themselves more naturally to the matrix based approach of Diederich and Busemeyer (2003), compared to our analysis in Appendix D, since our analysis requires a change in time scale that the matrix method does not. In the specific case of a multistage model (2), our work provides semi-analytic formulas that can be easily computed or studied further (see below). In general, the modeler should be ready to employ the most suitable approach given their situation.

The reader should also be aware of the software tools available for each approach, as there already exist several good non-martingale based software packages for computing FPT statistics. One package implementing the integral equation approach is that of Drugowitsch (2014), which computes first passage time densities using the stable numerical approximations developed in Smith (2000). More recently, highly optimized codes for a broad class of diffusion models have been developed by Verdonck et al. (2015), with implementations on both CPUs and GPUs. Compared to other available codes, Diederich and Busemeyer (2003)'s matrix approach is substantially simpler and more elegant to implement – one can compute desired choice probabilities and mean decision times in less than a few dozen lines of MATLAB code. For practitioners wishing to write all of their own model code from scratch, this may be a considerable advantage. The MATLAB code released with this report⁵ provides implementations of the results from §5 and allows one to reproduce all of the figures from §7. Unlike the work of Verdonck et al. (2015), our work is not immediately focused on developing a rapid numerical tool for simulation, but rather on introducing martingale theory as a useful mathematical tool for analyzing multistage decision models. Thus, the codes released with this report are not intended to compete with the efficiency of the aforementioned codes, which have been highly optimized and tuned for throughput, but instead to demonstrate the simplicity and effectiveness of our analysis. These considerations notwithstanding, our results do suggest promising avenues for future numerical work. Particularly relevant is work by Navarro and Fuss (2009); Blurton et al. (2012); Gondan et al. (2014) who developed efficient numerical schemes for evaluating the relevant infinite sums involved in FPT calculations. Similar methods could be applied to results in §5.1 and §5.2 to develop efficient multistage codes, which could in turn contribute to the growing collection of numerical tools available for practitioners using diffusion models to study decision making.

In §3 we re-derived classical mean decision times and choice probabilities equivalent to results first derived for a discrete time random walk model using the Wald identity (Laming, 1968; Link and Heath, 1975; Link, 1975; Smith, 1990), which itself is a corollary of the optional sampling theorem. Smith (1990) notes that results concerning the discrete time random walk model with Gaussian increments and the continuous time single stage Wiener diffusion model of §3 should be equivalent, which is indeed the case. Furthermore, the moment generating function derived in §3.3 is identical to that obtained by (Smith, 1990, p. 9) via the Wald identity. Our aim in presenting §3 was to introduce the martingale analysis used in §5 by first demonstrating its utility in re-deriving these classical results. We hope these calculations provide an intuition for the computations in §5 in a less technical setting. For reviews of the classical results on random walks in psychological decision making, see (Townsend and Ashby, 1983, pp. 300-301) and (Luce, 1986, pp. 328-334).

One particularly useful aspect of equations (20) through (32) is that they demonstrate how various other measures of performance, such as the lower-threshold hitting probability during each stage, evolve as the underlying dynamics change. Using these, one may efficiently compute a variety of behavioral measures of performance without resorting to first computing the FPT densities. Our results may also serve as a starting point for further analysis of more complicated stochastic decision models. In appendix D, we show how our results apply to Ornstein-Uhlenbeck processes, which approximate leaky integration over the course of evidence accumulation, e.g., the Leaky Competing Accumulator model [LCA] (Usher and McClelland, 2001). Given that the LCA itself can in certain cases approximate a reduced form of more complex and biologically plausible models of interactions across neuronal populations (e.g., Wang, 2002; Wong and Wang, 2006; Bogacz, 2007), our work may help analysts better understand time-varying dynamics within and across neural networks, and how such dynamics relate to complex cognitive phenomena.

Beyond its analytic and numerical utility, our uses of martingale theory, and the optional sampling theorem in particular, provide some theoretical insights into random walk decision models. For example, a single stage model with zero drift rate (i.e. Wiener process) and lower/upper thresholds at 0 and 1 necessarily has the qualitative property that the probability of hitting the upper threshold equals the initial (nonrandom) position x₀. Although these results are known in the probability literature (Doob, 1953), they provide practitioners and experimentalists a unique way of planning and analyzing experiments based on decisions thought to evolve according to diffusion processes. In summary, we hope that the tools described in this article and the results of §5 will encourage computational and mathematical analyses of decision models involving time-dependent parameters.

Highlights.

1. Multistage processes model decisions involving time-varying stimuli

2. Martingale theory provides derivations of classical reaction time properties

3. Semi-analytic formulas for mean decision time and hitting probability are derived

4. These derivations contribute to the study richer decision processes

Appendix A. Alternative expression for FPT density of the single stage model

An alternative expression to the FPT density (13) that can be obtained by solving the Fokker-Planck equation (Feller, 1968) is:

$$f(t;x_0,\theta )=\frac{\pi {\sigma }^2}{4{\zeta }^2}\exp \left(-\frac{{\mu }^{2}t}{2{\sigma }^2}\right)\sum _{n=1}^{+\infty }{(-1)}^{n-1}n\exp \left(-\frac{n^2{\pi }^2{\sigma }_1^{2}t}{8{\zeta }^2}\right)\left(\exp \left(\frac{\mu (\zeta -x_0)}{{\sigma }^2}\right)\sin \left(\frac{n\pi (\zeta +x_0)}{2\zeta }\right)+\exp \left(-\frac{\mu (\zeta +x_0)}{{\sigma }^2}\right)\sin \left(\frac{n\pi (\zeta -x_0)}{2\zeta }\right)\right).$$

Appendix B. Derivation of expressions in §5.1

We first establish (20). First consider the case μ_i > 0. Let ${\left\{{\mathcal{F}}_t^i\right\}}_{t\ge t_{i-1}}$ be the filtration defined by the evolution of the multistage process (2) until time t conditioned on τ > t_i–1. A filtration can be thought of as an increasing sequence of available information. For some s ∈ (t_i–1, t), as shown in §3, {exp(−2s_ix(t))}_t≥ti–1 is a martingale, i.e., $\mathbb{E}[\exp (-2s_{i}x\left(t\right))\mid {\mathcal{F}}_s^i]=\exp (-2s_{i}x\left(s\right))$.

Furthermore, ${\widehat{\tau }}_i≔\min \{{\tau }_i,t_i\}$ is a stopping time. Therefore, it follows from optional sampling theorem that

$$\begin{array}{cc}\hfill \mathbb{E}\left[\exp (-2s_i{X}_{i-1})\right]& =\mathbb{E}\left[\exp (-2s_{i}x\left({\widehat{\tau }}_i\right))\right]\hfill \\ \hfill & =\mathbb{E}[\exp (-2s_{i}x\left({\tau }_i\right))\mid {\tau }_i\le t_i]\mathbb{P}[{\tau }_i\le t_i]+\mathbb{E}\left[\exp (-2s_i{X}_i)\right]\mathbb{P}[{\tau }_i>t_i]]\hfill \\ \hfill & =(\exp (-2s_i\zeta )(1-P_i^{-})+\exp \left(2s_i\zeta \right)P_i^{-})\mathbb{P}[{\tau }_i\le t_i]+\mathbb{E}\left[\exp (-2s_i{X}_i)\right]\mathbb{P}[{\tau }_i>t_i]].\hfill \end{array}$$

Solving the above equation for $P_i^{-}$, we obtain the desired expression.

For μ_i = 0, , as shown in §3, {x(t)}_t≥ti is a martingale. Therefore, applying the optional sampling theorem, we obtain

$$\begin{array}{cc}\hfill \mathbb{E}\left[X_{i-1}\right]& =\mathbb{E}\left[x\left({\widehat{\tau }}_i\right)\right]\hfill \\ \hfill & =\mathbb{E}[x\left({\tau }_i\right)\mid {\tau }_i\le t_i]\mathbb{P}[{\tau }_i\le t_i]+\mathbb{E}\left[X_i\right]\mathbb{P}[{\tau }_i>t_i]]\hfill \\ \hfill & =(1-2P_i^{-})\zeta \mathbb{P}[{\tau }_i\le t_i]+\mathbb{E}\left[X_i\right]\mathbb{P}[{\tau }_i>t_i].\hfill \end{array}$$

Solving the above equation for $P_i^{-}$, we obtain the desired expression.

The formulas (21) and (22) immediately follow from applying expectation to (14) and (15), respectively.

To establish (23) for μ_i ≠ 0, we note from §3 that for the i-th stage, {x(t) – μ_it}_t≥ti is a martingale. Therefore, applying the optional sampling theorem, we obtain

$$\begin{array}{cc}\hfill \mathbb{E}\left[X_{i-1}\right]-{\mu }_i{t}_{i-1}& =\mathbb{E}[x\left({\widehat{\tau }}_i\right)-{\mu }_i{\widehat{\tau }}_i]\hfill \\ \hfill & =\mathbb{E}[x\left({\tau }_i\right)-{\mu }_i{\tau }_i\mid {\tau }_i\le t_i]\mathbb{P}[{\tau }_i\le t_i]+\mathbb{E}[(X_i-{\mu }_i{t}_i]\mathbb{P}[{\tau }_i>t_i]\hfill \\ \hfill & =(\zeta (1-P_i^{-})-\zeta P_i^{-}-{\mu }_i\mathbb{E}[{\tau }_i\mid {\tau }_i\le t_i])\mathbb{P}[{\tau }_i\le t_i]+(\mathbb{E}\left[X_i\right]-{\mu }_i{t}_i)\mathbb{P}[{\tau }_i>t_i].\hfill \end{array}$$

Solving the above equation for $\mathbb{E}[{\tau }_i\mid {\tau }_i\le t_i]$ yields the desired expression.

For μ_i = 0, we note from §3 that ${\{x{\left(t\right)}^2-{\sigma }_i^{2}t\}}_{t\ge t_{i-1}}$ is a martingale. Therefore, applying the optional sampling theorem, we obtain

$$\begin{array}{cc}\hfill \mathbb{E}\left[X_i^2\right]-{\sigma }_i^2{t}_{i-1}& =\mathbb{E}[x{\left(\widehat{\tau }\right)}^2-{\sigma }_i^2{\widehat{\tau }}_i]\hfill \\ \hfill & =\mathbb{E}[x{\left({\tau }_i\right)}^2-{\sigma }_i^2{\tau }_i\mid {\tau }_i\le t_i]\mathbb{P}[{\tau }_i\le t_i]+\mathbb{E}[X_i^2-{\sigma }_i^2{t}_i]\mathbb{P}[{\tau }_i>t_i]]\hfill \\ \hfill & =({\zeta }^2-{\sigma }_i^2\mathbb{E}[{\tau }_i\mid {\tau }_i\le t_i])\mathbb{P}[{\tau }_i\le t_i]+(\mathbb{E}\left[X_i^2\right]-{\sigma }_i^2{t}_i)\mathbb{P}[{\tau }_i>t_i].\hfill \end{array}$$

Solving the above equation for $\mathbb{E}[{\tau }_i\mid {\tau }_i\le t_i]$ yields the desired expression.

Next, we need to establish that the Laplace transform of the density for the FPT for a particular decision made before t_i is

$$\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left({\tau }_i\right)=\zeta \mid {\tau }_i\le t_i)\right]=\frac{\exp (-\alpha t_{i-1})\mathbb{E}\left[T_{+}^{\mathrm{mgf}}({\mu }_i,{\sigma }_i,\zeta ,X_{i-1},\alpha )\right]-\exp (-\alpha t_i)\mathbb{E}\left[T_{+}^{\mathrm{mgf}}({\mu }_i,{\sigma }_i,\zeta ,X_i,\alpha )\right]\mathbb{P}[{\tau }_i>t_i]]}{\mathbb{P}[{\tau }_i\le t_i]},$$

$$\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left({\tau }_i\right)=-\zeta \mid {\tau }_i\le t_i)\right]=\frac{\exp (-\alpha t_{i-1})\mathbb{E}\left[T_{-}^{\mathrm{mgf}}({\mu }_i,{\sigma }_i,\zeta ,X_{i-1},\alpha )\right]-\exp (-\alpha t_i)\mathbb{E}\left[T_{-}^{\mathrm{mgf}}({\mu }_i,{\sigma }_i,\zeta ,X_i,\alpha )\right]\mathbb{P}[{\tau }_i>t_i]]}{\mathbb{P}[{\tau }_i\le t_i]}.$$

To establish this, we consider the stochastic process $\exp (\lambda x\left(t\right)-\lambda {\mu }_{i}t-{\lambda }^2{\sigma }_i^{2}t∕2)$. From §3, we note that ${\left\{\exp (\lambda x\left(t\right)-\lambda {\mu }_{i}t-{\lambda }^2{\sigma }_i^{2}t∕2)\right\}}_{t\ge t_{i-1}}$ is a martingale for each $\lambda \in \mathbb{R}$, i.e.,

$$\mathbb{E}[\exp (\lambda x\left(t\right)-\lambda {\mu }_{i}t-{\lambda }^2{\sigma }_i^{2}t∕2)\mid {\mathcal{F}}_s^i]=\exp (\lambda x\left(s\right)-\lambda {\mu }_{i}s-{\lambda }^2{\sigma }_i^{2}s∕2).$$

We choose two particular values of λ:

$${\lambda }_1=\frac{-{\mu }_i-\sqrt{{\mu }_i^2+2\alpha {\sigma }_i^2}}{{\sigma }_i^2},\text{and}{\lambda }_2=\frac{-{\mu }_i+\sqrt{{\mu }_i^2+2\alpha {\sigma }_i^2}}{{\sigma }_i^2}.$$

Note that for $\lambda \in \{{\lambda }_1,{\lambda }_2\},\lambda {\mu }_{i}t+{\lambda }^2{\sigma }_i^{2}t∕2=\alpha$. Therefore, stochastic processes {exp(λ₁x(t) – αt)}_t≥0 and {exp(λ₂x(t) – αt)}_t≥ti–1 are martingales. Now applying the optional sampling theorem, we obtain

$$\mathbb{E}\left[\exp ({\lambda }_1{X}_{i-1}-\alpha t_{i-1})\right]=\mathbb{E}\left[\exp ({\lambda }_{1}x\left(\widehat{\tau }\right)-\alpha \widehat{\tau })\right]=\exp \left({\lambda }_1\zeta \right)\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left({\tau }_i\right)=\zeta \&{\tau }_i\le t_i)\right]+\exp (-{\lambda }_1\zeta )\mathbb{E}\left[\exp (-\alpha \tau )1(x\left({\tau }_i\right)=-\zeta \&{\tau }_i\le t_i)\right]+\exp (-\alpha t_i)\mathbb{E}\left[\exp \left({\lambda }_1{X}_i\right)\right]\mathbb{P}[{\tau }_i>t_i].$$ (B.1)

Similarly,

$$\mathbb{E}\left[\exp ({\lambda }_2{X}_{i-1}-\alpha t_{i-1})\right]=\exp \left({\lambda }_2\zeta \right)\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left(\tau \right)=\zeta \&{\tau }_i\le t_i)\right]+\exp (-{\lambda }_2\zeta )\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left(\tau \right)=-\zeta \&{\tau }_i\le t_i)\right]+\exp (-\alpha t_i)\mathbb{E}\left[\exp \left({\lambda }_2{X}_i\right)\right]\mathbb{P}[{\tau }_i>t_i]].$$ (B.2)

Equations (B.1) and (B.2) are two simultaneous equations in two unknowns $\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left({\tau }_i\right)=\zeta \&{\tau }_i\le t_i)\right]$ and $\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left({\tau }_i\right)=-\zeta \&{\tau }_i\le t_i)\right]$. Solving for these unknowns, we obtain

$$\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left({\tau }_i\right)=\zeta \&{\tau }_i\le t_i)\right]=\frac{1}{\exp \left(2{\lambda }_1\zeta \right)-\exp \left(2{\lambda }_2\zeta \right)}\left(\exp (-\alpha t_{i-1})\mathbb{E}[\exp \left({\lambda }_1(X_{i-1}+\zeta )\right)-\exp \left({\lambda }_2(X_{i-1}+\zeta )\right)]-\exp (-\alpha t_i)\mathbb{E}[\exp \left({\lambda }_1(X_i+\zeta )\right)-\exp \left({\lambda }_2(X_i+\zeta )\right)]\mathbb{P}[{\tau }_i>t_i]\right),$$

and

$$\mathbb{E}\left[\exp (-\alpha {\tau }_i)1(x\left({\tau }_i\right)=-\zeta \&{\tau }_i\le t_i)\right]=\frac{1}{\exp (-2{\lambda }_1\zeta )-\exp (-2{\lambda }_2\zeta )}\left(\exp (-\alpha t_{i-1})\mathbb{E}[\exp (-{\lambda }_1(\zeta -X_{i-1}))-\exp (-{\lambda }_2(\zeta -X_{i-1}))]-\exp (-\alpha t_i)\mathbb{E}\left[(\exp (-{\lambda }_1(\zeta -X_i))-\exp (-{\lambda }_2(\zeta -X_i)))\right]\mathbb{P}[{\tau }_i>t_i]\right).$$

Simplifying these expressions, we obtain the desired expression.

Finally, (24) and (25) follow from differentiating the Laplace transform with respect to −α, and then evaluating at α = 0.

Appendix C. Performance metrics for the overall multistage process

We start by establishing (26). Since t ∈ (t_k–1, t_k],

$$\begin{array}{cc}\hfill \mathbb{P}[\tau \le t]& =\mathbb{P}[\tau \le t\&\tau \le t_{k-1}]+\mathbb{P}[\tau \le t\&\tau >t_{k-1}]\hfill \\ \hfill & =\mathbb{P}[\tau \le t_{k-1}]+\mathbb{P}[\tau \le t\mid \tau >t_{k-1}]\mathbb{P}[\tau >t_{k-1}]\hfill \\ \hfill & =1-\prod _{i=1}^{k-1}\mathbb{P}[\tau >t_i\mid \tau >t_{i-1}]+\mathbb{P}[{\tau }_k\le t]\prod _{i=1}^{k-1}\mathbb{P}[\tau >t_i\mid \tau >t_{i-1}]\hfill \\ \hfill & =1-\prod _{i=1}^{k-1}\mathbb{P}[{\tau }_i>t_i]+\mathbb{P}[{\tau }_k\le t]\prod _{i=1}^{k-1}\mathbb{P}[{\tau }_i>t_i].\hfill \end{array}$$

We now establish (27). We note that

$$\begin{array}{cc}\hfill \mathbb{E}\left[\tau \right]& =\sum _{i=1}^n\mathbb{E}\left[\tau 1(t_{i-1}<\tau \le t_i)\right]\hfill \\ \hfill & =\sum _{i=1}^n\mathbb{E}[\tau 1(\tau \le t_i)\mid \tau >t_{i-1}]\mathbb{P}[\tau >t_{i-1}]\hfill \\ \hfill & =\sum _{i=1}^n\left(\mathbb{E}[{\tau }_i\mid {\tau }_i\le t_i]\mathbb{P}[{\tau }_i\le t_i]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_j]\right).\hfill \end{array}$$

To establish (28), we note that

$$\begin{array}{cc}\hfill P^{-}& =\sum _{i=1}^{n+1}\mathbb{P}(x\left(\tau \right)=-\zeta \text{and}{t}_{i-1}\le \tau <t_i)\hfill \\ \hfill & =\sum _{i=1}^{n+1}\mathbb{P}(x\left(\tau \right)-\zeta \text{and}\tau <t_i\mid \tau >t_{i-1})\mathbb{P}[\tau >t_{i-1}]\hfill \\ \hfill & =\sum _{i=1}^n\left(P_i^{-}\mathbb{P}[{\tau }_i<t_i]\prod _{j=1}^{i-1}\mathbb{P}[{\tau }_j>t_j]\right).\hfill \end{array}$$

Equations (29) and (30) follow similarly to (27), and Equations (31) and (32) follow similarly to (26).

Appendix D. Time-varying Ornstein-Uhlenbeck model

In this section, we discuss how the ideas presented in §5 and §6 can be used to computed FPT properties for a time varying Ornstein-Uhlenbeck (O-U) model.

The O-U model captures decision-making through the first passage of trajectories of an O-U process (Cox and Miller, 1965) through two thresholds. Our calculations for the multistage process also help analyze the Ornstein-Uhlenbeck (O-U) model. Similar to the multistage process, the n-stage O-U process with piecewise constant parameters is defined by

$$dx\left(t\right)=\mu \left(t\right)dt-\lambda \left(t\right)x\left(t\right)dt+\sigma \left(t\right)dW\left(t\right),x\left(t_0\right)=x_0,$$ (D.1)

where

$$\begin{array}{cc}\hfill \mu \left(t\right)={\mu }_i,\hfill & \hfill \text{for}{t}_{i-1}\le t<t_i,\hfill \\ \hfill \sigma \left(t\right)={\sigma }_i,\hfill & \hfill \text{for}{t}_{i-1}\le t<t_i,\hfill \\ \hfill \lambda \left(t\right)={\lambda }_i,\hfill & \hfill \text{for}{t}_{i-1}\le t<t_i,\hfill \end{array}$$

for each i ∈ {1, . . . , n}. Due to the extra −λ(t)x(t)dt term, the O-U process is a leaky integrator, while the original single stage process is a perfect integrator (λ(t) = 0).

Here, leaky integration means that as the noisy signal is integrated in time with exponentially increasing (λ < 0) or decreasing (λ > 0) weights on past observations. Such exponential weights lead to ‘recency’ or ‘decay’ effects, i.e., the earlier stages (or late stages) may have greater influence on the ultimate decision, whereas with the single stage process, all of the signal throughout the entire decision period is weighed equally.

Appendix D.1. The O-U process as a transformation of the Wiener process

In this section we show how our calculations for the multistage process can be easily applied to decision models driven by O-U processes via a transformed Wiener process (Cox and Miller, 1965, §5.9).

Consider the single-stage O-U process

$$dx\left(t\right)=\mu dt-\lambda x\left(t\right)dt+\sigma dW\left(t\right),x\left(t_0\right)=x_0.$$ (D.2)

The O-U process (D.2) can be written as a time-varying location-scale transformation of the Wiener process (Cox and Miller, 1965, §5.9), i.e.,

$$x\left(t\right)=\frac{\mu }{\lambda }(1-\exp (-\lambda t))+\exp (-\lambda t)x_0+\exp (-\lambda t)W\left(\frac{{\sigma }^2(\exp \left(2\lambda t\right)-1)}{2\lambda }\right).$$ (D.3)

In order to derive (D.3), note that for O-U process (D.2)

$$x\left(t\right)=x\left(0\right)\exp (-\lambda t)+\frac{\mu }{\lambda }(1-\exp (-\lambda t))+\sigma {\int }_0^t\exp (-\lambda s)dW\left(s\right),$$

Note that the stochastic process $\sigma {\int }_0^t\exp (-\lambda s)dW\left(s\right)$ is equivalent to the stochastic process σW((1 – exp(−2λt))/2λ) in the sense of distribution. Furthermore, σW((1 – exp(−2λt))/2λ) is equivalent to the stochastic process e^−λtW(σ²(exp(2λt) – 1)/2λ) in a similar sense. This means that for each realization of the process $\sigma {\int }_0^t\exp (-\lambda s)dW\left(s\right)$ there exists an identical realization of the process e^–λtW(σ²(exp(2λt) – 1)/2λ).

If we define the transformed time by $u\left(t\right)≔\frac{{\sigma }^2(\exp \left(2\lambda t\right)-1)}{2\lambda }$ so that $t=\frac{1}{2{\lambda }_1}\log (1+\frac{2\lambda u}{{\sigma }^2})$. Then

$$x_0+W\left(u\left(t\right)\right)=\left(x\left(t\right)-\frac{\mu }{\lambda }\right)\exp \left(\lambda t\right)+\frac{\mu }{\lambda }=\left(x\left(t\right)-\frac{\mu }{\lambda }\right)\sqrt{1+\frac{2\lambda u\left(t\right)}{{\sigma }^2}}+\frac{\mu }{\lambda }.$$

We refer to this process as a Wiener process evolving on exponential time scale.

Appendix D.2. First passage time of the O-U process

We now consider the first passage time of the O-U process (D.2) with respect to symmetric thresholds ±ζ. If x(t) = ±ζ, we have $x_0+W\left(u\right)=(\pm \zeta -\frac{\mu }{\lambda })\sqrt{1+\frac{2\lambda u}{{\sigma }^2}}+\frac{\mu }{\lambda }$. We denote this last quantity ζ^±(u). Consequently, the FPT for x(t) with respect to thresholds ±ζ is a continuous transformation of the FPT of a Wiener process starting at x₀ and evolving on the transformed time u with respect to time-varying thresholds at ζ^±(u). Since u is a monotonically increasing function, the distribution of the first passage time τ can be obtained from u(τ), the FPT distribution of the Wiener process. Furthermore, the lower threshold hitting probabilities of the two processes are the same.

Note that in transforming the FPT problem for the O-U process (D.2) to the FPT problem for the Wiener process evolving on exponential time scale, removes all the parameters from the underlying process x₀ + W(u) and puts them in thresholds ζ^±(u) and exponential time scale u. In addition to utilizing the results of §5 to multistage O-U processes, the above transformation is also helpful is speeding up Monte-Carlo simulations of the O-U process. Since the transformed process evolves on exponential time scale, the Monte-Carlo simulations with transformed process should roughly take time that is a logarithmic function of time taken by the O-U process (D.2).

Computation of FPT distributions for the Wiener process with time-varying thresholds is, to our knowledge, not analytically tractable. However, the time-varying thresholds can be approximated by piecewise constant time-varying thresholds and approximate FPT distributions can be computed using the multistage model. While the thresholds ζ^± are asymmetric for the transformed process (i.e. ζ⁺ ≠ −ζ⁻), unlike in the case described for the multistage process; such a case can be easily handled by replacing the expression in (13) and (18) with corresponding expressions for asymmetric thresholds (see Douady, 1999; Borodin and Salminen, 2002).

Appendix D.3. Approximate computation of the FPT distribution of the multistage O-U process

Similar to the transformation described in Appendix D.1, the multistage O-U process (D.1) for t ∈ [t_i–1, t_i) can be written as

$$\begin{gathered} x\left(t\right)=\frac{{\mu }_i}{{\lambda }_i}(1-\exp (-{\lambda }_i(t-t_{i-1}))) \\ +\exp (-{\lambda }_i(t-t_{i-1}))x\left(t_{i-1}\right) \\ +\exp (-{\lambda }_i(t-t_{i-1}))W\left(\frac{{\sigma }_i^2(\exp \left(2{\lambda }_i(t-t_{i-1})\right)-1)}{2{\lambda }_i}\right). \end{gathered}$$ (D.4)

Let $u_i\left(t\right)={\sigma }_i^2(\exp \left(2{\lambda }_i(t-t_{i-1})\right)-1)∕2{\lambda }_i$. Also, let τ_i and X_i–1, i ∈ {1, . . . , n} be defined similarly to the multistage model. Then, conditioned on a realization of X_i–1, the FPT problem of the i-th stage O-U process can be equivalently written as the FPT problem of the Wiener process X_i–1 + W(u_i(t)) with respect to thresholds

$${\zeta }_i^{\pm }\left(u_i\left(t\right)\right)=\left(\pm \zeta -\frac{{\mu }_i}{{\lambda }_i}\right)\sqrt{1+\frac{2{\lambda }_i{u}_i\left(t\right)}{{\sigma }_i^2}}+\frac{{\mu }_i}{{\lambda }_i}.$$

We can approximate each stage of the O-U process (D.1) by a multistage process representing the above Wiener process with time varying thresholds. This sequence of multistage processes is itself a larger multistage process that approximates (D.1) and its FPT distribution can be computed using the method developed in §5. Note that this method only yields FPT distributions. The expected decision times and probability of hitting a particular threshold can be computed by integrating these distributions.