Time-varying perturbations can distinguish among integrate-to-threshold models for perceptual decision-making in reaction time tasks

Abstract

Several integrate-to-threshold models with differing temporal integration mechanisms have been proposed to describe the accumulation of sensory evidence to a prescribed level prior to motor response in perceptual decision-making tasks. An experiment and simulation studies have shown that the introduction of time-varying perturbations during integration may distinguish among some of these models. Here, we present computer simulations and mathematical proofs that provide more rigorous comparisons among one-dimensional stochastic differential equation models. Using two perturbation protocols and focusing on the resulting changes in the means and standard deviations of decision times, we show that, for high signal-to-noise ratios, drift-diffusion models with constant and time-varying drift rates can be distinguished from Ornstein-Uhlenbeck processes, but not necessarily from each other. The protocols can also distinguish stable from unstable Ornstein-Uhlenbeck processes, and we show that a nonlinear integrator can be distinguished from these linear models by changes in standard deviations. The protocols can be implemented in behavioral experiments.

1 Introduction

Reaction time tasks have long been used to study human decision making (Luce, 1986). One paradigm requires subjects to detect or discriminate sensory signals by making a voluntary motor response. Response time distributions obtained in such perceptual-motor tasks allow inference of cognitive information processing (Posner, 1978). With the advent of methods for recording neural activity in awake behaving animals, such tasks have been adopted by neurophysiologists, especially in studies on monkeys (Schall, 2003; Gold and Shadlen, 2007), and neuronal firing rates in several brain areas have been shown to correlate with motor responses. The lateral intraparietal area, frontal eye fields, and superior colliculus (Gold and Shadlen, 2007) all exhibit activities that ramp up over time toward a fixed level before a decision is signaled (e.g. by a saccadic eye movement in the direction of the recorded response field) (Hanes and Schall, 1996; Roitman and Shadlen, 2002; Churchland et al., 2008). The slopes of the ramps not only correlate with task difficulty (the harder the task, the lower the slope), but also with response time (higher slopes precede faster responses). These areas may therefore provide neural substrates for integrating sensory information toward a decision criterion before a perceptual decision is made.

Various integrate-to-threshold models have been proposed to describe both response times and neurobiological mechanisms (Smith and Ratcliff, 2004), including drift-diffusion models (Ratcliff, 1978; Mazurek et al., 2003; Smith and Ratcliff, 2004; Ditterich, 2006a; Simen et al., 2006; Ratcliff and McKoon, 2008) and attractor neural networks with mutual inhibition (Brown and Holmes, 2001; Usher and McClelland, 2001; Wang, 2002; Wong and Wang, 2006; Lo and Wang, 2006). All share a common mechanism: accumulation or integration of sensory inputs toward a prescribed threshold, the first crossing of which determines the decision and response time. These models may nonetheless be distinguished by the details of the integration process: drift-diffusion models typically accumulate evidence at a constant rate like a biased random walk, while attractor networks have unstable or stable steady-states that can, respectively, accelerate or decelerate the integration process (Usher and McClelland, 2001; Wong and Wang, 2006).

Often only behavioral data are collected for human subjects, and fits based on response times and choice accuracies are sometimes unable to distinguish among competing models (Ratcliff et al., 1999; Ratcliff, 2006). Moreover, while cellular recordings in awake behaving animals offer direct insights into the integration process, fitting of both behavioral and neural data may still not suffice, especially when models incorporate multiple features and depend on multiple parameters. For example, a drift-diffusion model with a time-dependent ‘urgency’ signal (Ditterich, 2006a; Churchland et al., 2008) may be difficult to separate from a recurrent network model with strong self-excitation (Wang, 2002; Wong and Wang, 2006). Thus far, few principled attempts have been made to tease apart different integration mechanisms.

Subthreshold electrical microstimulation of neural activities in behaving animals may provide more conclusive tests (Cohen and Newsome, 2004). In Ditterch et al. (2003), microstimulation of sensory neurons was shown to affect the speed of the decision. More interestingly, in Hanks et al. (2006), stimulation of cells with choice targets in the recorded neuronal response fields speeded decisions, but also reduced decision speeds when choice target directions were opposite to the recorded response fields, thus providing evidence of mutual inhibition. Perturbations need not be invasive, and so can be used in human studies: e.g., Huk and Shadlen (2005) employed a brief motion pulse in the background of the primary visual stimulus; although it neither determined the choice nor influenced rewards, the pulse had a significant effect on response times.

Earlier modeling efforts (Huk and Shadlen, 2005; Wong et al., 2007; Wong and Huk, 2008) have addressed the data of Huk and Shadlen (2005), but they employed many parameters and direct comparisons were not made between models. Here, we conduct a more rigorous study using simpler models, and seek more objective comparisons among them. We approximate four neural firing rate models as linear, scalar, stochastic differential equations (SDE), and we ask how their predicted response times are affected by short piecewise-constant perturbations with varying onset times and amplitudes. By comparing changes in means and standard deviations of response times, we demonstrate that the perturbations suffice to distinguish among the models. Finally, we show that a nonlinear integrator model, qualitatively similar to that of Wong and Wang (2006), behaves much like one of the linear models.

2 Methods

2.1 Reduction to one-dimensional linear integrate-to-threshold models

Two-alternative forced-choice decision processes can, in essence, be modeled by two populations of excitatory neurons each endowed with self-excitatory connections and mutual inhibition via a shared inhibitory population. Each of the three populations can then be represented in a coarse-grained firing rate model by a single unit whose state describes the population-averaged activity r_j(t) of the corresponding neuronal pool (Wilson and Cowan, 1972, 1973). Here we review the further reduction of a firing rate model, under suitable hypotheses, to a one-dimensional dynamical system. For additional details, see Brown et al. (2005); Bogacz et al. (2006).

Consider first the deterministic equations describing the firing rates of two excitatory (E) populations and a common inhibitory (I) population:

$$\begin{array}{c}{\tau }_E\frac{d{r}_1}{dt}=-r_1+F_E(r_1,r_2,r_I,I_1),\hfill \\ {\tau }_E\frac{d{r}_2}{dt}=-r_2+F_E(r_2,r_1,r_I,I_2),\hfill \\ {\tau }_I\frac{d{r}_I}{dt}=-r_I+F_I(r_1,r_2).\hfill \end{array}$$ (1)

Here τ_E and τ_I are the synaptic time constants for the E and I units, and r₁, r₂ and r_I are respectively the activity of neural units 1, 2 and I. The excitatory populations 1 and 2 are selective to stimuli 1 and 2, respectively. F_E and F_I are their input-output functions, and the overall input to each unit, with i = E or I, is I_i = I_recurrent,i + I_stimulus,i. The decision time is the first passage time from stimulus onset to the first of r₁ or r₂ reaching a prescribed decision threshold, which thereby signals choice 1 or 2. Since non-decision latencies (e.g. signal transduction and motor preparation) are usually assumed to be independent of stimulus strength I_stimulus, we shall model the response time as the decision time plus a constant latency. Hence-forth we use the phrases ‘decision time’, ‘reaction time’ and ‘response time’ interchangeably.

If the decision dynamics passes near a saddle point and moves along its unstable manifold (Brown and Holmes, 2001; Bogacz et al., 2006), the functions F_E, F_I may be linearized such that Eq. 1 simplifies to:

$$\begin{array}{c}{\tau }_E\frac{d{r}_1}{dt}=-r_1+(\alpha r_1-\beta r_I+I_1+I_{0,E}),\hfill \\ {\tau }_E\frac{d{r}_2}{dt}=-r_2+(\alpha r_2-\beta r_I+I_2+I_{0,E}),\hfill \\ {\tau }_I\frac{d{r}_I}{dt}=-r_I+(\gamma r_1+\gamma r_2+I_{0,I}).\hfill \end{array}$$ (2)

Here α, β, γ, are the recurrent synaptic coupling strengths for self-excitation, inhibitory-to-excitatory, and excitatory-to-inhibitory connections, and I_0,E (I_0,I) is the constant background input to all the excitatory (inhibitory) cells from outside the local circuit. Unlike Wang (2002), we exclude excitatory connections between r₁ and r₂ and self-inhibitory connections, retaining only essential features. If τ_I ≪ τ_E, we can further assume that the relatively fast dynamics of r_I equilibrates rapidly, such that r_I ≈ γ(r₁+r₂)+I_0,I, and Eq. 2 becomes

$${\tau }_E\frac{d{r}_1}{dt}=-r_1+(\alpha -\beta \gamma )r_1-\beta \gamma r_2+I_1+(I_{0,E}-\beta I_{0,I}),$$ (3)

$${\tau }_E\frac{d{r}_2}{dt}=-r_2+(\alpha -\beta \gamma )r_2-\beta \gamma r_1+I_2+(I_{0,E}-\beta I_{0,I}).$$ (4)

Defining a new variable X ≡ r₁-r₂ and subtracting Eq. 4 from 3, we obtain

$$\frac{dX}{dt}=kX+(I_1\left(t\right)-I_2\left(t\right))∕{\tau }_E,$$

where k ≡ (α - 1)/τ_E contains the excitatory coupling strength and leak, and (I₁ - I₂)/τ_E is proportional to the difference in inputs. The background inputs I_0,E and I_0,I and coupling strengths β and γ cancel out.

Generalizing the inputs I₁, I₂ to be time-varying and including additive noise, the reduced dynamics is described by a one-dimensional SDE of the form

$$d{X}_t=b(X_t,t)dt+\sigma d{W}_t,$$ (5)

where b(X_t, t) ≡ kX_t + (I₁(t) - I₂(t))/τ_E and σ is the standard deviation of the noise, which is assumed to be a Wiener process with increments dW_t drawn from a normal distribution with zero mean and unit variance. Eq. 5 provides a general description for noisy accumulator models, as illustrated schematically in Fig. 1A.

Figure 1.

One-dimensional integrate-to-threshold models and perturbation protocols. A: a general model for a two-alternative forced-choice reaction time task. Here the noisy state X_t, which quantifies the difference in firing rates between two competing excitatory cell populations, reaches the upper decision threshold in a sample trial, and choice is correct since threshold is in same direction as drift rate. Multiple sampling provides the response time distribution. B: the SPB model with unbiased input exhibits a subcritical pitchfork bifurcation at ε = 0. Black curves show branches of stable (solid) and unstable (dashed) fixed points in the noiseless limit; branches for biased input shown in gray. C: a short pulse can be repeated over a range of onset times T after primary stimulus onset at t = 0 to test effects on decision times. D: a pulse-antipulse perturbation whose leading pulse amplitude (red) can be adjusted to determine when decision times are unaffected by the combined pulse, yielding a zero-effect perturbation (ZEP). See text for further details.

2.2 Linear integrate-to-threshold models

We are now in a position to introduce four linear instances of Eq. 5. When k < 0 (e.g. self-excitation α is relatively weak), we have a classical Ornstein-Uhlenbeck (OU) process (Uhlenbeck and Ornstein, 1930; Wang and Uhlenbeck, 1945). If the decision threshold does not intervene, solutions eventually decelerate and approach a stable steady-state X_steady-state = (I₁(t)-I₂(t))/(|k|τ_E). We denote this stable OU process SOU; in contrast, if k > 0 (stronger self-excitation), sample paths accelerate away from a fixed point in an unstable OU process, denoted UOU. SOU and UOU provide reduced descriptions of the leaky competing accumulator of Usher and McClelland (2001) that respectively produce recency and primacy effects.

In the special case of α = 1 (i.e. k = 0) and constant stimuli I₁, I₂, Eq. 5 becomes a pure drift-diffusion (CD) equation (Ratcliff, 1978)

$$d{X}_t=b_{0}dt+\sigma d{W}_t,$$ (6)

in which the constant drift rate b = b₀ ≡ (I₁ - I₂)/τ_E is proportional to the difference in stimuli. This CD model is similar to, but differs in detail from, the balance between leakage and inhibition in Bogacz et al. (2006).)

In some variants of the CD model, drift rates can vary with time. This could be due to dynamic stimuli, to some form of ‘urgency’ induced by the task design (Ditterich, 2006a,b; Churchland et al., 2008), to fluctuating attention (Smith et al., 2004) or to other ‘top-down’ effects (Liu et al., 2008). This time-dependent (TD) diffusion model is described by:

$$d{X}_t=b\left(t\right)dt+\sigma d{W}_t.$$ (7)

We focus on the simple case b(t) = b₀t, since similar results obtain for other functions of time, but in Appendix B, we treat a modified TD model with time-varying perturbation amplitude.

2.3 A nonlinear integrate-to-threshold model

The final model to be considered is qualitatively similar to that of Wang (2002); Wong and Wang (2006). It captures stochastic dynamics in the neighborhood of a subcritical pitchfork bifurcation (Guckenheimer and Holmes, 1983; Brown and Holmes, 2001; Strogatz, 2001) that occurs as the stimulus input level is varied (Roxin and Ledberg, 2008; Wong and Wang, 2006):

$${\tau }_{X}d{X}_t=[\epsilon X_t+X_t^3-X_t^5+b\left(t\right)]dt+\sigma d{W}_t.$$ (8)

Here b(t) ≡ b₀ (constant) and ε represent the biased and non-biased stimulus inputs respectively (Fig. 1B). τ_X controls the overall temporal dynamics. Eq. 8 may be derived by normal form theory (Guckenheimer and Holmes, 1983; Roxin and Ledberg, 2008): see Fig. 1B for an illustration of the branches of stable and unstable fixed points in the noise-free limit. If |ε| and |b| are sufficiently small, this model without the $X_t^5$ term approaches that of Wang (2002); Wong and Wang (2006); Roxin and Ledberg (2008). We include $-X_t^5$ to prevent unrealistic runaway activity, and, for simplicity, first assume a symmetrical system with b₀ = 0, and then consider biased stimuli |b₀| > 0. This model is denoted SPB.

2.4 Perturbation protocols

2.4.1 Single pulse perturbation with varying onset time

We consider additive perturbations b₁(t), under which Eq. 5 becomes dX_t = [b(X_t, t) + b₁(t)]dt + σdW_t, where b₁(t) is applied from time T to T + ΔT and b₁(t) ≡ 0 otherwise. We employ two piecewise-constant forms, the first being the step function used by Huk and Shadlen (2005):

$$b_1\left(t\right)=\{\begin{array}{cc}0,\hfill & t\le T,\hfill \\ p,\hfill & T<t\le T+\Delta T,\hfill \\ 0,\hfill & t>T+\Delta T,\hfill \end{array}\phantom{\}}$$ (9)

in which the amplitude p can be positive or negative, assisting or opposing the unperturbed drift rate b₀ due to the ‘primary’ stimulus. We fix the duration ΔT at 10% of the unperturbed mean first passage time τ₀/10, apply b₁ at different onset times T and ask how the mean and standard deviation of the decision time change.

2.4.2 Zero-effect pulse-antipulse perturbation

The second protocol uses a double pulse of the form

$$b_1\left(t\right)=\{\begin{array}{cc}0,\hfill & t\le T,\hfill \\ -\lambda p,\hfill & T<t\le T+\Delta T∕2,\hfill \\ p,\hfill & T+\Delta T∕2\le t<T+\Delta T,\hfill \\ 0,\hfill & t>T+\Delta T,\hfill \end{array}\phantom{\}}$$ (10)

where λ is the relative height of the first pulse to the second. The second opposite-signed pulse attempts to reduce or cancel the expected change due to the first, and if the overall perturbation b₁ leaves the mean first passage time unchanged, we call it a zero-effect perturbation or ZEP. This protocol, a variant of the paired pulse suggested in Wong et al. (2007), is only applied early in the integration process to avoid interference from the decision threshold. We shall seek critical values of λ for which ZEPs occur for each model. Figs. 1C and D illustrate both protocols.

2.5 Simulations and parameter values

Except for the SPB model, we first consider integration to a single decision threshold in the direction of the drift. This reduces the number of parameters, simplifies mathematical analyses, and helps isolate key effects. It applies to easy tasks in which drift rates are (relatively) high and errors rare. We subsequently relax this condition in our simulations. For the SPB model we first set I₁ = I₂, corresponding to difficult tasks and high error rates, and employ two thresholds. We then consider I₁ ≠ I₂ and a single threshold, representing easy tasks.

Signal-to-noise ratios were chosen such that the variance in first passage (decision) times is significant, but not so great that an unreasonably large number of trials is needed to average out the noise, and we selected perturbation amplitudes, durations and onset times such that both protocols cause small but significant effects (e.g. maximum changes of ~ 10% in mean decision times). In particular, durations were an order of magnitude smaller than mean decision times τ₀, as in Huk and Shadlen (2005). For the first protocol, onset times T were varied from stimulus onset at t = 0 until there were no significant effects on mean decision time. In the second protocol we require T ≪ τ₀. We set X₀ = 0 at t = 0 to represent unbiased initial conditions, and the remaining parameters are chosen to ensure ‘realistic’ behavior (e.g. the stable fixed point is above threshold for SOU; the unstable fixed point is below X₀ = 0 for UOU). Table 1 lists parameter values.

Table 1.

Parameters used in simulations

Model	b(Xt, t)	Parameters	ζ	ΔT	p
CD	b0 + b1	b0 = 5, σ = 2.449	20	0.4	5
TD	b0t + b1	b0 = 5, σ = 2.828	20	0.1	4
SOU	kXt + b0 + b1	k = -1, b0 = 8, σ = 1.414	7	0.4	2
UOU	kXt + b0 + b1	k = 0.2, b0 = 5, σ = 1.414	20	1	2
SPB	$\epsilon X_t+X_t^3-X_t^5+b_0+b_1$	ε = -0.3, 0.05, b0 = 0 or 0.004, σ = 0.01, τX = 20	±0.75	5	0.005 or 0.0008 (b0 = 0.004)

Drift-diffusion model with constant drift rate (CD), and with time-varying drift rate (TD); SOU (UOU) stable (unstable) Ornstein-Uhlenbeck processes; SPB: nonlinear model. See Eqs. 5-8 and protocols 9 and 10 for details. Here ζ is the distance from starting point X = 0 to decision threshold, and for SPB ε = -0.3 before stimulus appears at t = 0 and ε = 0.05 for t ≥ 0. Perturbation durations ΔT and amplitudes p are values used in the first perturbation protocol. Parameters for the linear models with smaller signal-to-noise ratios are specified in Section 3.3.

We use a forward Euler-Maruyama scheme (Higham, 2001), integrating a sample path until it hits the prescribed decision threshold and recording the corresponding first passage time, at which the trial ends and the next begins. After collecting an appropriately large ensemble, we extract the first and second moments of the first passage time from the samples. The simulation is run once for each one of the different perturbations. For the models CD, TD, SOU and UOU, the step size is 10^-3 of a time unit and the sample size is N = 10⁶, so that errors in the moments are of order $1∕\sqrt{N}\approx {10}^{-3}$. Since changes in moments due to the perturbations are of order 10^-2, these choices of time step and sample size reliably capture the effects of perturbation. The mean exit time for the unperturbed SPB model is ~ 80, so that a time step of 0.01 and a sample size is 10⁶ suffices. We ran noisy simulations of this model with ε = -0.3 for t < 0 to represent pre-stimulus activity, and then switched to ε = 0.05 for t ≥ 0 so that X = 0 undergoes a pitchfork bifurcation and becomes unstable, thus forcing a choice (see Fig. 1B. Magnitudes of ε and σ for SPB are much smaller than the corresponding b₀ and σ for the linear models because the $+X_t^3$ term accelerates solutions toward the thresholds).

The parameters are not independent, and their number can be reduced by one in all models by dividing Eq. 5 by b₀, and rescaling σ → σ/b₀, p → p/b₀, ζ → ζ/b₀, k → k/b₀ and the dynamical variable X_t → X_t/b₀. The resulting CD and TD models are described by 2 parameters while the SOU and UOU models require 3 and the SPB model 4. Two additional parameters describe the amplitude p and duration ΔT of the perturbation.

We focus on qualitative patterns of changes in response times and, to make unbiased comparisons, we adopt a dimensionless measure of the changes in means and standard deviations by normalizing the relative changes with respect to their unperturbed values.

3 Results

3.1 Single pulse perturbations of linear models

Using a single pulse perturbation with varying onset time T, we estimated changes in mean decision time for the four linear integrate-to-threshold models as described above, obtaining the results shown in Fig. 2. Imposed early during the integration process, a brief pulse in the same (opposite) direction as the drift rate significantly decreases (increases) mean decision times in all four models, as shown by the dashed (solid) curves. These effects fade as T increases due to a thresholding effect: for later perturbations, more trials have already crossed threshold, and so there are relatively fewer trials that are perturbed than unperturbed. Thus, when averaging over all trials, the influence of perturbations is progressively reduced. Note that unlike the work of Huk and Shadlen (2005); Wong et al. (2007), trials that have crossed threshold before and during perturbation are not excluded in the averaging process. This helps to reduce noise in the data, especially with late perturbation onset times.

Figure 2.

Changes in mean decision times under single pulse perturbations with varying onset times superimposed on positive drift rates for linear integrators CD, TD, SOU, and UOU (panels A-D). Vertical axes: normalized changes in mean first passage times (τ - τ₀)/τ₀; horizontal axes: normalized perturbation onset times T/τ₀. Changes due to negative pulses shown solid (-); changes due to positive pulses dashed (+). CD and TD show near-constant changes for T/τ₀ ≤ 0.5 that are substantially greater for CD (note vertical scale on panel B), SOU exhibits a maximum at T/τ₀ ≈ 0.5 (“optimal” perturbation), and UOU shows monotonic decrease. effects decrease in all cases as T approaches and passes τ₀ due to thresholding (see text).

In Appendix A, we prove that this basic pattern - positive pulses advance mean decision times and negative pulses retard them - must hold provided that the perturbation occurs sufficiently early. The proof applies to a broad class of nonlinear systems including the SPB model of Eq. 8, provided that ε > 0 and thresholds lie inside the region in which drift magnitude increases with X (i.e., well below the activation levels of the stable fixed points).

Note that the drift-diffusion models (CD, TD) and the OU models (SOU, UOU) respond distinctly to early perturbations: the former exhibiting almost constant changes in mean decision times (panels A and B), and the latter respectively showing increasing and decreasing effects (panels C and D). Moreover, due to thresholding, SOU is alone in having an onset time for which the perturbative effect is maximized.

We next investigate how standard deviations are affected by perturbation onset time: see Fig 3. Richer patterns arise than those of Fig. 2. Both CD and TD exhibit similar near-constant changes for early onsets, but the directions of changes for TD are opposite to those for CD, and the curves cross before reaching maxima. Standard deviations for SOU have a similar pattern to its mean, but UOU exhibits initial decreases followed by increases to a peak, unlike its mean. Optimal onset times exist for which the four standard deviations are maximally affected, and all four cases show distinct patterns.

Figure 3.

Changes in standard deviations of decision times under single pulse perturbations with varying onset times. All cases exhibit a maximum in normalized standard deviation (std(τ)-std(τ₀))/std(τ₀), but only UOU has a (shallow) minimum. effects of early pulses on all four integrators are distinct, as described in text. Format and conventions are as in Fig. 2.

In the TD model considered above we assumed that the perturbation is not affected by the time-dependent gain b(t). This may not hold if the perturbation enters via the same sensory pathway as the stimuli. In Appendix B we consider a case in which perturbation and drift are affected in the same way. We show that this modified TD model differs qualitatively only for early perturbation onset, for which it yields reduced effects.

3.2 Zero-effect perturbations of linear models

We now consider the zero-effect protocol of Eq. 10, appealing to the fact that, in the noise-free case, the first passage time is a monotone function of the ratio λ of the amplitudes of the opposing pulses. The critical λ = λ* that determines a ZEP is therefore given by the unique zero of

$$\tau \left(\lambda \right)={\tau }_0,$$ (11)

where τ(λ) describes the functional dependence of passage time on λ. Mean first passage times appear to remain monotone with respect to λ when noise is included. Asymptotic methods can be used to approximate passage time moments for small perturbations (Lindner, 2004), but the lack of compact formulae for passage time distributions (PDFs) in most cases, including OU processes, and the need for double integrations (Lindner, 2004) makes this approach generally intractable.

However, ZEPs may be analytically approximated for early onset times T with T’ := T + ΔT ≪ τ₀, such that the probability of threshold crossing prior to T’ is negligible. In this case a sufficient condition is that the PDFs of the perturbed and unperturbed processes are identical at T’, since for t > T’, both processes are governed by the same drift and noise. They are therefore indistinguishable in the limit T, ΔT → 0. Before deriving explicit expressions for λ* under this assumption we present the results of numerical simulations with an intuitive explanation.

Since the unperturbed SDE (5) is independent of the current state X_t in the CD and TD cases, we expect that antisymmetric pulses with λ* = 1, in which the pulses precisely cancel, will produce ZEPs. In contrast, responses to inputs decay with time for SOU, implying that the first pulse must be larger than the second for their net effect to cancel at perturbation o set t = T’ (λ* > 1). For UOU the reverse should hold (λ* < 1). Fig. 4 confirms that this is the case. Table 2 lists the parameters used in the simulations; note that the early onset time conditions are only weakly satisfied (T’ ranges from 25 - 40% of τ₀).

Figure 4.

Zero-effect perturbations of linear integrate-to-threshold models CD, TD, SOU, UOU (panels A-D). Vertical axes: normalized change in mean first passage times (τ - τ₀)/τ₀; horizontal axes: relative amplitude of first pulse compared to second. Zeroes of solid lines (-, positive pulse followed by negative pulse, p < 0) and dashed lines (+, negative pulse followed by positive pulse, p > 0) identify ZEPs at λ* ≈ 0.9987, 1.0039 (CD); 0.9952, 1.0004 (TD); 1.2271, 1.2230 (SOU) and 0.9066, 0.9034 (UOU).

Table 2.

Parameters used in ZEP simulations

Model	τ0	std(τ0)	T	ΔT	p	λ* (thy.)	λ* (sim.)
CD	4.005	0.960	0.5	0.5	5	1	1.0013
TD	3.145	0.420	0.5	0.5	5	1	0.9978
SOU	1.831	0.374	0.1	0.4	2	1.2214	1.2251
UOU	2.954	0.142	0.2	1.0	2	0.9048	0.9050

τ₀ and std(τ₀) are the rstand second moments of the passage time without perturbation, T and ΔT are the onset time and total duration of the perturbation; p is the amplitude of the second pulse and λ* (thy.) and λ* (sim.) are the ZEP pulse ratios predicted by the theory and obtained by averaging p < 0 and p > 0 results of Fig. 4.

To derive explicit approximations for λ*, we use a comparison method similar to that of Appendix A. Let X_t denote the unperturbed process and Z_t the perturbed process. If X₀ = Z₀ at t = 0, then X_t = Z_T at the onset time T. When the perturbation ends at t = T’ the solution of the unperturbed SDE of Eq. 5 with constant drift rate b₀ is

$$X_{T\prime }=(X_T+b_0∕k)e^{k\Delta T}-b_0∕k+\sigma {\int }_T^{T+\Delta T}{e}^{k(T^{\prime }-s)}d{W}_s,$$ (12)

while the perturbed system satisfies

$$Z_{T\prime }=(X_T+b_0∕k)e^{k\Delta T}-b_0∕k+{\int }_T^{T+\Delta T}{e}^{k(\Delta T-s)}{b}_1\left(s\right)ds+\sigma {\int }_T^{T+\Delta T}{e}^{k(T^{\prime }-s)}d{W}_s.$$ (13)

These expressions well approximate the true activity levels for models CD, SOU and UOU with absorbing thresholds only when both processes have low probability of hitting the threshold during the interval [0, T’], but this condition holds for T’/τ₀ ≪ 1, since the integrated drift term is almost zero at early times. Eqs. 12 and 13 differ in the term $e^{k\Delta T}{\int }_0^{\Delta T}{e}^{-kt}{b}_1\left(t\right)dt$, which enters Z_T’ due to the action of the perturbation b₁ during the interval [T, T’]. The ZEP condition for arbitrary perturbations is therefore

$${\int }_T^{T+\Delta T}{e}^{-kt}{b}_1\left(t\right)dt=0,$$ (14)

which for the piecewise constant pulses of Eq. 10 implies that:

$${\lambda }^{\ast }=\frac{e^{-k\Delta T∕2}-e^{-k\Delta T}}{1-e^{-k\Delta T∕2}}=e^{-k\Delta T∕2}.$$ (15)

Hence for k > 0 (UOU), λ* < 1, while for k < 0 (SOU), λ* > 1. In the special case of CD, k = 0 and λ* = 1.

For TD the drift term is time-dependent but the linear term kX_t is absent and so the corresponding solutions of the SDEs are

$$X_{T^{\prime }}={\int }_T^{T+\Delta T}{b}_0\left(s\right)ds+\sigma {\int }_T^{T+\Delta T}d{W}_s$$

and

$$Z_{T^{\prime }}={\int }_T^{T+\Delta T}{b}_0\left(s\right)ds+{\int }_T^{T+\Delta T}{b}_1\left(s\right)ds+\sigma {\int }_T^{T+\Delta T}d{W}_s.$$

In this case, a ZEP must satisfy

$${\int }_T^{T+\Delta T}{b}_1\left(t\right)dt=0,$$ (16)

which yields λ* = 1 for the pulses of Eq. 10, as for CD. This result holds for all time-dependent drift rates, including those used in Smith et al. (2004); Ditterich (2006b); Churchland et al. (2008).

Summarizing, ZEPs occur for the CD and TD models when the opposing pulse amplitudes are equal (λ* = 1), but for SOU and UOU λ* = e^-kΔT/2 is larger and smaller than one, respectively. The ratios λ* predicted by Eq. 15 for the parameters of Table 2 are given in the final column of the Table. In all four cases they agree well with the zero crossings of the mean first passage times in Fig. 4, in spite of the fact that T’/τ₀ ≈ 0.25 - 0.4 is not very small.

3.3 Signal-to-noise ratio can influence perturbation effects

To what extent do the results of Section 3.1 hold when signal-to-noise ratios are reduced and a second threshold is added to track errors? To investigate this question we select new parameter values according to the criteria of Section 2.5.

For the CD model, we increase σ from 2.449 to 2.828 and set thresholds at ζ = ±5 instead of ±20, yielding an error rate of 5%. This introduces more complex behavior in which correct and error trials respond to the perturbations in different manners, as shown in Fig. 5A. The combined averages (as computed in Huk and Shadlen (2005) and Wong et al. (2007), see black solid and dashed curves) preserve some features of previous results: for early perturbations the direction of changes in mean RT agrees with Fig. 2A, but the approximately-constant change in mean decision time early in the trial is replaced by a monotonic decline. Increased noise yields earlier threshold crossings, advancing the thresholding effect and masking the signatures of Fig. 2A. Moreover, the change in mean error RTs can reverse sign for late perturbation onsets. We checked this seemingly counterintuitive phenomenon numerically by fixing the random generating seed for noise in a sample trial, finding that perturbation of long RT trials can change an impending error to a correct choice (data not shown).

Figure 5.

Changes in mean decision times for linear models with lower signal-to-noise ratios. Perturbation protocol is as in Fig. 2. Green (red) curves: perturbation p in the same (opposite) direction as drift rate b₀; solid: correct choices; dash-dotted: error choices. Solid (dashed) black curves are averages of correct and error choices with perturbation in the same (opposite) direction as drift rate. Grey (black) curves in panel A are for error rate of 0.02% (5%).

Increasing the noise level to σ = 7.071 with thresholds ζ = ±20 produces an error rate of 10% in the TD model and changes in mean RTs for correct and error choices similar to those of CD. However, averaged over correct and error choices, overall changes in mean RT exhibit a pattern similar to that of Fig. 2B (see black curves in Fig. 5B). In this respect TD maintains its signature more robustly with increasing error rate than CD, although the effects are progressively masked.

For the SOU model, we increase σ to 6.325, yielding an error rate of 17%. The optimal perturbation onset time for changes in mean RT averaged over correct and error trials is masked (compare Fig. 2C with black curves in Fig. 5C). This error rate is relatively high, but we have checked that for error rates as small as 3% SOU model features can also be masked, indicating that, like the CD model, SOU is sensitive to noise.

For the UOU model, more parameters must be adjusted to obtain reasonable RT changes. Here we set σ = 2, ζ = ±10, k = 0.02 and b₀ = 0.5, yielding a high error rate of 23.5%, but similar trends occur for error rates from 0.1% to 35%. Fig. 5D shows that these are comparable to the previous UOU results with high signal-to-noise ratio. Overall, we are unable to distinguish between noise-masking effects and intrinsic features of the UOU integrator.

Using the same parameters, we find that changes in standard deviations for all the linear models show optimal perturbation onset times (Fig. 6). The CD, SOU and UOU models are mutually indistinguishable (cf. Figs. 6A, C and D), but TD exhibits a crossing effect (Fig. 6B) similar to Fig. 3B.

Figure 6.

Changes in standard deviations of decision times for the linear models with lower signal-to-noise ratios. Format is as in Figs. 3 and 5.

3.4 Perturbations of the nonlinear model

We next apply both perturbation protocols to the nonlinear SPB model. The mean and standard deviation of decision times for the unperturbed system are 80 and 22 time units, respectively. The parameters used for the first protocol are given in Table 1, while for the ZEP we set p = 0.005, T = 30 and T = 5, consistent with application early in the accumulation process. Figs. 7A and B show that the means and standard deviations of decision times behave like those for UOU (cf. panels D of Figs. 2 and 3), decreasing monotonically and approaching zero as onset times increase. ZEPs occur in the pulse ratio range 0.8 < λ* < 1 for both mean and standard deviation (Figs. 7C and D). Assuming a linear change in mean in Fig. 7C, we find that λ* ≈ 0.87.

Figure 7.

Perturbations of means (left column) and standard deviations (right column) of first passage times for the SPB model. Panels A, B: effects of single pulse perturbations with varying onset times; vertical axes as in Figs. 2 and 3. -+ denotes effect of negative pulse at positive threshold, +- denotes effect of positive pulse at negative threshold, etc. Due to symmetry, -+ and +- have same effect (solid curves) as do ++ and -- (dashed curves). Panels C, D: responses to zero-effect perturbations; vertical axis in C as in Fig. 4; vertical axis in D shows normalized change in standard deviation as in Fig. 3.

To compare with the behavior of the linear systems in the single threshold case, we now modify the SPB model so that it makes as few errors as they do. This is done by changing b₀ from 0 to 0.004, which breaks the symmetry of the subcritical pitchfork bifurcation (Fig. 1B) so that decisions favor one choice, representing easy tasks (cf. Roxin and Ledberg (2008)); all other parameters remain the same. Fig. 8 shows that the basic behavior of Figs. 7C and D is preserved: effects on both mean and standard deviation of RTs decrease monotonically as perturbation onset time increases. While the effects on mean passage times of SPB are much like those for UOU, the fact that standard deviations of all four linear models exhibit maxima (Fig. 3) potentially allows us to distinguish SPB from them. The effects of ZEPs in this biased regime is similar to that shown in Figs. 7C and D (data not shown).

Figure 8.

Changes in mean (A) and standard deviation (B) of decision times of the SPB model with high signal-to-noise ratio. All choices are correct. Perturbation protocols and plot formats are as in Figs. 7A and B.

4 Discussion

In this work, we have investigated how several simple stochastic integrate-to-threshold decision-making models respond to short perturbations. We focus on two-alternative forced-choice reaction time tasks for which general firing rate models with three coupled neural populations can be reduced to one-dimensional integrate-to-threshold systems. These reduced systems, which may be pure drift-diffusion or Ornstein-Uhlenbeck processes, can be compared analytically and in simulations that require few parameters.

We examined two perturbation protocols, the first being a brief pulse with variable onset time, and the second a pulse-antipulse pair whose amplitude ratio can be adjusted to produce minimal effects (the zero-effect perturbation, ZEP). The simulations of §3 (Figs. 2 and 3) show that the changes in means and standard deviations of decision times allow both perturbation protocols to distinguish between drift-diffusion models and stable and unstable Ornstein-Uhlenbeck processes, provided that perturbations are delivered sufficiently early in the integration process and the signal-to-noise ratio is not too small. Changes in standard deviations can also assist in distinguishing drift-diffusion models with constant drift from those with time-dependent drift rates (cf. Figs. 3A and B), although differences are small and may not be detectable experimentally. (As Ditterich (2006a) notes, entire reaction time distributions can further assist in this.) For early perturbations the ZEP conditions can be approximated analytically, making predictions that are confirmed by computer simulations (Fig. 4). While general analytical expressions for systems with finite noise appear elusive, in Appendix A we prove inequalities for more general nonlinear systems that partially explain our simulation results.

Although we focus on linear models, our methods extend to other systems, and we include results for a nonlinear model that capture the dynamics near a subcritical pitchfork bifurcation which is typical of reduced population models (Wang, 2002; Wong and Wang, 2006; Roxin and Ledberg, 2008). As the results of Appendix A suggest, it behaves in a manner similar to the unstable OU model under both perturbation protocols, although perturbative effects on standard deviations may allow distinctions to be made (Figs. 7-8). We have also confirmed that the TD model’s behavior remains similar with other increasing drift rates (e.g. b(t) ~ b₀t²; simulation results not shown).

These results reinforce the simulations and claims in Wong et al. (2007) that time-varying perturbations can in principle reveal integration mechanisms. However, some parameter ranges frustrate clear-cut comparisons among models. The first occurs for low signal-to-noise ratios and high error rates (cf. Figs 5-6), and so can be mitigated by using perturbations with more easily discriminated stimuli. Secondly, if there is a ‘dead time’ between stimulus onset and the start of evidence integration, the CD and TD models can display similar signatures to the SOU model, even with high signal-to-noise ratios.

Since perturbations can be delivered through the senses as well as by direct electrical stimulation, the method can be noninvasive and is therefore appropriate for human subjects. Furthermore, unlike fitting neuronal firing rates, both perturbation protocols can distinguish linear and nonlinear UOU-type models (Wang, 2002; Wong and Wang, 2006; Roxin and Ledberg, 2008; Wong and Huk, 2008) from drift-diffusion models with an ‘urgent’ time-dependent increase in drift rate (Smith et al., 2004; Ditterich, 2006a; Churchland et al., 2008). However, care is required in designing such experiments. Although a high signal-to-noise ratio can in principle help identify integrators, the reaction times can become too short, masking the signature behaviors of the integrators with the ‘thresholding effect’. A possible solution could be to prolong the response deadline in relatively simple reaction time tasks and analyze only data from long RT trials.

Perturbation inputs to decision-making circuits may originate in a single sensory pathway or in other brain areas, as in multisensory and ‘top-down’ inputs (e.g. due to attention (Smith et al., 2004; Liu et al., 2008)) which could modulate a decision throughout a sensorimotor pathway. The general notion of perturbation developed in the reduced model context extends to such modulatory inputs. The methods may also generalize to other tasks that recruit neural integrators (Goldman et al., 2008), such as interval timing (Shea-Brown et al., 2006).

Appendix A: Rigorous estimates on first passage times

In this appendix we analyze short single pulse perturbations, First in the noiseless limit, and then with additive noise. The inequalities proved here suggest explanations for the monotonicity of mean first passage times of SOU and UOU processes with respect to early onset times (cf. Figs. 2C and D); moreover, they apply to more general, nonlinear one-dimensional systems such as SPB with thresholds |ζ| sufficiently close to X_t = 0.

The noiseless limit

As noise amplitude tends to zero, the SDE Eq. 5 becomes an ordinary differential equation (ODE):

$$\frac{dX}{dt}≔\stackrel{.}{X}=f\left(X\right).$$ (17)

Consider two compact perturbations p and $\tilde{p}$ with the same amplitude profile g(t) and duration ΔT, applied at different onset times T₁ and ${\tilde{T}}_1$:

$$p\left(t\right)=1_{[T_1,T_2]}\left(t\right)g(t-T_1),\tilde{p}\left(t\right)=1_{[{\tilde{T}}_1,{\tilde{T}}_2]}\left(t\right)g(t-{\tilde{T}}_1)$$ (18)

(here 1_[T1,T2] denotes the indicator function that takes the value 1 for T₁ < t < T₂ and zero otherwise). We choose $0<T_1<T_2<{\tilde{T}}_1<{\tilde{T}}_2$ and $T_2-T_1={\tilde{T}}_2-{\tilde{T}}_1=\Delta T$ such that p and $\tilde{p}$ do not overlap and that neither perturbed solution reaches the threshold X = ζ before $t={\tilde{T}}_2$. We shall compare solutions Y_t and ${\tilde{Y}}_t$ of the corresponding perturbed ODEs,

$$\stackrel{.}{Y}=f\left(Y\right)+p\left(t\right)\text{and}\dot{\tilde{Y}}=f\left(\tilde{Y}\right)+\tilde{p}\left(t\right),$$ (19)

started at the same initial condition $Y_0={\tilde{Y}}_0=0$, to show how the signs of f’ and g determine their threshold passage times τ and $\tilde{\tau }$. We allow smooth nonlinear functions f, but assume that f(X) > 0 for all 0 ≤ X < ζ and that g does not change sign. The fact that solutions of scalar ODEs cannot cross each other is a key tool.

Lemma 1

Let X_t(x) denote the solution of Eq. 17 with initial value x. Then X_t(x₁) < X_t(x₂) for any t > 0 if and only if x₁ < x₂.

Proof

This follows from the uniqueness of solutions of ODEs and the ordering of points in a one-dimensional phase space.

Since neither perturbation acts after $t={\tilde{T}}_2$, and neither solution has reached threshold at ${\tilde{T}}_2$, Lemma 1 implies that $\tau <\tilde{\tau }$ if and only if $Y\left({\tilde{T}}_2\right)>\tilde{Y}\left({\tilde{T}}_2\right)$. We now prove the main result summarized in Table 3, which is a corollary of Proposition 2.

Table 3.

Monotonicity of first passage dependence on onset time for four sign combinations of f’ and g, in the noise-free limit. L is some constant

	f’ ≥ L > 0	f’ ≤ -L < 0
g > 0	$\tau <\tilde{\tau }$	$\tau >\tilde{\tau }$
g < 0	$\tau >\tilde{\tau }$	$\tau <\tilde{\tau }$

Proposition 2

Assume that the function g(t) in Eqs. 18 is strictly positive for t ∈ (0, ΔT). Then for some constant L (a) If f’ ≥ L > 0, then $Y\left(t\right)>\tilde{Y}\left(t\right)$ for all $t\ge {\tilde{T}}_2$; (b) If f’ ≤ -L < 0, then $Y\left(t\right)<\tilde{Y}\left(t\right)$ for all $t\ge {\tilde{T}}_2$.

Proof of part (a)

We need only establish the inequalities at $t={\tilde{T}}_2$, since no perturbations occur after this time. We compare solutions of Eqs. 19 using the fact that f(X) > 0 and f’(X) ≥ L > 0 for X ≥ 0 implies that $f\left(Y\right)-f\left(\tilde{Y}\right)\ge L(Y-\tilde{Y})$. Letting $u\left(t\right)=Y\left(t\right)-\tilde{Y}\left(t\right)$ we have

$$\stackrel{.}{u}=f\left(Y\right)-f\left(\tilde{Y}\right)+p\left(t\right)-\tilde{p}\left(t\right)\ge Lu+p\left(t\right)-\tilde{p}\left(t\right).$$ (20)

Since the solutions coincide until the first perturbation occurs at t = T₁, u(T₁) = 0, and integrating inequality 20 over the interval [T₁, T₂], on which p(t) = g(t - T₁) and $\tilde{p}\left(t\right)\equiv 0$, yields:

$$u\left(T_2\right)\ge {\int }_{T_1}^{T_2}{e}^{L(T_2-t)}g(t-T_1)dt>0.$$ (21)

Both p(t) and $\tilde{p}\left(t\right)\equiv 0$ on $[T_2,{\tilde{T}}_1]$, and integration of 20 and use of 21 gives

$$u\left({\tilde{T}}_1\right)\ge u\left(T_2\right)e^{L({\tilde{T}}_1-T_2)}\ge e^{L({\tilde{T}}_1-T_2)}{\int }_{T_1}^{T_2}{e}^{L(T_2-t)}g(t-T_1)dt>0.$$ (22)

Finally, using the fact that p(t) ≡ 0 on $[{\tilde{T}}_1,{\tilde{T}}_2]$, we integrate 20 again to obtain

$$\begin{array}{cc}\hfill u\left({\tilde{T}}_2\right)& \ge u\left({\tilde{T}}_1\right)e^{L({\tilde{T}}_2-{\tilde{T}}_1)}-{\int }_{{\tilde{T}}_1}^{{\tilde{T}}_2}{e}^{L({\tilde{T}}_2-t)}g(t-{\tilde{T}}_1)dt\hfill \\ \hfill & \ge [e^{L({\tilde{T}}_2-T_2)}-1]{\int }_0^{\Delta T}{e}^{L(\Delta T-t)}g\left(t\right)dt>0,\hfill \end{array}$$ (23)

where we also use inequality 22 and the fact that the additive perturbations of Eq. 18 are identical over the time intervals [T₁, T₂] and $[{\tilde{T}}_1,{\tilde{T}}_2]$ to rewrite the integral term. Hence $Y\left({\tilde{T}}_2\right)>\tilde{Y}\left({\tilde{T}}_2\right)$ and $\tau <\tilde{\tau }$.

Proof of part (b)

We again compare solutions of Eqs. 19 but the direction of the inequalities is now reversed, so that we are in essence using Gronwall’s inequality (Guckenheimer and Holmes, 1983). In this case Eq. 20 becomes

$$\stackrel{.}{u}\le -Lu+p\left(t\right)-\tilde{p}\left(t\right),$$ (24)

inequality 22 becomes

$$u\left({\tilde{T}}_1\right)\le u\left(T_2\right)e^{-L({\tilde{T}}_1-T_2)}\le e^{-L({\tilde{T}}_1-T_2)}{\int }_{T_1}^{T_2}{e}^{-L(T_2-t)}g(t-T_1)dt<0,$$ (25)

and the final inequality reads

$$u\left({\tilde{T}}_2\right)=Y\left({\tilde{T}}_2\right)-\tilde{Y}\left({\tilde{T}}_2\right)\le -[1-e^{-L({\tilde{T}}_2-T_2)}]{\int }_0^{\Delta T}{e}^{-L(\Delta T-t)}g\left(t\right)dt<0.$$ (26)

This implies that $\tau >\tilde{\tau }$, as claimed. Proofs are similar for g < 0.

Extension to noisy systems

The conclusions of Proposition 2 and Table 3 may be extended to apply to expected passage times for SDEs with additive noise. Specifically, replacing Eqns. 19 by

$$d{Y}_t=[f\left(Y_t\right)+p\left(t\right)]dt+\sigma d{W}_t,$$ (27a)

$$d{\tilde{Y}}_t=[f\left({\tilde{Y}}_t\right)+\tilde{p}\left(t\right)]dt+\sigma d{\tilde{W}}_t,$$ (27b)

we have the following result:

Proposition 3

Assume that the function g(t) in Eqs. 18 is strictly positive for t ∈ (0, ΔT). Then for some constant L (a) If f’ ≥ L > 0, then $\mathbb{E}\left[\tau \right]<\mathbb{E}\left[\tilde{\tau }\right]$; (b) If f’ ≤ -L < 0, then $\mathbb{E}\left[\tau \right]>\mathbb{E}\left[\tilde{\tau }\right]$, where τ and $\tilde{\tau }$ denote the first passage times for Eqns. 27a and 27b respectively.

Proof

We first compare solutions Y_t and ${\tilde{Y}}_t$ obtained from Eqns. 27a and 27b under the same sample noise path with increments $d{\tilde{W}}_t\equiv d{W}_t$. Since the stochastic integral ${\int }_0^{t}d{W}_t$ is almost surely continuous (Feller, 1957), and the integrated noise terms cancel, the comparison of solutions using $u\left(t\right)=Y_t-{\tilde{Y}}_t$ proceeds as in the deterministic case above, and we conclude that $\tau <\tilde{\tau }$ for g > 0 and f’ ≥ L > 0, and $\tau >\tilde{\tau }$ for g > 0 and f’ ≤ -L < 0. Finally, averaging each process over an ensemble of sample paths, we may conclude that the mean first passage times of the two processes satisfy similar inequalities to those in the top row of Table 3: $\mathbb{E}\left[\tau \right]<\mathbb{E}\left[\tilde{\tau }\right]$ and $\mathbb{E}\left[\tau \right]>\mathbb{E}\left[\tilde{\tau }\right]$ respectively. A similar argument applies to the case g < 0.

Appendix B: A TD model with time-varying perturbation amplitude

In the body of the paper, perturbations of the TD model were assumed unaffected by the time-dependent gain. Here we extend the simulations to a simple case in which perturbation and stimulus (drift) share the same overall linear increase in amplitude. Specifically, we assume the dynamics to be governed by

$$d{X}_t=(b_0+p\left(t\right))tdt+\sigma d{W}_t.$$ (28)

Otherwise, we change only the perturbation amplitude, taking p = ±2 instead of 4. Fig. 9A shows that the changes in mean and standard deviations of exit time for this model remain similar to the previous TD model, but with smaller perturbative effects near the beginning of the trial (quite similar to a SOU model) due to the monotonic time-dependence of perturbation amplitude. However, although the previous pattern of mean exit times no longer applies, the behavior remains qualitatively dissimilar to the CD, UOU and SPB models.

Figure 9.

Normalized changes in mean (A) and standard deviation (B) of first passage times for single pulse perturbations with varying onset times applied to a variant of the TD model. Vertical axes and curve conventions are as in Figs. 2 and 3.