A Readout Mechanism for Latency Codes

Abstract

Response latency has been suggested as a possible source of information in the central nervous system when fast decisions are required. The accuracy of latency codes was studied in the past using a simplified readout algorithm termed the temporal-winner-take-all (tWTA). The tWTA is a competitive readout algorithm in which populations of neurons with a similar decision preference compete, and the algorithm selects according to the preference of the population that reaches the decision threshold first. It has been shown that this algorithm can account for accurate decisions among a small number of alternatives during short biologically relevant time periods. However, one of the major points of criticism of latency codes has been that it is unclear how can such a readout be implemented by the central nervous system. Here we show that the solution to this long standing puzzle may be rather simple. We suggest a mechanism that is based on reciprocal inhibition architecture, similar to that of the conventional winner-take-all, and show that under a wide range of parameters this mechanism is sufficient to implement the tWTA algorithm. This is done by first analyzing a rate toy model, and demonstrating its ability to discriminate short latency differences between its inputs. We then study the sensitivity of this mechanism to fine-tuning of its initial conditions, and show that it is robust to wide range of noise levels in the initial conditions. These results are then generalized to a Hodgkin-Huxley type of neuron model, using numerical simulations. Latency codes have been criticized for requiring a reliable stimulus-onset detection mechanism as a reference for measuring latency. Here we show that this frequent assumption does not hold, and that, an additional onset estimator is not needed to trigger this simple tWTA mechanism.

Introduction

Neuronal response latency has been shown to be tuned to various features of external stimuli across many sensory modalities including the somatosensory (Panzeri et al., 2001; Johansson and Birznieks, 2004; Panzeri and Diamond, 2010), auditory (Brugge et al., 1996, 2001; Klug et al., 2000; Leibold and van Hemmen, 2005; McAlpine, 2005; Zhou et al., 2005; Chase and Young, 2007; Joris and Yin, 2007; Goodman and Brette, 2010; Grothe et al., 2010; Ashida and Carr, 2011; Grothe and Koch, 2011; Lüling et al., 2011; Zohar et al., 2011; Shamir, 2014), and visual systems (Hubel and Wiesel, 1968; Gawne et al., 1996; Van Rullen and Thorpe, 2001; Gollisch and Meister, 2008; Shriki et al., 2012; Shamir, 2014). In addition, latency tuning has been observed in various stations along the information processing pathway from the receptor level (Hall et al., 1995; Van Rullen and Thorpe, 2001; Gollisch and Meister, 2008; McGinley et al., 2012), the brainstem (Klug et al., 2000; Joris and Yin, 2007; Grothe and Koch, 2011; Zohar et al., 2011) and up to the cerebral cortex (Hubel and Wiesel, 1968; Brugge et al., 1996, 2001; Shriki et al., 2012). For this reason, response latency has been suggested as a possible source of information in cases where fast decisions are required (Gawne et al., 1996; Gautrais and Thorpe, 1998; Van Rullen and Thorpe, 2001; Johansson and Birznieks, 2004; Chase and Young, 2007; Gollisch and Meister, 2008; Shamir, 2009, 2014; Goodman and Brette, 2010; Panzeri and Diamond, 2010; Zohar et al., 2011; Shriki et al., 2012).

The accuracy of latency codes has been investigated using the framework of a simple latency based competitive readout, the “temporal-winner-take-all” (tWTA), in a modeling study (Shamir, 2009) and more recently in the auditory (Zohar et al., 2011) and the visual systems (Shriki et al., 2012). The tWTA estimates the stimulus on the basis of the preferred stimulus of the neuron (or population) that fired first (or reached a certain decision threshold). The utility of the tWTA is that it is sufficiently simple and well-defined to enable analytical investigation of readout speed and accuracy. It was shown that the tWTA can account for fast and accurate discrimination between a small number of alternatives. However, it remains unclear how the central nervous system implements such a readout, if at all. This raises crucial questions as to the utility of latency coding in the brain.

The conventional rate winner-take-all (WTA) readout that estimates the stimulus based on the preferred stimulus of the neuron that fired the most spikes (rather than the first as in tWTA) has been widely used and studied in neuroscience (Fukai and Tanaka, 1997; White et al., 1998; Jin and Seung, 2002; Laing and Chow, 2002). There is a general consensus that the computation of the WTA decision is based on reciprocal inhibition between groups of neurons with similar preferences.

Here we show that the basic WTA architecture is also sensitive to the temporal order of its inputs such that it is able to implement the tWTA readout. This paper is organized as follows. First we define the basic reciprocal inhibition architecture of the tWTA mechanism and analyze the system using the framework of a rate toy model to facilitate the analytical investigation. Then, we study the generalization of our analysis to a model of spiking neurons in a numerical study of conductance-based neurons. Finally, we discuss the issue of estimating the stimulus onset time and the implications of the findings.

Methods

Spiking neurons model

In the numerical simulations of spiking neurons we used the following conductance-based model with typical parameters taken from Shriki et al. (2003) and Shamir et al. (2009).

$$\begin{array}{r}C\frac{d{V}_i}{dt}=I_i^{leak}+I_i^{net}+I_i^{ext}+I_i^{active}\end{array}$$

where $I_i^{leak},I_i^{net},I_i^{ext},I_i^{active}$ denote the leak, reciprocal inhibition, upstream input and active currents, respectively, and C = 1μF/cm² is the membrane capacitance. The currents obey $I_i^{leak}=g_L(E_L-V_i(t)),I_{i\in \left\{po{p}_A\right\}}^{net}=\frac{2{\text{g}}_{\text{S}}}{\text{N}}{\displaystyle \sum _{j\in \left\{po{p}_B\right\}}{s}_{ij}\left(t\right)\left(E_s-V_i\right)},I_i^{active}=I_{Na}+I_K+I_A$ with $I_{Na}=g_{Na}{m}_{\infty }^{3}h(V-E_{Na})$, $I_K=g_K{n}^4(V-E_K)$, $I_A=g_A{a}_{\infty }^{3}b(V-E_K)$. The parameters E_L, E_s, E_Na, E_K denote the reversal potentials of the ionic currents in m V and are −65, −80, 55, −80^, respectively. Conductance is given in units of ms/cm², and g_L, g_Na, g_K, g_A are 0.05, 100, 40, 20^, respectively. The value of g_S is discussed below. The variables s_ij, m, h, n, a, b obey the following dynamics;

$$\begin{array}{rcl}d{s}_{ij}/dt& =& 1+(\text{tanh}\left(V_{pre}/10\right)/2)(1-s)/({\tau }_R)-s/{\tau }_D,db/dt\\ =& \left(b_{\infty }-b\right)/{\tau }_b\end{array}$$

dx/dt = (x_∞ − x)τ_x for x = n, h, b with b_∞ = 1/(exp((V + 80)/6) + 1), a_∞ = 1/(exp(−(V + 50/20)) + 1) and x_∞ = α_x/α_x + β_x for x = m, h, n.

Time constants are measured in ms, andτ_b, τ_D, τ_R are 20, 10, 0.5^, respectively, and τ_x = ϕ/α_x + β_x for x = m, h, n, where ϕ = 0.1 and

$$\begin{array}{l}{\beta }_m=4exp\left(-(V+55)/18\right),\end{array}$$

$${\alpha }_m=-(0.1(V+30)/(\exp (-0.1(V+30)-1),$$

$$\begin{array}{l}{\beta }_h=1/(exp\left(-0.1\left(V+14\right)\right)+1),\end{array}$$

$$\begin{array}{l}{\alpha }_h=0.07exp\left(-(V+44)\right),\end{array}$$

$$\begin{array}{l}{\alpha }_n=-(0.01\left(V+34\right))/(exp\left(-0.1\left(V+34\right)\right)-1),\end{array}$$

$$\begin{array}{l}{\beta }_n=0.125exp\left(-(V+44)/80\right).\end{array}$$

$I_{x,i}^{ext}(t)=r\Theta \left(t-\left[T+{\tau }_x+{\delta }_{x,i}\right]\right)+{\xi }_{x,i}(t)$ is the input current to neuron i in population x ∈ {1, 2} from the upstream population, where Θ is the Heaviside step function, T is an absolute delay in the response and τ_x is stimulus selective delay (see A with τ_i = 0 at the preferred stimulus, and τ_i = τ). The input current contained two sources of noise. One was a random jitter in the input latency to each neuron, denoted by δ_{x, i}, that were taken to be i.i.d. Gaussian random variables with zero mean and standard deviation Δ, ${\delta }_{x,i}~N(0,{\Delta }^2)$. The second source was an additive Gaussian white noise term, ξ_{x, i}(t), that was drawn i.i.d with ξ(t) ~ N(0, Ξ²). In the numerical simulations we took r = 5mA/cm², and the white noise, ξ_{x, i}(t), was binned at 1ms time intervals with Ξ = 5mA/cm².

Results

We first studied the tWTA readout mechanism using the framework two-alternative forced choice discrimination in a rate model. The readout mechanism receives two inputs, I₁ and I₂. These inputs represent the activities of two upstream populations of neurons responding to an external stimulus, each of which is characterized by its own preferred stimulus. For example, I₁ can be thought of as the response of a population of inferior colliculus auditory neurons with a preferred sound source azimuth to the right of the animal, whereas I₂ is the response to the left. The response latency of the two inputs is assumed to be tuned to the external stimulus. The task of the readout mechanism is to infer the stimulus based on the inputs it receives. For purposes of studying the ability of the readout to select according to the latency of its inputs, we further assume that both inputs have the same strength; however, the input corresponding to the “correct choice” responds faster in some sense. Specifically, we modeled the inputs as a step function, I_i(t) = Θ(t − T − τ_i), where Θ is the Heaviside step function, T is an absolute delay in the response and τ_i is the stimulus selective delay (latency tuning: τ_i = 0 at the preferred stimulus, and τ_i = τ > 0 otherwise); Figure 1A.

Figure 1.

Schematic illustration of the model. (A) The input model: we consider input from two upstream populations that encode the stimulus identity by their response latency. The traces show the rate of the two populations (in different colors) as a function of time, given the preferred stimulus of population 1 is presented at time t = 0. (B) Network architecture. The upstream population's responses serve as input to a reciprocal inhibition WTA type network.

Our tWTA mechanism is based on reciprocal inhibition between the two populations, Figure 1B. Denoting by r_i the mean firing rate of neurons in the ith population (i = 1, 2), the neuronal dynamics in our model obey

$${\dot{r}}_1=-r_1+g(I_1-J{r}_2)$$ (1)

$${\dot{r}}_2=-r_2+g(I_2-J{r}_1)$$ (2)

where J is the strength of the reciprocal inhibition, and g(x) is typically modeled by a sigmoidal function. Here, for simplicity of the analysis, we used a threshold-linear function, g(x) = x for x > 0 and g(x) = 0 otherwise.We now consider the dynamics in the case where the stimulus is the preferred stimulus of population 1, consequently population 1 receives the “faster” input. During the absolute delay period (i.e., from stimulus onset at time t = 0 to time T, see Figure 1A) the downstream populations have yet to respond to the stimulus, I₁ = I₂ = 0. In this case, the system is governed by a single stable fixed point at the origin, r₁ = r₂ = 0, to which the system converges exponentially; Figure 2A. For the duration of the stimulus selective response, t ∈ [T, T + τ] the single fixed point changes to (1, 0) in which population 1 “outrivals” population 2, in this case; Figure 2B. This fixed point will serve as an attractor and the system will be drawn toward it. At longer times, t > (T + τ), both populations receive the same level of input from the upstream population. If the reciprocal inhibition is weak, J < 1, no population can outrival the other and the system will converge to a symmetric fixed point in which r₁ = r₂. On the other hand, for sufficiently strong inhibition, J > 1, the system has two stable fixed points in which one population outrivals the other, Figure 2C. Due to symmetry, the basin of attraction of the two fixed points is separated by the identity line, r₁ = r₂.

Figure 2.

The phase plane of the WTA rate model during different stages of the dynamics. The arrows show the vector $\left(\begin{array}{c}{\dot{r}}_1\\ {\dot{r}}_2\end{array}\right)$ in the plane of [r₁, r₂]. (A) Before the input populations begin to respond to the stimulus, t < T, the system is dominated by a single fixed point at the origin. (B) During the stimulus selective response period of the upstream input, T < t < T + τ, the single fixed point shifts to a state in which population 1 outrivals population 2 (in the case of input in the preferred stimulus of population 1, as in the example in Figure 1). (C) After the stimulus selective response, T + τ < t, both inputs to the competing populations are the same. The system has two stable fixed points corresponding to the states in which one population outrivals the other and fully suppresses it. The identity line (solid black) is the borderline between the basins of attraction of the two fixed points. The pink lines show the borderlines between the regions of the phase plane in which the initial conditions will result in a correct decision (below the line) and an incorrect decision (above) for different values of τ = 0.2, 0.4, 0.6, and 0.8; that is, the initial conditions that lead to r₂(T + τ) = r₁(T + τ).

In the absence of noise, the initial conditions are expected to be at the origin. Consequently, any amount of stimulus selective delay, τ > 0, will tip the system into the correct basin of attraction. Thus, this trivial architecture produces infinite sensitivity to response latency, in the sense that it achieves correct discrimination for every positive τ.

This hypersensitivity results from bi-stable dynamics where the initial conditions are exactly on the borderline between the basins of attraction of both fixed points. However, neural activity is inherently stochastic and it is unreasonable to assume the WTA type competition will start from such fine-tuned initial conditions. To study the robustness of this readout mechanism we added noise to the initial conditions and investigated the dependence of the tWTA accuracy on the noise level. Thus, instead of starting the WTA competition exactly at the origin, r₁ = r₂ = 0, we now assume that prior to the stimulus selective response period the activity of both populations fluctuates such that the WTA dynamics starts from a “cloud” of distribution close to the origin. Specifically, for the sake of analytical simplicity, we assume the initial conditions r₁(T) and r₂(T) to be independent and identically distributed exponential random variables with a standard deviation σ, at the onset of the tWTA competition at the beginning of the stimulus selective response, time t = T. Assuming further that the noise level is small relative to1/J, one obtains that for t + T ∈ [T, T + τ]

$$\begin{array}{l}{r}_1(T+t)=1-r_1(T)J{\text{e}}^{-t}t+\left(r_1(T)-1\right){\text{e}}^{-t}\\ r_2(T+t)=r_2(T){\text{e}}^{-t}\end{array}$$

where we have assumed without loss of generality that stimulus 1 was presented first. During the stimulus selective response the dynamics attracts the system toward the “correct” fixed point, “outrival 1” in this case. An incorrect decision will occurs when the system is at the basin of attraction of “outrival 2” at the end of the stimulus selective period, namely r₂(T + τ) > r₁(T + τ), which is translated to r₂(T) > ar₁(T) + b, where a = 1/(1 + Jτ), and b = (e^τ − 1)/(1 + Jτ); see pink lines in Figure 2C. Averaging over the distribution of the initial conditions yields the probability of a correct discrimination, Pc, in this approximation

$$\begin{array}{l}{P}_C=1-\frac{{\text{e}}^{-\frac{b}{\sigma }}}{a+1}\end{array}$$ (3)

Figure 3 shows the probability of correct discrimination as a function of τ for different noise levels in the initial conditions, σ, in different colors. The solid lines show the analytical approximation of Equation (3), and the squares depict the numerical estimation of P_C. Note that according to Equations (1) and (2), τ is measured in units of the neuronal rate dynamics, which are typically in the range of about 10 ms. In the absence of noise (blue line), the readout mechanism discriminates correctly, P_C = 1, for any positive τ. As the noise level increases, the probability of a correct discrimination deteriorates for any given τ, and higher values of τ are required to obtain the same level of accuracy. Thus, the neural noise sets the scale of the sensitivity that this readout mechanism can achieve. In other words, for larger noise levels the signal, τ, has to be scaled up to achieve the same level of performance; Figure 3 inset. Note that the accuracy of the tWTA readout has already been addressed in the past, (Shamir, 2009; Zohar et al., 2011; Shriki et al., 2012). Here we focus on the dynamical system that can implement this computation. However, the parameter σ does not reflect noise in the input that limits the information in it, but rather an inherent variability of the readout mechanism itself that limits its sensitivity.

Figure 3.

The probability of correct discrimination is shown as a function of the stimulus selective delay, τ, for different initial conditions of noise levels, σ (different colors). The solid lines depict the analytical approximation of Equation (3). The squares show the numerical estimation obtained by averaging the results of the simulation decision over 10⁴ repetitions. The inset shows the “signal”, τ, required to obtain a specific level of performance, P_C, as a function of noise level, σ. Here we used J = 1.1.

Rate models describe neural activity by a continuous parameter, and as a result can react immediately to changes in their input. Real spiking neurons have an inherent delay before emitting a spike, and even then their response is sparse in time. This makes it crucial to test the generalization of our results beyond the threshold-linear rate model to a model of spiking neurons. For this purpose, we simulated a tWTA readout mechanism based on competition by reciprocal inhibition between two populations of N Hodgkin-Huxley neurons. This was facilitated by the mapping between rate and conductance-based models (Shriki et al., 2003). The dynamic equation for the membrane potential of neuron i in population a ∈ {1, 2} is as follows

$$\begin{array}{l}C\frac{d{V}_{a,i}}{dt}=g_L\left(E_L-V_{a,i}\left(t\right)\right)-I_{a,i}^{active}+I_{a,i}^{ext}+I_{a,i}^{net}\end{array}$$

where g_L and E_L are the leak conductance and reversal potential, C is the membrane capacitance, and $I_{a,i}^{active}$ is the voltage dependent current $I_{a,i}^{active}=I_{Na}+I_K+I_A$. The term $I_{a,i}^{net}$ denotes the reciprocal inhibition from the competing population. Unless otherwise stated, the parameters of the model follow (Shriki et al., 2003); see Methods. The term $I_{a,i}^{ext}$ denotes the input to cell i in population a ∈ {1, 2} from the input layer that is latency-tuned to the stimulus. In our simulations, $I_{a,i}^{ext}$ is the input current from the upstream population to neuron i in population a, and is modeled by $I_{a,i}^{ext}(t)=r\Theta \left(t-\left[T+{\tau }_a+{\delta }_{a,i}\right]\right)+{\xi }_{a,i}(t)$, where r is the strength of the response of the upstream population to the stimulus, ξ_{a, i}(t) is a stimulus-independent white noise of the upstream population, T and τ_a are the absolute and stimulus selective delays, respectively, and δ_{a, i} represents the trial-to-trial variability in the delay of the input to the neuron. We modeled {δ_{a, i}} by i.i.d. Gaussian random variables with zero mean and a standard deviation Δ. The term I^net denotes currents resulting from the lateral connection. Specifically, in this case I^net is the reciprocal inhibition, and its strength is governed by the synaptic strength g_s (see Methods for more details).

As we are interested in a WTA-type competitive readout mechanism, the reciprocal inhibition must be sufficiently strong to enable a bi-stable regime in which one population can outrival the other and fully suppress its activity. For this reason, we need to set the total maximum conductance of population internetwork (inhibitory) connection to a level that causes the network to be in a winner-take-all state (just over 0.7μF/cm²).

Figure 4 shows the result of a competition between the two populations of the readout mechanism during a single trial with: T = 20ms, τ₁ = 0ms, τ₂ = 1ms (T + τ shown by the dashed lines), and Δ = 5ms. The blue plus signs (“+”) denote the specific onset time of the input to each cell within this trial, T + τ_a + δ_{a, i}. The neurons in each group are arranged in the Figure according the latency of their input (early to late from bottom to top). Note that the mean latency difference between the inputs to the two populations was τ ≡ τ₂ − τ₁ = 1ms, whereas within each group input latency varied with a much greater standard deviation of 5 ms. Given the considerable variability in the input times to single cells, some neurons in population 2 fired up to three spikes in response to the stimulus. Nevertheless, within several tens of milliseconds the reciprocal inhibition dynamics converged to a state in which population 1 fired with a high rate and the activity of population 2 was fully suppressed (Figure 4). Thus, even though in their steady state both populations received the same input level, this relatively small mean latency difference was sufficient to enable the reciprocal inhibition mechanism to “choose” the correct group in this specific example.

Figure 4.

Typical example of tWTA competition between two populations of 1000 neurons during a single trial. The Figure depicts a “population raster” where each row shows the spike times of a single neuron (black dots). Bottom half shows neurons from population 1, and upper half population 2. The blue “+” signs depict the specific input latency time for each neuron (T + τ_a + δ_{a, i}), and the neurons within each group are arranged according to their input latency. The dashed vertical line shows the mean latency for each group, T + τ_a. Here we used T = 20ms, τ₁ = 0ms, τ₂ = 1ms,Δ = 5ms, and $g_L=0.71{\text{mS}/\text{cm}}^2$.

To evaluate the extent to which the result in Figure 4 is typical, we estimated numerically the probability of correct discrimination. To do so, we first needed to define an objective criterion for a “correct” response by the system. In our simulations we chose the winner in the WTA competition to be the population that fired more spikes during the time interval of 50ms ≤ t ≤ 100ms. Other choices yielded qualitatively similar results.

Figure 5A shows the probability of the Hodgkin-Huxley type model to discriminate correctly between the two alternatives as function of the mean input latency difference between the two populations, τ ≡ τ₂ − τ₁, for different levels of within-group onset variation Δ (shown by the different colors). The solid lines are the analytical approximation of Equation (3) with J = 2 and fitted values of σ (note that time in Equation (3) is measured in units of τ_m = g_L/C). As can be seen from the figure, even small mean latency differences of less than τ≈ 1 ms can be detected with a high reliability by this simple mechanism. Note, that it is expected that in the limit of large input population sizes the results of the rate model will hold, and in particular Equation (3). Figure 5B shows the probability of the spiking neurons model to discriminate correctly between the two alternatives as function of the mean input latency difference, τ ≡ τ₂ − τ₁, for different sizes of input populations (shown by different colors). As can be seen from the figure, even though discrimination accuracy deteriorates with the decrease in population size, this simple readout mechanism is still able to extract information from the neuronal response latencies. Furthermore, fitting the analytical approximation of Equation (3) (solid lines), we find that the parameter that reflects the noise in the input population in the rate model, σ, decays to zero as one over the square root of the input population size, Figure 5C. Hence, one may attribute the deterioration in discrimination accuracy to a decrease in signal-to-noise ratio in the responses of the input populations for smaller population sizes.

Figure 5.

The probability of correct response of the tWTA is plotted as a function of the stimulus selective delay,τ, (A) for different levels of noise in the input latency, Δ(different colors, Δ is measured in ms, input population size of N = 50 was used) (B) for different sizes of input populations (different colors, Δ = 1 ms was used). The open markers show the numerical estimation of the accuracy of the conductance-based tWTA competition between two populations of N neurons, and the error bars show the SEM. The accuracy was estimated by averaging over 1000 simulated trials. The tWTA decision was the preferred stimulus of the population that fired the most spikes during the time period of 50–100 ms following stimulus onset at t = 0. The solid lines show the analytical approximation of Equation (3) using J = 2 and fitting σ, with time rescaled by the membrane time constant τ_m = g_L/C. (C) The scaling of the internal noise with the size of the input population. The (inverse of the square of the) fitted parameter σ is shown as a function of the input population size for different values of Δ (open symbols in different colors, Δ is measured in ms). The solid lines show linear approximations, for comparison. In the simulations we used T = 20ms, τ₁ = 0ms, τ₂ = τ, and $g_L=0.71{\text{mS}/\text{cm}}^2$.

Discussion

Previous studies of the WTA algorithm have focused on the conventional rate-WTA decision mechanism. However, a number of studies reported that the WTA is also sensitive to the temporal structure of its inputs (Coultrip et al., 1992; Lee et al., 1999; Jin and Seung, 2002; Sheliga et al., 2006; Standage et al., 2005; Kurt et al., 2008). Here we focused on the ability of a dynamical system to implement a specific latency-based decision mechanism; namely, the tWTA readout. We found that the simple reciprocal inhibition architecture of the WTA mechanism is sufficient to implement the tWTA. To implement the tWTA we assumed that the system was in the “strong inhibition” regime (J > 1), such that it was bi-stable when the inputs to the two competing population were similar. This condition was needed, as we assumed that the input strength to both populations was similar and that response latency alone depended on the stimulus. However, in many cases neurons tend to respond with a higher firing rate to the same stimuli to which they respond with a shorter latency (Zohar et al., 2011; Shriki et al., 2012). In these cases the assumption of “strong inhibition” can be relaxed.

The robustness of the reciprocal inhibition mechanism for latency coding was tested against noise in the initial conditions. Additional parameters that govern the tWTA dynamics may also vary. In particular, we assumed that the reciprocal inhibition between the two populations is identical. This assumption contributed to the symmetry that is underlying the infinite sensitivity of the tWTA mechanism in the absence of noise. How will asymmetry in the reciprocal inhibition affect the tWTA? Assume, for example that the effective inhibition from population 2 to 1, J_1←2, is stronger than from population 1 to 2, J_2←1. We find that as long the system is in the strong inhibition regime, i.e., J_2←1, J_1←2 > 1, then for sufficiently large τ the system will converge to the correct fixed point (in the absence of noise). However, for any positive difference, (J_2←1 − J_1←2) > 0, there exists a critical value τ_c, such that for any τ < τ_c (here positive τ denotes input to population 1 preceding input to population 2) population 2 will win the tWTA competition. The critical value τ_c decays to zero as the system approaches symmetry, (J_2←1 − J_1←2) → 0. Consequently, an additional “learning mechanism” that can fine-tune the reciprocal inhibition is required if high sensitivity for very short latencies is necessary. Note that the parameter J reflects the effective inhibition strength that is a product of the strength of a single synapse by the number of synapses.

It has been argued that latency code readouts require an additional stimulus onset detector which serves as a reference from which latency can be measured. A number of studies have reported that onset can be detected by pooling information from neurons that show poor latency tuning (Zohar et al., 2011; Brasselet et al., 2012; Shriki et al., 2012). Here we suggest that such an additional mechanism is not essential. Figure 6A shows the spiking response of the tWTA network to a series of stimuli. In the first, third and fifth presentations the mean latency of the inputs to population 1 was shorter, whereas in the second and fourth presentations population 2 received the fastest input (on average). By examining the population firing rate in Figure 6B, stimulus1, stimulus 2, and no stimulus can easily be discriminated, by adding a decision threshold (black horizontal line in Figure 6B). Setting a reasonable value for the decision threshold results in correct detection and discrimination for all 5 presentations without the use of an additional neural population or detectors; Figure 6C. Figure 7 quantifies the tradeoff between generating false alarms at a high rate by setting the decision threshold too low, and failing to detect the stimulus by setting the threshold too high. The considerable difference between the spontaneous firing rate and the firing rate in response to the stimulus allows for a high detection probability and a negligible false alarm rate, as shown by the sharp ROC curve. However, decision threshold also affects the speed of the response and the probability of false alarm during spontaneous activity; Figure 7 inset.

Figure 6.

Typical example of tWTA competition between two populations of 50 neurons during a single trial in which a series of five alternating stimuli were presented every 100 ms. Thus, for the first presentation T = 20 ms, for the second T = 120 ms, and so on. Each stimulus lasted ΔT = 50ms. (A) Population raster plot: each row shows the spike times of a single neuron (black dots). Bottom half shows neurons from population 1, and upper half population 2. The open circles depict the specific input latency times for each neuron (T + τ_a + δ_{a, i}) in different colors for the different presentations. The thick horizontal lines show the mean times in which each population received input; i.e., from time T + τ_a to time T + ΔT + τ_a. (B) The population firing rate for population 1 (green) and population 2 (blue). The rate was estimated by the total spike count in a sliding window of a 10 ms time bin. Horizontal black line shows the decision threshold of 3.5 spikes/ms used to generate C. (C) The decision of the tWTA network. In the simulations we used τ = 0ms for the preferred stimulus, τ = 5ms to the un-preferred, and $g_L=1.1{\text{mS}/\text{cm}}^2$.

Figure 7.

The tWTA onset detection accuracy. The stimulus detection ROC is presented, depicting hit probability (correct detection) as function of false alarm. Here, onset detection corresponds to a “detection” of a stimulus during the absolute delay period, 0 < t < T. Consequently, false alarm may scale with the absolute delay period, T. Moreover, a false alarm may be flagged in the absence of stimuli due to spontaneous firing of the neurons. Thus, a more relevant characterization of false alarm for stimulus onset detection task would be the false alarm rate during spontaneous activity. The inset shows average decision time and STD for different criterions (dots and bars, respectively) and the probability of having at least one false alarm during 1 s of spontaneous activity as a function of the detection threshold (asterisk). The parameters used for the simulation are the same as the parameters in Figure 6.

The tWTA mechanism does not operate like a working memory but rather more like a sensory system. Thus, in the example in Figure 6 the neural activity decays to a spontaneous firing rate after each stimulus offset. However, this decay is not instantaneous. Therefore, if the inter-stimulus-interval is very brief, the tWTA decision may be affected by its past. This scenario is illustrated in Figure 8, in which due to the short inter-stimulus-interval the tWTA decision is dictated by the preceding decision rather than by the stimulus itself. This is a hallmark of tWTA competition which may serve as an empirical prediction both on the psychophysical as well as the neural level. In this respect the well-studied psycho-acoustical phenomenon termed the precedence effect (Litovsky et al., 1999) may have some bearing. In the precedence effect, when estimating sound source location, the perception of the delayed stimulus is suppressed. This is believed to assist in overcoming the corrupting effects of echoes in the computation of the sound source location.

Figure 8.

Typical example of tWTA competition between two populations of 50 neurons during a single trial in which a series of five alternating stimuli were presented every 60 ms. Thus, for the first presentation T = 20 ms, for the second T = 80 ms, and so on. Each stimulus lasted ΔT = 50ms. During the odd presentations (first, third, fifth…) population 1 received the faster input whereas in the even presentations population 2. (A) Population raster plot. Each row shows the spike times of a single neuron (black dots). Bottom half shows neurons from population 1, and upper half population 2. The open circles depict the specific input latency times for each neuron (T + τ_a + δ_{a, i}) in different colors for the different presentations. The thick horizontal lines show the mean times in which each population received input; i.e., from time T + τ_a to time T + ΔT + τ_a. (B) The population firing rate for population 1 (green) and population 2 (blue). The rate was estimated by the total spike count in a sliding window of a 10 ms time bin. Horizontal black line shows the decision threshold of 3.5 spikes/ms used to generate C. (C) The decision of the tWTA network. In this simulations we used τ = 0ms for the preferred stimulus, τ = 5ms to the non-preferred stimulus, and $g_L=1.1{\text{mS}/\text{cm}}^2$.

Conventional rate-WTA mechanisms have been suggested to play an important role in various computations in the brain and in numerous systems (Lee et al., 1999; Standage et al., 2005; Sheliga et al., 2006, 2007; Kurt et al., 2008). Their implementation only requires a very rudimentary reciprocal inhibition design. Nevertheless, even this simple architecture yields a mechanism that is intrinsically sensitive to the response latency of its inputs. Consequently, by construction, almost every conventional WTA mechanism is highly sensitive to latency cues as well. This highlights the possible role of latency as a source of information in the central nervous system.

Author contributions

OZ, MS: Methodological, analysis, and interpretation of results.

Funding

This work was supported in part by Israel Science Foundation ISF grants No 722/10 and 300/16, and by the Helmsley Charitable Trust through the Agricultural, Biological and Cognitive Robotics Initiative of Ben-Gurion University of the Negev.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.