Dynamic afferent synapses to decision-making networks improve performance in tasks requiring stimulus associations and discriminations

Abstract

Animals must often make opposing responses to similar complex stimuli. Multiple sensory inputs from such stimuli combine to produce stimulus-specific patterns of neural activity. It is the differences between these activity patterns, even when small, that provide the basis for any differences in behavioral response. In the present study, we investigate three tasks with differing degrees of overlap in the inputs, each with just two response possibilities. We simulate behavioral output via winner-takes-all activity in one of two pools of neurons forming a biologically based decision-making layer. The decision-making layer receives inputs either in a direct stimulus-dependent manner or via an intervening recurrent network of neurons that form the associative layer, whose activity helps distinguish the stimuli of each task. We show that synaptic facilitation of synapses to the decision-making layer improves performance in these tasks, robustly increasing accuracy and speed of responses across multiple configurations of network inputs. Conversely, we find that synaptic depression worsens performance. In a linearly nonseparable task with exclusive-or logic, the benefit of synaptic facilitation lies in its superlinear transmission: effective synaptic strength increases with presynaptic firing rate, which enhances the already present superlinearity of presynaptic firing rate as a function of stimulus-dependent input. In linearly separable single-stimulus discrimination tasks, we find that facilitating synapses are always beneficial because synaptic facilitation always enhances any differences between inputs. Thus we predict that for optimal decision-making accuracy and speed, synapses from sensory or associative areas to decision-making or premotor areas should be facilitating.

a fundamental role of an animal's brain is to allow the animal to adapt to an ever-changing environment such that the animal's response, or behavior, leads to most reward or least harm given a particular confluence of environmental stimuli. Subtle differences in stimuli can require opposite behaviors. For example, those characteristics differentiating a friend from a foe or a stronger opponent from a weaker one could appear minor amidst the general overlap of features pertaining to a fellow member of the same species. However, the animal must learn to discriminate differences in such salient features to produce opposite responses, even if much of the sensory information (and neural activity within the brain) is similar. Thus how appropriate responses, or decisions that lead to those responses, are learned and produced from patterns of overlapping and similar neural activity is a matter of intense investigation, both experimentally (Curtis and Lee 2010; Glimcher and Rustichini 2004; Gold and Shadlen 2007, Harris et al. 2008; Histed et al. 2009; Jay 2003; Kennerley and Walton 2011; Takeda et al. 2005) and theoretically (Eckhoff et al. 2008; Feng et al. 2009; Rigotti et al. 2010a, 2010b; Sakai et al. 2006; Sakai and Miyashita 1991; Shen et al. 2008; Soltani and Wang 2008, 2010; Wang 2008).

In our studies, we focus on the inputs to a binary winner-takes-all (WTA) decision-making circuit (Wang 2002). The WTA network serves as a model for perceptual decision-making and the animal's behavioral choice in a two-alternative forced-choice task (Wang 2002; Wong and Wang 2006). Response of the WTA network depends on bias in the input to its two decision-making pools. In our model, input synapses to the decision-making neurons are modified by a dopamine (DA)-modulated Hebbian reward-based plasticity rule (Reynolds and Wickens 2002; Soltani and Wang 2006) that, in principle, can lead the decision-making pools to produce responses that maximize reward, given sufficient information about the stimuli within the neural activity of its input cells. For the networks of spiking neurons in our study, the cells providing input to the decision-making circuit reside in a randomly recurrently connected associative layer. Within such a layer, stimulus-selective cells can arise, even when the inputs are broadly overlapping (Bourjaily and Miller 2011a; Rigotti et al. 2010b). A general linear readout, as used in support vector machines, could produce appropriate responses to information within the high-dimensional space of associative layer neural firing rates via a trained or fitted set of synaptic weights (Burges 1998). However, the requirement that such weights are produced by biologically realistic learning rules generally constrains the readout process such that information can be contained within neurons in the associative layer, but firing of those neurons cannot produce a reliable response. In this report, we suggest that the use of dynamic synapses between the associative layer and decision-making layer can enhance the readout process beyond that achievable by a general linear classifier.

Multiple decision-making studies have focused on the formation and function of the internal dynamics of the decision-making networks (Albantakis and Deco 2011; Cisek et al. 2009; Daw et al. 2006; Lee and Seo 2007; Rolls et al. 2010; Theodoni et al. 2011; Wang 2002; Wong and Wang 2006; Xie et al. 2002). In computational studies of decision-making, the inputs have been primarily via static synapses (i.e., synapses without short-term synaptic plasticity), whose postsynaptic output increases approximately linearly with presynaptic firing rate (until saturation at a rate proportional to the inverse of the postsynaptic receptor's time constant) (Wang 2002; Wong and Wang 2006). However, numerous experimental studies across different cortical regions (Abbott et al. 1997; Chance et al. 1998; Markram et al. 1998b; Nelson and Turrigiano 1998; Wang et al. 2006) have shown that synaptic strength is dynamic on the timescale of interspike intervals and that synaptic transmission increases nonlinearly with presynaptic firing rate (Chapman et al. 2000; Grillner et al. 2005; Tsodyks et al. 1998). Computational studies have shown a number of functions for both facilitating and depressing synapses, such as persistent neural activity (Barak and Tsodyks 2007; Conboy et al. 2010; Markram 1997; Martinez-Garcia et al. 2011), filtering neural signals (Farajidavar et al. 2008; Lindner et al. 2009), and gain control (Abbott et al. 1997), among others.

Some of the most difficult behavioral tasks are linearly nonseparable (Grand and Honey 2008; Harris 2006; Harris et al. 2008; Melchers et al. 2008; Sanderson et al. 2006); that is, the choice of response cannot be determined by an appropriately weighted combination of individual stimuli (Minsky and Papert 1987). Behavioral tasks that are linearly nonseparable include those that employ exclusive-or (XOR) logic, such as biconditional discrimination (Melchers et al. 2008; Sanderson et al. 2006) and negative patterning (Harris et al. 2008), inference tasks such as associative transitive inference (Bunsey and Eichenbaum 1993), certain paired association tasks (Takeda et al. 2005), and transverse patterning (Sanderson et al. 2006). Although these tasks vary in design and sensory modality, they all require animals to produce stimulus pair-selective responses to solve the task. In biconditional discrimination, as in other linearly nonseparable tasks, successful decision-making requires responses selective to stimulus pairs (e.g., A + B vs. C + B vs. C + D) (Bourjaily and Miller 2011a), with different responses to the most similar pairs. Thus, if the animal only learns a preferred response to one member of the pair (e.g., stimulus B from stimulus pair A + B), then if it responds correctly to stimulus pair A + B, it would respond incorrectly to stimulus pair C + B. Our results for the biconditional discrimination task can be applied to the other linearly nonseparable associative learning tasks mentioned above.

In artificial and/or feedforward neural networks with nonlinear behavior, XOR has been solved using methods such as selective suppression of feedback (Hasselmo and Cekic 1996) and associative learning (Alkon et al. 1990) of input correlations in conjunction with stochastic gradient ascent reinforcement learning (Iannella and Back 2001; Park et al. 2000). At the synaptic level, XOR has been solved using stochastic gradient ascent learning, where stochastic synapses and/or irregular spiking provided requisite noise for stochastic gradient ascent learning to a readout cell (Seung 2003; Xie and Seung 2004).

A critical property of dynamic synapses is that they lead to nonlinear synaptic transmission (Dayan and Abbott 2001; Fiete et al. 2004; Markram et al. 1998a, 1998b), which could enhance the circuit's ability to produce solutions to linearly nonseparable tasks. Here, we address how short-term plasticity in afferent synapses to a decision-making network (Wang 2002; Wong and Wang 2006) with a biophysically based reinforcement learning rule (Jay 2003; Reynolds and Wickens 2002; Shen et al. 2008) can affect performance and reaction times in both a linearly nonseparable task and simpler tasks. Although individual synapses are known to have both facilitating and depressing dynamics (Grillner et al. 2005; Maass and Zador 1999; Markram et al. 1998b), to investigate the specific role of each, we look at either purely facilitating or purely depressing synapses (Chapman et al. 2000; Dayan and Abbott 2001; Grillner et al. 2005; Markram et al. 1998a) compared with static synapses in each network and task.

For the overwhelming majority of network types trained on linearly nonseparable tasks and for all tests of linearly separable tasks, we find that facilitation's superlinear synaptic transmission improves performance. Specifically, facilitation improves decision-making accuracy and speed of response, whereas depression's sublinear synaptic transmission worsens both relative to static synapses. The beneficial role of facilitation suggests that synapses from sensory and associative areas of the brain to decision-making regions such as prefrontal cortex and premotor areas should be facilitating.

MATERIALS AND METHODS

Simplified Firing Rate Model

We produced a simplified firing rate model network to demonstrate clearly the interaction between the following three factors: nonlinearity of associative layer responses, dynamics due to short-term plasticity at the synapse, and the details of the reward-based learning rule. The simplified model was written in Matlab (The MathWorks, Natick, MA), and the code is available online (http://people.brandeis.edu/∼pmiller/code.html).

As in the full network (see below), four stimulus pairings were used (A + B, A + D, C + B, or C + D). We assume four types of neural response in the associative layer (AB, AD, CB, or CD), corresponding to groups of cells whose preferred stimulus is one of the four stimulus pairings (see Table 1). In the associative layer, each cell group i is represented by a single firing rate, which evolves during each stimulus pair presentation s according to

$$\tau \frac{\text{d}{r}_i}{\text{d}t}=-r_i+f\left(I_{i,s}^{\text{App}}\right),$$

where τ = 5 ms and the applied current I_i,s^App = 40 for the group's preferred stimulus pair, I_i,s^App = 20 for the opposite (nonoverlapping) stimulus pair, and I_i,s^App = 0 for each of the two intermediate stimulus pairs, as given in Table 1.

Table 1.

Firing rate response of linear neurons to paired inputs

	Stimulus Pair Input, Hz
Cell Group's Preferred Stimulus	A + B	A + D	C + B	C + D
AB	40	20	20	0
AD	20	40	0	20
CB	20	0	40	20
CD	0	20	20	40

The steady-state firing rate, f(I^App), in response to an applied current, I^App, can take on three forms. The response function may be linear, f^lin(I^App) = I^App; sublinear, f^sub(I^App) = I^App/(1 + αI^App); or superlinear, f^supra(I^App) = I^App·(1 + αI^App). We set the degree of nonlinearity as α = 0.1 in the results (presented in Fig. 3).

Fig. 3.

Interaction of firing rate and synaptic nonlinearity in a firing rate model. A and B: fraction of reward accumulated over all trials (blue = 0; green = 0.5; red = 1). C and D: asymmetry in the synaptic strengths from associative layer excitatory cells to decision layer pools (blue = asymmetry of −1, i.e., input to the incongruous pool; green = asymmetry of 0, i.e., equal input to both pools; red = asymmetry of 1, i.e., input to the congruous pool). In A and C, the presynaptic rate dependence of the reinforcement learning rule matches the synaptic input to the decision layer cells. In B and D, the reinforcement learning rule producing the maximum reward for each combination of firing rate and synaptic nonlinearity is used. Stat, static; Dep, depression; Fac, facilitation; Sublin, sublinear; Supralin, supralinear; f-l, firing rate-linearity relationship.

The four associative layer cell groups i provide synaptic input to each of two decision layer cell groups, j = 1, 2, whose firing rates evolve according to

$$\tau \frac{\text{d}{r}_j}{\text{d}t}=-r_j+f^{\text{dec}}\left(I_j^{\text{In}}+I_j^{\text{Rec}}+I_j^{\text{Noise}}\right),$$

where I_j^In is input from the associative layer, I_j^Rec is recurrent self-excitation and recurrent cross-inhibition from within the decision layer, and I_j^Noise is uncorrelated noise input in the decision layer to produce stochastic responses. The total input I_j^In to a decision layer cell group j depends on the synaptic strength, W_ij^AD, which is modified through reward-based plasticity and the rate-dependent synaptic transmission, s_i(t), from presynaptic associative layer cell group i: I_j^In(t) = ∑_i W_ij^ADs_i(t).

For static (i.e., linear) synapses, s_i^stat(t) = r_i(t); for depressing (i.e., sublinear) synapses, s_i^dep(t) = r_i(t)/[1 + αr_i(t)]; and for facilitating (i.e., superlinear) synapses, s_i^fac(t) = r_i(t)⋅[1 + αr_i(t)]. Note that the above formulas for the simplified rate model include only the steady-state change in amplitude wrought by the short-term plasticity, to demonstrate its effect on decision-making; they do not account for the transient dynamics that occur when firing rates change (Abbott et al. 1997). We modeled the complete dynamics in the full spiking-neuron network model [see Supplemental Table S1, where simulation details are presented in a form suggested by Nordlie et al. (2009)]. (Supplemental data for this article is available online at the Journal of Neurophysiology website.)

The synaptic strengths from the four associative layer groups to the two decision layer groups were initially equal, set to W_ij⁽⁰⁾ = 10/〈 s_i(∞)〉, where s_i(∞) is the steady-state synaptic input of cell group i and the average (in angle brackets) was over the four associative layer cell groups. Thus the initial mean input to the decision layer before training was held constant when we varied the nonlinearity in the associative layer or the short-term dynamics of the afferent synapses. The synaptic strengths were updated by ΔW_ij (defined below) on a trial-by-trial basis in a reward-dependent manner to the “winning” decision layer pool with the higher firing rate after a 500-ms stimulus presentation. If the winning decision layer pool was the designated one for obtaining reward (group 1 for stimulus pair A + B or C + D and group 2 for stimulus pair A + D or C + B), then ΔW_ij = ΔW⁺(A_i − 〈A_i〉); otherwise, ΔW_ij = −ΔW⁻(A_i − 〈A_i〉). A_i depended on the firing rate of the associative layer cell and could be linear, A_i = r_i(t^max); sublinear, A_i = r_i(t)/[1 + αr_i(t^max)]; or superlinear, A_i = r_i(t)·[1 + αr_i(t^max)]. Thus we present our results in a 3 × 3 × 3 matrix form, because they depend on the linearity, sublinearity, or superlinearity of neural response, synaptic dynamics, and the presynaptic rate dependence of the reinforcement rule. All other terms in the decision layer were kept constant across these 27 combinations of network.

The recurrent current to the decision layer cell groups (j = 1, 2) is given by I_j^Rec(t) = W^symr_j(t) − W^Xr_2−j(t) with W^sym = 0.8 and W^X = 0.6. The noise current to the decision layer cell groups is given by I_j^Noise(t) = ση(t), where σ = 0.2 and η(t) is a white-noise process with unit variance. The firing rate curve of the decision layer neurons is sigmoidal, f^dec(I) = r^max/{1 + exp [(I^Th − I)/I^σ]} with r^max = 50 Hz, I^Th = 50, and I^σ = 8.

We simulated 10 independent sessions of 1,000 consecutive trials with each linearity-type configuration and calculated the fraction of correct responses across these trials. We also calculated the net asymmetry, S^W, of the final weight matrix produced by the learning rule as a number that can vary from −1 to +1:

$$S^W=\frac{W_{11}^{\text{AD}}+W_{12}^{\text{AD}}+W_{21}^{\text{AD}}+W_{22}^{\text{AD}}+W_{31}^{\text{AD}}+W_{32}^{\text{AD}}+W_{41}^{\text{AD}}+W_{42}^{\text{AD}}}{W_{11}^{\text{AD}}-W_{12}^{\text{AD}}-W_{21}^{\text{AD}}-W_{22}^{\text{AD}}-W_{31}^{\text{AD}}-W_{32}^{\text{AD}}-W_{41}^{\text{AD}}-W_{42}^{\text{AD}}}.$$

A weight asymmetry of S^W = −1 means that associative layer cell groups selectively potentiate to decision layer cell groups that produce the incongruous response, i.e., opposite to the one desired for their preferred cue.

Two-stimulus discrimination task.

To assess the effect of dynamic synapses on the inputs to a decision-making network in a two-stimulus discrimination task, we used the rate model, as described above. In this task, we used two distinct associative layer pools, each connected to a separate decision-making pool. Inputs to the two associative layer pools differed (by 5% of the mean in the results presented) to simulate concurrent visual input of different strengths in opposing directions (Wang 2002), as used in visual motion coherence discrimination tasks (Shadlen and Newsome 1996). To compensate for the change of effective synaptic strength inherently produced by dynamic synapses, we evaluated discrimination accuracy, namely, the probability that the “winning” decision-making pool was the one with greater stipulated input, as a function of baseline synaptic strength of afferent synapses to the decision-making circuit.

Spiking Network Simulations

Task logic and inputs.

In the biconditional discrimination task (Sanderson et al. 2006), four stimulus pairings were activated, one per trial (A + B, C + D, A + D, or C + B), with each stimulus producing inputs as Poisson spike trains. If either both A and B were activated or neither was activated (i.e., C + D), the decision was to release the lever. If either A or B but not both were present (i.e., A + D or C + B), the opposite decision was made, namely, to hold the lever until the end of the trial. In the stimulus-response association task, single stimuli were presented and the decision was to release for inputs A or D and to hold for inputs B or C.

We produced networks with different degrees of input correlations (as shown in Fig. 4A) by altering the number of independent inputs per stimulus as 2, 4, 6, 10, or 20 to excitatory and inhibitory cells in the associative layer (see Fig. 4B). Each input comprised a train of independent Poisson spikes with a mean firing rate defined by r̄ = 480 Hz/(no. of inputs per stimulus). For example, in a network with 20 inputs, any stimulus produced 20 independent Poisson spike trains at 24 Hz, with each input projecting to independent sets of cells within the associative layer, whereas with 2 inputs per stimulus, the 2 trains of 240-Hz Poisson spikes (a firing rate much higher than that produced by an individual cell) could be considered as 20 independent Poisson spike trains of 24 Hz grouped into 2 sets of 10: while the receiving cells between sets were uncorrelated, the receiving cells within such a set of 10 would be identical. Thus, as the number of inputs per stimulus decreased, the correlation in connectivity from afferent cells increased (see Fig. 4A).

Fig. 4.

Inputs and architecture for the full spiking network with decision-making accuracy. In this study, we have examined the robustness of our findings across networks with varying input correlation and input probability, thus creating multiple network conditions. In A, we provide a general description of our inputs and show an example of input configurations from a single stimulus-responsive input population for 2 different input correlations (see materials and methods, Spiking Network Simulations). Each input population is represented by 20 cells, which output independent Poisson spike trains but correlated connections. The total number of independent sets of connections ranges from 2 to 20. Top: with 2 independent inputs per stimulus population and low input connection probability (1/5 is shown), associative layer cells receive initially sparse selective inputs. Bottom: with 10 independent inputs per stimulus population and low input connection probability (1/5 is shown), associative layer cells receive more uniform, less correlated inputs. Poisson input groups randomly project to a sparse-random recurrent network of excitatory (red) and inhibitory (blue) cells. Input projection probability ranges from 1/20 (sparse) to 1/2 (dense), with input connections selected independently between each set of independent input cells per input population. B: the network consists of an input population of Poisson independent inputs per stimulus that project randomly (represented by dashed lines) to a random recurrent network of excitatory (red) and inhibitory (blue) cells called the associative layer. Excitatory-to-excitatory connections (arrows) and inhibitory-to-excitatory connections (colored circles) are probabilistic and plastic. All-to-all inhibitory-to-inhibitory synapses are also present but not plastic. Triplet spike-timing-dependent plasticity (STDP) occurs at excitatory-to-excitatory and input-to-excitatory synapses, whereas long-term potentiation of inhibition (LTPi) occurs at inhibitory-to-excitatory synapses. Homeostasis by multiplicative synaptic scaling is present on all plastic synapses within the associative layer. Inhibition is feedforward only (i.e., the network does not include recurrent excitatory-to-inhibitory synapses) (Bourjaily and Miller 2011a). C: excitatory cells from the associative layer project all-to-all with dynamic synapses (red arrowheads), initially with equal synaptic strength to excitatory cells in both the hold and release pools of the decision-making network, which also receive an urgency-gating signal (Cisek et al. 2009), modeled as a linear ramping conductance to ensure decisions are made each trial. The decision-making network consists of 2 excitatory pools with strong recurrent connections, which compete via cross-inhibition. Strong self-recurrent excitation ensures bistability for each pool, while the cross-inhibition generates winner-take-all (WTA) dynamics such that only 1 population can be active following the stimulus, resulting in 1 decision (Wang 2002). Whether the motor output (based on the decision of hold vs. release) is correct or incorrect for the corresponding cue (e.g., A + B), according to the rules of the behavioral tasks, determines the presence or absence of dopamine (DA) at the input synapses (dashed lines, double arrowhead). D–G: biconditional discrimination decision-making network accuracy depends on afferent short-term plasticity. Each matrix contains the results for 25 networks trained with LTPi and triplet STDP, with 5 levels of input correlation (x-axis) and 5 levels of sparseness (y-axis) in 1 of 3 conditions. The color bar in D represents stimulus pair selectivity at the population level with a range of 0–2. For decision-making networks in E–G, the color bar represents percent correct with a range of 50–100% correct. D: trained network stimulus pair selectivity. E: depressing synapses produce 2 reliable networks via sublinear synaptic transmission. F: static synapses generate 6 reliable networks due to associative learning generating selectivity within the associative layer (Bourjaily and Miller 2011a). G: networks with facilitating synapses generate 12 reliable networks via superlinear synaptic transmission. These results demonstrate the beneficial role of facilitation for reliable decision-making. [Panels A–C reprinted from Bourjaily and Miller (2011b).]

Input sparseness was defined via the probability of any input group projecting to any given cell. As input connection probability increased, sparseness decreased. We used the following five values for input connection probability: 1/2, 1/3, 1/5, 1/10, and 1/20 (1/5 in the default network). Five levels of input sparseness, combined with five different degrees of input correlations, led to 25 variant networks in each regime.

Associative layer connectivity.

Both input and recurrent connectivity in the initial network were sparse and random. We simulated 400 neurons with an excitatory-to-inhibitory ratio of 4:1 (Abeles 1991). Excitatory-to-excitatory connections were sparse and random. Inhibition was feedforward only. Inhibitory-to-inhibitory connections were all-to-all. Finally, inhibitory-to-excitatory synapses connected randomly. Connections to the decision layer were initially all-to-all from excitatory neurons with a uniform strength. We used a stronger initial constant input weight for the sparse networks (1/10, 1/20), because their initial low average activity was too small to drive the decision-making layer otherwise (Bourjaily and Miller 2011a).

Decision layer connectivity.

The decision-making network, based on Wang (2002), comprised two 200-cell excitatory and 50-cell inhibitory pools. Connections within each pool were all-to-all. Strong recurrent excitation within each pool and cross-inhibition from each inhibitory pool to the opposing excitatory pool together generated WTA activity so that only one pool was stably active, modeling the decision. The decision-making network received an “urgency-gating” input, which ramped up linearly from cue onset to offset, ensuring that a decision was made each trial (Cisek et al. 2009).

Neuron and synapse properties.

We used leaky integrate-and-fire neurons (LIF) (Tuckwell 1988), defined according to the equations in Supplemental Table S1D, with parameters defined in Supplemental Table S2, D1 and D2. The spiking threshold potential was dynamic, because it depolarized with increased firing and decreased exponentially to the base value as the firing rate decreased. After a spike, there was a dynamic refractory conductance, g_ref(t).

We included voltage noise by a Gaussian distribution of zero mean with unit variance and amplitude σ_V in the associative layer. We also modeled noise as independent excitatory AMPA and inhibitory GABA_A conductance variables, drawn from a uniform distribution [0 σ_S] (see Supplemental Table S2 for parameters).

Synaptic conductances were modeled as instantaneous steps after a spike followed by an exponential decay (described in Supplemental Table S1). Recurrent excitatory currents were modeled by AMPA and voltage-dependent NMDA receptors. Inhibitory currents were modeled by GABA_A receptors. NMDA receptors were also defined by the voltage term (Compte et al. 2000; Jahr and Stevens 1990). Total conductance was the weighted sum of all inputs (see Supplemental Table S1). Initial values of weights depended on synapse type as given in Supplemental Table S2 and were modified according to the plasticity rules described below and in Supplemental Table S1.

Short-term synaptic plasticity.

Short-term facilitation and depression were modeled as instantaneous steps after a presynaptic spike followed by an exponential decay (Dayan and Abbott 2001), described by the equations below, which mimic in vitro measurements (Markram and Tsodyks 1996; Markram et al. 1998a). The total conductance of the presynaptic synapse to the decision-making network was a multiplication of the facilitation or depression variable (defined below) by the normal conductance [e.g., s(t)^AMPAfac = s_AMPA(t) × F(t), where F(t) is the facilitation variable].

Dynamics of the facilitation or depression variables are given in Supplemental Table S1. The facilitation variable represents empirically the gradual decay of calcium concentration and calcium-dependent receptor activation that follows calcium influx at the axon terminal following each action potential. The depression variable represents empirically the fraction of vesicles available for release, which decreases following each presynaptic spike but recovers to its base level with a time constant dependent on vesicle recycling.

Long-term plasticity.

For all plastic connections, changes in synaptic strength per trial were limited to a maximum of 50% per trial to constrain the model to biologically observed change from in vitro and slice procedures (Bi and Poo 1998; Maffei et al. 2006), while across all trials, synaptic strength was hard bounded between 0 and 20× WEE (associative excitatory-to-excitatory base synaptic weight), 20× WEI (associative inhibitory-to-excitatory base synaptic weight), and 10× Input synE (excitatory input synaptic weight), the initial mean synaptic weights given in Supplemental Table S2.

We modeled long-term potentiation of inhibition (LTPi) from recent experimental work by Maffei et al. (2006) and as in our prior article (Bourjaily and Miller 2011a). LTPi occurred when an inhibitory cell spiked and its postsynaptic excitatory cell was depolarized (explicitly modeled with a voltage threshold) but silent, as described in Supplemental Table S1. If the excitatory cell was co-active (i.e., spiking), synapse strength underwent no change. We referred to this as a “veto” effect in our model of LTPi. Any excitatory spike within a uniform temporal window about the inhibitory spike, the LTPi window, resulted in a veto. For each inhibitory spike (“non-vetoed”), the synapse was potentiated. LTPi has been reported experimentally as a mechanism for increasing (but not decreasing) the strength of inhibitory synapses in cortex (Maffei et al. 2006). We used homeostasis by multiplicative postsynaptic scaling (Turrigiano et al. 1998) at the inhibitory-to-excitatory synapses to compensate for LTPi's inability to depress synapses. Hard upper bounds on inhibitory synaptic strength limited further potentiation of the inhibitory synapses of the most inhibited cells.

Triplet spike-timing-dependent plasticity (STDP) was modeled according to the rule published by Pfister and Gerstner (2006) using the parameters cited from the full model “all-to-all” cortical parameter sets (see Supplemental Table S2). The weight changes are detailed in Supplemental Table S1.

Synaptic stability was maintained by multiplicative postsynaptic scaling (Turrigiano et al. 1998) that was updated each trial. The change in synaptic strength was proportional to the difference between the mean rate, r̄, and a goal rate, r_goal, with a rate constant, ε_H, shared by all synapses. The goal rates, r_gE and r_gIE, were heterogeneous about their means with an added 5-Hz random spread from a uniform distribution.

DA-modulated Hebbian reward rule and reaction times.

Reward plasticity updated the associative layer excitatory to decision layer excitatory synapses according to Supplemental Table S1. If the decision matched the instructed cue, then reward was delivered (e.g., for stimulus pair A + B, a “release” response resulted in reward = 1, producing LTP when pre- and postsynaptic cells were active, whereas a “hold” response to A + B resulted in reward = 0, and the sign of the change to the synapse was negative, i.e., LTD between coactive cells). We used this rule because it has been suggested by the DA dependence of correlational synaptic plasticity in vitro (Jay 2003; Reynolds and Wickens 2002; Shen et al. 2008) and has been applied successfully by others in computational studies (Soltani and Wang 2006, 2010) for biologically based reinforcement learning. If the associative layer contains sufficient numbers of stimulus pair-selective cells, then this rule will maximize reward and generate selective potentiation from excitatory associative layer to excitatory decision layer cells necessary for reliable decision-making.

Decisions and reaction times were calculated as follows. The reaction time for a decision was made when the mean activity of one excitatory pool (e.g., release) in a 20-ms time bin was at least 20 Hz greater than that of the opposite excitatory pool (e.g., hold). We averaged reaction times across the final 80 trials and then across the 4 random instantiations for each of the 25 networks. For a reward of 1, we required the “correct” excitatory pool to maintain at least 20 Hz greater activity than that of the opposite pool in the final time bin of the stimulus presentation. Repeating these calculations with alternate bin sizes of 10, 50, and 100 ms did not produce significant differences in reaction times.

Single-Stimulus and Stimulus Pair Selectivity Metric

Single-stimulus selectivity defined each neuron's selectivity for one stimulus (e.g., A) over the other three stimuli (e.g., B, C, and D). Likewise, stimulus pair selectivity defined each neuron's selectivity for one stimulus pair (e.g., A + B) over the other three stimulus pairs (e.g., A + D, C + B, and C + D). We defined selectivity, S_i, of each excitatory neuron i as its maximum firing rate minus its mean response across all four stimuli, normalized by the mean response: S_i = (r_i^max − 〈r_i〉)/〈r_i〉, where r_i^max was its maximal response and 〈r_i〉 was its mean response.

Task Performance Metrics, Statistics, and Curve Fitting

We addressed how short-term plasticity affected a network's accuracy at biconditional discrimination, a nonlinear XOR operation. Linear operations can produce up to 75% correct responses; thus we designated networks with response accuracy in the range of 76–84% as borderline networks, whereas those with >85% correct were designated as reliable. Final decision-making accuracy (% correct) was calculated from the mean of the final 80 trials.

We computed statistical significance between networks with different synaptic dynamics using a two-sample t-test. Specifically, we compared 4 instantiations of each of the 25 network input configurations. We also combined the network input configurations and instantiations to strengthen statistical power (4 instantiations, 25 network input configurations, 100 networks total) before statistical comparison by paired t-test for an overall global change in accuracy.

To assess whether cells preferentially strengthened connections to the congruous decision-making pool, for each cell we calculated a response bias as the mean firing rate to stimuli requiring a response of “release” minus that to stimuli requiring a response of “hold” (i.e., r_release − r_hold) and a connection bias as the mean synaptic weight to cells in the release pool minus that to cells in the hold pool (i.e., W_release − W_hold).

Numerical Procedures

Simulations were run for at least 400 trials using the Euler-Maruyama method of numerical integration with a time step dt = 0.02 ms. To ensure network stability, we ran key networks up to 2,000 trials with sustained results. For all configurations, we repeated with four random instantiations of initial network structure, cell/synapse heterogeneity, and noise. Robustness was further ensured by simulating key networks using 10 distinct random instantiations. Simulations were written in C++ and run on Intel Xeon machines. Matlab r2011a was used for rate models, as well as all data analysis and visualization.

RESULTS

Biconditional Discrimination: Firing Rate Model

Necessity for nonlinearity of inputs to the decision-making network.

In a task such as biconditional discrimination, one expects to find in a successful final network that those associative layer cells most selective to a particular stimulus pair connect preferentially to congruous decision-making neurons, namely, those whose activity generates the correct response to that stimulus pair (Fig. 1B). That is, associative layer neurons most responsive to stimulus pairs A + B or C + D should connect to the release pool of neurons, whereas those associative layer neurons most responsive to stimulus pairs A + D or C + B should connect more strongly to decision layer neurons in the hold pool. However, even with this appropriate connectivity, the output from these cells to the decision layer neurons must be a nonlinear function of the associative layer inputs for the network to produce a correct response in a biconditional discrimination task.

Fig. 1.

Synaptic nonlinearity can combine with asymmetric connectivity to produce above-chance performance in biconditional discrimination. Simplified networks are shown with linear input-output associative layer neurons. A: the initial pretrained linear network has all possible connections, so excitatory associative layer cells do not preferentially project to either decision-making pool. B: after training with synaptic plasticity, linear excitatory associative layer neurons project selectively to either the release or hold decision-making pool. For the example A + B trial, each input (A, B) fires at 20 Hz, generating a 40-Hz AB cell that projects to release but also is matched by the combined output of 2 cells, AD and CB, that each fire at 20 Hz and project to hold. Thus the total synaptic transmission from information-containing cells in the associative layer is equal to the 2 decision-making pools, leading to chance performance. C: synaptic facilitation produces superlinear synaptic transmission, which generates nonlinear input to the 2 decision-making pools, leading to the ability to form reliable decisions. (Arrow thickness represents strength of synaptic transmission.) D: synaptic depression with the same connectivity as static or facilitating synapses causes the opposite pool to win, because the congruous pool is driven by associative layer neurons firing at a higher rate but with lower synaptic transmission, which is sublinear as a function of firing rate or depressing synapses. E: however, with synaptic depression, many neurons can potentiate to the incongruous decision-making pool (e.g., A + B → hold). Given sufficient numbers of such oppositely connected neurons from the associative layer to the decision-making circuit, networks with depressing synapses can drive reliable decisions, because lower synaptic transmission from the cells with the highest firing rates to the incongruous pool allows the congruous pool to win and generate selective potentiation over trials. DA, dopamine; WTA, winner take all.

If the associative layer neurons operate in a linear regime, the response of a cell to its preferred stimulus pair (e.g., its firing rate r_AB to stimulus pair A + B, see Table 1) is equal to the sum of two responses to nonpreferred stimulus pairs (e.g., its firing rate r_AD to A + D plus firing rate r_CB to C + B, see Table 1) (Fig. 1B). The corollary is that on a trial, for example, A + B, AB-selective cells that drive activity in the release pool of the decision-making layer are opposed and matched by equal drive from CB-selective and AD-selective cells, each firing at one-half the rate, promoting activity in the hold pool (Fig. 2A).

Fig. 2.

Impact of nonlinear synaptic transmission as a function of presynaptic activity in a biconditional discrimination task. A: synaptic transmission rates as a function of mean firing rates are shown as solid curves. Static synapses produce linear synaptic transmission (blue line). This is problematic for biconditional discrimination, because on an A + B trial the activity of the most responsive cell (e.g., AB; black marker on blue line) is always equal to the combined activity from 2 cells receiving input from either A or B, but not both (e.g., AD and CB; blue circle at double the value of the blue cross). Thus these synapses produce equal input to the 2 decision-making pools, as shown in B. Facilitating synapses produce superlinear synaptic transmission: r〈P_rel^F〉 = r(P₀^F + f_Frτ_F)/(1 + f_Frτ_F) (Dayan and Abbott 2001), which results in the synaptic transmission rates produced by the maximal AB cell (black marker on green line) being greater than the sum of the synaptic transmission rates from the AD and CB cells (green circle at double the value of the green cross). Depressing synapses produce sublinear synaptic transmission rates: r〈P_rel^D〉 = rP₀^D/[1 + (1 − D_frac)rτ_D], which results in the synaptic transmission rates produced by the maximal AB cell (black marker on red line) being less than the sum of the synaptic transmission rates from the AD and CB cells (red circle at double the value of the red cross). B–D: linearly nonseparable tasks [e.g., exclusive-or (XOR)] such as biconditional discrimination can be aided by the appropriate type of efferent synapse type (blue, static; red, depressing; or green, facilitating) depending on the nonlinearity of the input-output function in the operating regime of neurons in the associative layer. Colored lines represent constant synaptic output as a function of the firing rate of 2 cells, 1 receiving A and B inputs (y-axis) and the other receiving C and D inputs (x-axis). The mean rate of the cells (across all 4 input combinations) is r₀. B: neurons with linear responses and static synapses cannot solve linearly nonseparable tasks, because no straight line can separate AB/CD from AD/CB, but the nonlinear synaptic transmission of facilitation or depression can separate the responses to solve the task. C: if neurons in the associative layer have a superlinear response, as produced by the spiking threshold, then responses to the optimal stimulus pair are more than double the response to a single component of the pair. In this case, static synapses are sufficient, but the superlinear synaptic transmission of facilitation enhances separability. Depression, however, with its sublinear synaptic transmission, compromises the threshold nonlinearity, reducing or eliminating any separability achieved. D: if neurons are in a saturating regime, for example, if firing near their maximal rates, the separability achieved by static synapses with linear output is enhanced by depressing synapses but is compromised by facilitating synapses.

However, nonlinearity in the synaptic output as a function of presynaptic firing rate as produced by dynamic synapses (Fig. 2A) can produce preferential input to one of the decision-making pools and improve performance. Short-term facilitation produces superlinear synaptic transmission rates, whereas depression produces sublinear synaptic transmission rates. Because the facilitation function of the rates is superlinear, the synaptic transmission rates to the decision-making network for AB neurons during trials A + B is more than double the sum of the synaptic transmission rates from AD and CB neurons firing at one-half the rate [e.g., F(r_AB) > F(r_AD) + F(r_CB)] (Fig. 2A). Thus, on each trial in the presence of facilitation, the maximally driven cells will reliably drive the correct decision-making pool (Fig. 1C), producing improved task performance (Fig. 3A).

Short-term depression produces the opposite result of synaptic facilitation such that a maximally driven cell produces less synaptic transmission than the sum of that produced by the two secondarily driven cells, which fire at one-half the rate [e.g., D(r_AB) < D(r_AD) + D(r_CB)] (Fig. 2A). This leads to poor decisions if the associative layer neurons become connected to the decision layer neurons via dopaminergic learning (i.e., reward-based plasticity) as shown in Fig. 1, A and D. Intriguingly, if the dopaminergic learning could lead to the connectivity shown in Fig. 1E, with associative layer cells more strongly connected to the incongruous decision-making pool, then the nonlinearity of synaptic depression would be sufficient to solve the task given linear associative layer responses. Indeed, Fig. 3, A and C (left columns, middle rows) demonstrates such a result in the linear associative layer network.

In Fig. 2, B–D, we motivate an expected interaction between any nonlinearity of the firing rate responses of associative layer neurons with nonlinearity in their synaptic transmission to the decision layer neurons. We assume four types of associative layer neurons, identified by their preferred stimulus pair. Thus the network's response to any input combination can be summarized by the firing rates of these four types of cells and expressed as a single point in a 4-dimensional (4-D) space of firing rates. For visualization, we project the 4-D space into 2-D, plotting the rate of the AB-responsive cells against the rate of the CD-responsive cells to each of the four possible stimulus pairs. Figure 2B is an example of such a plot and summarizes the results of Fig. 2A. In particular, the four responses fall on a straight line (the response to A + D is equal to the response to C + B in this projection, so only 3 circles are visible), indicating the linear dependence for all linearly responsive neurons:

$$r_i^{\text{A}+\text{B}}+r_i^{\text{C}+\text{D}}-r_i^{\text{A}+\text{D}}-r_i^{\text{C}+\text{B}}=0$$

for all cells i whose firing rate increases linearly with inputs A, B, C, and D. Equivalently, the combination of firing rates of the four cell types is linearly dependent:

$$r_{\text{AB}}^{X+Y}+r_{\text{CD}}^{X+Y}-r_{\text{AD}}^{X+Y}-r_{\text{CB}}^{X+Y}=0$$

for each stimulus pair X = A, C; Y = B, D. Thus, if synaptic transmission is simply proportional to presynaptic firing rate (dashed blue line denotes constant synaptic transmission), then the stimulus pairs cannot generate opposite responses. However, since the sign of

$$F\left(r_{\text{AB}}^{X+Y}\right)+F\left(r_{\text{CD}}^{X+Y}\right)-F\left(r_{\text{AD}}^{X+Y}\right)-F\left(r_{\text{CB}}^{X+Y}\right)\ne 0$$

is positive for the pairs X + Y =A + B and X + Y = C + D and opposite for the pairs X + Y = A + D and X + Y = C + B, synaptic transmission through facilitating synapses does separate these stimulus pairs (green line). Similarly, for depressing synapses,

$$D\left(r_{\text{AB}}^{X+Y}\right)+D\left(r_{\text{CD}}^{X+Y}\right)-D\left(r_{\text{AD}}^{X+Y}\right)-D\left(r_{\text{CB}}^{X+Y}\right)\ne 0,$$

although the sign of the result is opposite, so the points fall on the opposite side of the line of constant synaptic transmission (red curve) compared with facilitation (green curve).

In Fig. 3, the left columns demonstrate how either facilitating or depressing synaptic transmission can indeed produce high mean reward accumulation from neurons with linear responses. The differences in signs of the nonlinearity lead depressing synapses to connect preferentially to the incongruous decision-making pool (Fig. 3C, left column, middle row), as expected.

We should note that the schematic diagrams of Fig. 2, B–D, have two shortcomings. First, they are a 2-D projection of what is a 4-D space. However, the symmetry in our simplified model means the arguments are still valid; in particular, when the 4 points are linearly dependent, they fall on a 3-D hyperplane of 4-D space but are seen to fall on a line in this 2-D projection. Similarly, when a line can correctly separate the points in the 2-D projection, then any 3-D hyperplane that intersects the line will separate them in the full 4-D space.

The second shortcoming lies in the fact that just because the points can be correctly separated, meaning a set of synaptic weights can be chosen to correctly bias the decision-making pools, it does not mean that all reinforcement learning rules will lead to such a solution. Indeed, Fig. 3 demonstrates a dependence on the reinforcement learning rule. In biological terms, if the DA-modulated reward-based learning rule is purely postsynaptic, then any dependence on presynaptic firing rate must be through synaptic transmission. Thus a depressing synapse would lead to the presynaptic rate dependence of the reinforcement rule being sublinear, whereas for a facilitating synapse it would be superlinear, and for a static synapse it would be linear. Figure 3, A and C, show the results with such a reinforcement rule that matches the synaptic transmission. Alternatively, if we select the reinforcement rule (either linear, sublinear, or superlinear in presynaptic rate) that maximizes reward, we obtain the results from Fig. 3, B and D. The key features of the results do in fact match our insight from nonlinear synaptic transmission rates from Fig. 2, B–D. In particular, for a sublinear (saturating) neural response, the sublinear, depressing synapses produce greatest reward, whereas for superlinear (threshold) neural responses, the superlinear, facilitating synapses produce greatest reward. If the neural response is sublinear, or in some cases, if synapses are depressing, then a negative asymmetry arises in the synaptic connections, meaning that neurons connect preferentially to the incongruous decision-making pool, as in Fig. 3C (middle row).

Biconditional Discrimination: Spiking Network Simulations

Static synapses.

Static synapses from excitatory cells in the associative layer (Fig. 4B) to the decision-making layer (Fig. 4C) served as a control to assess the effect of facilitating or depressing synapses. In the biconditional discrimination task, one untrained associative layer with sparse input (1/20) could produce reliably correct decisions (at least 85% correct) by training only the excitatory synapses from the associative layer to the decision-making layer's excitatory cells via a DA-modulated reward-based Hebbian learning rule (5 other untrained networks produced borderline reliability of 76–84% correct). However, the addition of correlation-based synaptic plasticity (LTPi and triplet STDP), at inputs to and at recurrent synapses within the associative layer, increased stimulus pair selectivity and led to greater reward compared with the untrained associative layer (both with reward-based plasticity) (Bourjaily and Miller 2011a). Multiple trained associative layers (24%, 6 of 25) generated reliably correct decision-making with additional borderline networks (20%, 5 of 25) (Fig. 4F).

Facilitating synapses.

Because facilitating synapses produce superlinear synaptic transmission rates (Fig. 2A), we expected that facilitating synapses would improve task performance relative to static synapses by enhancing the superlinearity already present in the responses of associative layer cells. Indeed, the addition of facilitation increased the number of randomly connected initial networks that produced reliably correct decisions (8%, 2 of 25) as well as the number of borderline networks (20%, 5 of 25). Furthermore, nearly one-half of the trained associative networks (44%, 12 of 25) generated reliable decisions, whereas 20% were borderline (5 of 25) (Fig. 4G). Of the 12 reliable networks, 6 were significantly improved (P < 0.01) over networks using static synapses. Overall, 11 of 25 trained facilitating networks were significantly improved over those using static synapses (P < 0.05), and across all trained networks and input configurations, there was a significant improvement of ∼7% in decision-making accuracy (P < 0.001) in trained networks with facilitating synapses compared with those with static synapses to the decision-making layer.

To test whether facilitation simply increased accuracy only because it increased the strength of input from the associative layer to the decision layer, we varied initial synaptic strength. We observed the same number of reliable networks across a range of −25% to +25% of the default strength and a decrease of 2–3 reliable networks at ±50% of the default value. The loss of reliability arose because either no decision-making pool responded (too little drive) or both pools responded (too much drive), whereas networks with static synapses always produced a single active decision-making pool; indeed, their synaptic input strengths had been optimized to avoid such extremes. Thus the cause of increased performance was not simply a matter of linearly scaling the synaptic input strength.

The addition of facilitating synapses reduced the minimal amount of stimulus pair selectivity necessary to generate reliable decisions to a value of 0.88, compared with the value of 1.23 with static synapses. Furthermore, facilitating synapses reduced the mean time to reach a decision across all networks, on average 52 ms, compared with a value of 118 ms for static synapses, with the difference being significant for all 25 input configurations (P < 0.01 by 2-sample t-test in each case). Thus, for the biconditional discrimination task, facilitation enhanced both speed and accuracy of decisions.

Depressing synapses.

Depressing synapses produce sublinear synaptic transmission rates, counter to the observed superlinear activity responses in the associative network, so we expected they would worsen performance. Indeed, no untrained random associative networks with depressing synapses produced reliable decisions. Furthermore, only two of the trained associative networks (8%, 2 of 25) generated reliable decisions, of which one was significantly worse than the equivalent network with static synapses (P < 0.05), as well as a small number of networks with borderline accuracy (25%, 7 of 25) (Fig. 4E). Eight of 25 trained networks with depressing synapses were significantly worse than those with static synapses (P < 0.05), and there was a corresponding significant decrement in average decision-making accuracy of ∼13% (P < 0.001). The reduction in accuracy corresponded to a need for greater stimulus pair selectivity, a minimal value of 1.35, for reliable responses. Reaction times were also significantly longer for networks with depressing synapses (mean 158 ms) compared with static synapses, (mean 118 ms), with 17 of 25 networks producing significantly slower responses (P < 0.01). Thus depressing synapses worsened performance by both increasing reaction time and decreasing accuracy.

Selective potentiation of afferent synapses to the decision-making network.

We quantified selective potentiation across networks by the strength of afferent synapses to the congruous decision-making pool, that is, the decision-making pool, producing reward for the preferred stimuli of cells (e.g., the strength of synapse from an AB-tuned cell to “release”), versus that to the incongruous decision-making pool (e.g., the strength of synapse from an AB-tuned cell to “hold”). Compared with control networks, those with facilitating synapses produced an increase in the number of neurons projecting to the congruous decision-making pool and a corresponding decrease in the percentage of synapses to the incongruous decision-making pool. For networks with depressing synapses, we found the opposite to be the case and indeed found many cells to be connected preferentially to the incongruous decision-making pool (cf. Figs. 1E and 3C).

Stimulus-Response Association: Spiking Network Simulations

We simulated a more common task, stimulus-response association, to assess the value of nonlinear synaptic transmission when the task is linearly separable. Only a single stimulus was shown per trial and required to drive a decision. Because single-stimulus selectivity is inherently greater than stimulus pair selectivity, many untrained random associative layers with static synapses could produce reliable decision-making responses (84%, 21 of 25), a number that did not change upon training the associative layer with LTPi and triplet STDP. However, facilitating synapses caused all but one of the trained networks (96%, 24 of 25) to produce reliable decision-making, without affecting the number of reliable untrained networks. In the same task, depressing synapses worsened accuracy, with 76% (19 of 25 networks) producing reliable decision-making from both trained and untrained associative layers. Thus again, we found facilitation to be beneficial and depression to be detrimental to performance.

Two-Stimulus Discrimination Task: Firing Rate Model

Models of decision-making have mostly been based on a task requiring a comparison of the magnitude of two distinct stimuli (Ratcliff 1978; Usher and McClelland 2001; Wang 2002), such as whether coherence of rightward or leftward motion is greater in a pattern of randomly flickering dots (Shadlen and Newsome 1996). Thus we assessed how accuracy in such a task was affected by dynamic synapses at the inputs to the decision-making circuit. Figure 5A indicates that facilitating synapses boosted decision-making accuracy, whereas depressing synapses reduced accuracy. We found this to be true both when all of the noise in the system was in the inputs (using a deterministic decision-making circuit) as well as when the decision-making circuit itself was noisy. The deterministic effect was a straightforward consequence of the nonlinearity of synaptic transmission of dynamic synapses as a function of firing rate: facilitation enhanced whereas depression decreased the difference in postsynaptic conductance resulting from a difference in presynaptic firing rates (Dayan and Abbott 2001).

Fig. 5.

Facilitating synapses enhance accuracy in a 2-stimulus discrimination task. A–C: inputs differing by 5% of their mean are transmitted to separate decision-making pools through firing rate model cell groups, which respond either linearly (A), superlinearly (B), or sublinearly (C). Baseline synaptic strength of the inputs to the decision-making circuit is varied and decision-making accuracy is measured for input synapses that are static (blue), facilitating (green), or depressing (red). Performance peaks for a given total input that produces activation of just 1 of the 2 decision-making pools (no pools are active if input is too weak, whereas both are active if input is too strong). D: signal-to-noise ratio (SNR) of the postsynaptic conductance resulting from a Poisson input train is enhanced through facilitating synapses (green) and reduced through depressing synapses (red) compared with static synapses (blue). Solid lines indicate simulated data; dashed lines represent the analytic formula, assuming the synaptic time constant is much shorter than interspike intervals, an approximation that fails at high firing rates (see Appendix). Only the steady-state responses are measured and compared. Signal is produced by a difference in the presynaptic firing rate equal to 5% of the mean rate.

Dynamic synapses increase noise in the postsynaptic conductance, because they add variability to the amplitude of conductance pulses in addition to the variability in timing of spikes (see Lindner et al. 2009 and Appendix). However, for the typical parameters used, and even if all noise was at the level of inputs, via a combination of Poisson spike timing and fluctuating input firing rates, facilitating synapses led to higher decision-making accuracy (Fig. 5A) by increasing the signal-to-noise ratio (Fig. 5B). The benefit of facilitating synapses was even more striking if noise was greater within the decision-making circuitry than at the inputs, because in that case the deterministic enhancement of the signal by synaptic facilitation dominated its effect (data not shown). For all conditions with which we have studied the two-stimulus discrimination task, depressing input synapses reduced accuracy, as expected, because they reduced the downstream effect of any difference between input signals (Dayan and Abbott 2001) as well as enhancing noise (Lindner et al. 2009).

DISCUSSION

Short-term plasticity is a well-established feature of synapses in cortical circuits (Hempel et al. 2000; Markram et al. 1998b; Tsodyks and Markram 1997; Wang et al., 2006). Although presynaptic frequency-dependent dynamic changes in effective synaptic strength may be an inevitable consequence of the biological mechanisms underlying synaptic transmission, theoretical studies have shown that these dynamics, on a short timescale, can be extremely beneficial for multiple cortical functions (Maass and Zador 1999): filtering inputs in the frequency domain (Farajidavar et al. 2008; Matveev and Wang 2000; Natschlager et al. 2001; Tsodyks et al. 1998), dynamically controlling gain to enhance sensitivity to changes in inputs (Abbott et al. 1997), producing a temporal lag allowing for motion direction selectivity (Chance et al. 1998), providing a necessary slow time constant for memory maintenance (Barak and Tsodyks 2007; Martinez-Garcia et al. 2011; Mongillo et al. 2008), and producing an optimal estimator of presynaptic membrane potential dynamics (Pfister et al. 2010), among others. Here, we show how facilitating synapses can produce improved classifiers based on a nonlinear combination of presynaptic firing rates. Specifically, facilitation improves task performance by selectively enhancing synaptic transmission to a decision-making circuit from those input cells most responsive to a given stimulus. On the contrary, depressing synapses worsen performance by relatively enhancing synaptic transmission from less responsive cells. The effects of facilitation and depression depended monotonically on the time constant and amplitude of short-term plasticity. Enhancing the amplitude of facilitation and depression boosted their respective positive and negative changes in task performance more so than increasing the time constants.

Biconditional discrimination employs XOR logic, making it linearly nonseparable; synaptic facilitation provides nonlinearity necessary to solve such a task, even when associative layer neural responses are a linear combination of their inputs. In principle, any nonlinearity should allow a network to produce suitable performance in an XOR-like/linearly nonseparable task; thus we assessed whether depressing synapses could also be beneficial given the proper conditions. Indeed, for linearly responsive neurons, depressing synapses lead to good performance if associative layer neurons connect most strongly to the decision-making pool that produces the incongruous response to that of their preferred stimulus (e.g., AB associative neurons to “hold” decision pool neurons) (Figs. 1E and 5). Consistent with this expectation based on linear neurons, in simulated networks of spiking neurons trained with depressing synapses, we found an excess of these “incongruous” connected neurons. However, performance in these simulated networks proved to be worse with depressing synapses, because the saturating effect of synaptic depression on neural output opposed the superlinearity of the firing rate curve of spiking neurons in the associative layer. Thus, although synaptic facilitation enhanced the existing nonlinearity in the response of a biophysically based neural circuit, enhancing performance, synaptic depression reduced the nonlinearity, producing worse performance.

In addition to testing the afferent synapses to the decision-making network from the associative layer, we also incorporated short-term plasticity at the associative layer synapses plastic for triplet STDP and LTPi, independently/in turn. However, we did not find that the addition of short-term plasticity at these locations to and within the associative layer led to any appreciable difference in decision-making performance compared with static synapses as when applied to afferent synapses to the decision-making network, as presented here.

Many decision-making studies have looked at single-stimulus discrimination tasks based on motion coherence (Mensh et al. 2004; Wang 2002), natural images (Seung 2003; Theodoni et al. 2,011), and orientation (Suh et al. 2003), among others. Thus we included a stimulus-response association task and a discrimination task, to generalize our findings. Although responding selectively to distinct single stimuli is much easier than to stimulus pairs, facilitation and depression of inputs to the decision-making circuit still improved and worsened performance, respectively, in such tasks. Similarly, in a model of a discrimination task based on motion coherence (Shadlen and Newsome 1996), facilitation of synaptic input to the decision-making circuit enhanced discriminability, even when all of the noise in the system was in the stimulus and so transmitted through the same facilitating synapses, whereas depression of input synapses decreased discriminability (Fig. 5).

In summary, we find for three distinct types of tasks that networks with facilitating afferent synapses to a biologically based decision-making circuit produce responses that are both significantly more accurate (i.e., more often correct) and significantly quicker than those with either static or depressing synapses. In linearly nonseparable tasks, the superlinear synaptic transmission of facilitating synapses (Dayan and Abbott 2001) is advantageous because it boosts the inherent superlinearity in neural firing rates (Anderson et al. 2000; Carandini and Ferster 2000; Miller and Troyer 2002). In linearly separable tasks, facilitation can boost the signal-to-noise ratio by increasing the signal without producing a concomitant increase in the input noise (Lindner et al. 2009). Thus we expect synapses from associative areas to decision-making areas to be facilitating rather than depressing, as has been shown in hippocampal projections to prefrontal cortex (Mulder et al. 1997) and internal prefrontal connections (Berger et al. 2009; Hempel et al. 2000; Wang et al. 2006). We suggest that the optimal dynamics of a synapse may reflect not only the statistics of the presynaptic membrane potential (Pfister et al. 2010) but also the type of information it needs to transmit, given the computational function of the circuit.

GRANTS

We are grateful for financial support for this work from National Institute of Deafness and Other Communicative Diseases Award DC009945, the Department of Biology, Brandeis University, and from a National Science Foundation Integrative Graduate Education and Research Traineeship Award to the Neuroscience Graduate Program of Brandeis University.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

ENDNOTE

At the request of the authors, readers are herein alerted to the fact that additional materials related to this manuscript may be found at the institutional website of one of the authors, which at the time of publication they indicate is: http://people.brandeis.edu/∼pmiller/code.html. These materials are not a part of this manuscript, and have not undergone peer review by the American Physiological Society (APS). APS and the journal editors take no responsibility for these materials, for the website address, or for any links to or from it.

AUTHOR CONTRIBUTIONS

Author contributions: M.A.B. and P.M. conception and design of research; M.A.B. and P.M. performed experiments; M.A.B. and P.M. analyzed data; M.A.B. and P.M. interpreted results of experiments; M.A.B. and P.M. prepared figures; M.A.B. and P.M. drafted manuscript; M.A.B. and P.M. edited and revised manuscript; M.A.B. and P.M. approved final version of manuscript.

APPENDIX: SIGNAL-TO-NOISE RATIO FOR THE DIFFERENCE IN SYNAPTIC INPUT BETWEEN TWO POISSON SPIKE TRAINS

A decision-making circuit produces a decision based on the difference between two sets of synaptic inputs. We define a signal-to-noise ratio (SNR) for the difference between two synaptic conductance inputs, g₁ and g₂, from two presynaptic cells with respective rates r₁ and r₂, as

$$\text{SNR}=\frac{\langle g_1-g_2\rangle }{{\sigma }_{g_1-g_2}}=\frac{\langle g_1\rangle -\langle g_2\rangle }{\sqrt{{\sigma }_1^2+{\sigma }_2^2}},$$ (A1)

where 〈g₁〉 and 〈g₂〉 are the expected means of the two inputs with standard deviations σ₁ and σ₂, respectively. Note that any static scaling of the conductance of synapses will scale the means and standard deviations equally, not affecting the SNR.

However, the nonlinear rate-based filtering of a facilitating synapse increases the mean conductance of one set of inputs more than it does the other. In particular, if r₁ > r₂, then synaptic conductances are scaled, respectively, by the facilitation variables F₁ and F₂, with F₁ > F₂. Yet, because spike trains are not regular, the dynamics of facilitating synapses produces spike-to-spike variability in the postsynaptic conductance, which adds to the variability in postsynaptic input. Thus we must ask whether the boost to the signal produced by facilitating synapses is greater than the boost to the postsynaptic noise. We follow the methods of Lindner et al. (2009) below, to produce the formulas for mean and variance of synaptic transmission at steady state, via the synapses of our model.

Postsynaptic responses are defined by the conductance g(t), which increases to g₀F(t⁻)D(t⁻) immediately following a presynaptic spike at time t and then decays exponentially according to

$$\frac{\text{d}g\left(t\right)}{\text{d}t}=-\frac{g\left(t\right)}{{\tau }_s}.$$ (A2)

We assume the time constant τ_s is significantly shorter than the interspike interval. For AMPA receptor-mediated synapses, τ_s = 2 ms, so this condition is met for most realistic presynaptic spike trains. Note that F(t⁻) = 1 for static and depressing synapses, whereas D(t⁻) = 1 for static and facilitating synapses. In the following, we reproduce the analysis for facilitating synapses and later state the corresponding results from the equivalent derivation for depressing synapses.

For a spike train of average firing rate r, the mean postsynaptic conductance is

$$\langle g\rangle =g_{0}r{\tau }_s\langle F^{-}\rangle$$ (A3)

where 〈F⁻〉 = 〈F(t⁻)〉 is the expected mean facilitation variable immediately prior to a spike. The difference in mean conductance of two inputs to a decision-making circuit (the numerator in Eq. A1) is

$$\langle \Delta g\rangle =\langle g\rangle -\langle g_2\rangle \approx g_0{\Delta }_r{\tau }_s\overline{F}-g_0\overline{r}{\tau }_s\langle \Delta F\rangle ,$$ (A4)

where we assume Δ_r ≪ r̄ and r₁ = r̄ + Δ_r/2, r₂ = r̄ − Δ_r/2, 〈F₁〉 = F̄ + 〈ΔF〉/2, and 〈F₂〉 = F̄ − 〈ΔF〉/2.

The variance in postsynaptic conductance is calculated as

$$\langle g^2\rangle =\frac{g_0^{2}r{\tau }_s\langle F^2\rangle }{2}=\frac{g_0^{2}r{\tau }_s{\langle F\rangle }^2}{2}+\frac{g_0^{2}r{\tau }_s}{2}{\sigma }_F^2,$$ (A5)

where σ_F² is the variance in F. Thus the variance in conductance can be calculated as

$${\sigma }_g^2=\langle g^2\rangle -{\langle g\rangle }^2=g_0^{2}r{\tau }_s{\langle F\rangle }^2\left(\frac{1}{2}-r{\tau }_s\right)+\frac{g_0^{2}r{\tau }_s}{2}{\sigma }_{\text{F}}^2,$$ (A6)

where we have assumed rτ_s ≪ 1. It can be seen for synaptic facilitation that F > 1, so the conductance variance, and hence noise in the postsynaptic cell, is higher because of both a multiplicative increase in the conductance (first term) and, through σ_F², the variance in F. For the input to a decision-making circuit, the variance (the noise term) in the denominator of Eq. A1 is increased as

$${\sigma }_1^2+{\sigma }_2^2\approx 2g_0^2\overline{r}{\tau }_s{\overline{F}}^2\left(\frac{1}{2}-\overline{r}{\tau }_s\right)+g_0^2\overline{r}{\tau }_s\overline{{\sigma }_{\text{F}}^0}.$$ (A7)

Combining Eqs. A4 and A7 with Eq. A1 leads to the SNR for dynamic synapses,

$$\text{SNR}=\frac{g_0{\Delta }_r{\tau }_s\overline{F}+g_0\overline{r}{\tau }_s\langle \Delta F\rangle }{\sqrt{2g_0^2\overline{r}{\tau }_s{\langle \overline{F}\rangle }^2\left(\frac{1}{2}-\overline{r}{\tau }_s\right)+g_0^2\overline{r}{\tau }_s\overline{{\sigma }_{\text{F}}^2}}},$$ (A8)

which requires us to calculate the mean and variance of the facilitation variable, F.

We note that F decays to 1 between spikes with a time constant τ_F as

$$\frac{\text{d}F}{\text{d}t}=\frac{1-F}{{\tau }_F},$$ (A9)

whereas immediately following a spike, it increases from its prior value, F⁻, to F⁺ according to

$$F^{+}=F^{-}+f_F\left(\frac{1}{p_0}-F^{-}\right)=F^{-}+f_F\left(F_{\max }-F^{-}\right),$$ (A10)

where f_F is the facilitation factor indicating the amount of facilitation and p₀ is the initial, baseline vesicle release probability in the absence of spikes. The term in brackets prevents the release probability increasing above unity so that the maximum value of facilitation, F_max, is 1/p₀. The postsynaptic conductance produced by each spike is proportional to F⁻, the value of F immediately before the spike, whose mean and variance we now calculate.

For a Poisson train, the expected value of F⁻ is given by (see Dayan and Abbott 2001)

$$\langle F^{-}\rangle =1+\frac{r{\tau }_F{f}_F\left(\frac{1}{p_0}-1\right)}{1+r{\tau }_F{f}_F}.$$ (A11)

Thus, for the two input trains to a decision-making circuit, the difference in facilitation factor becomes

$$\langle \Delta F\rangle =\langle F_1\rangle -\langle F_2\rangle \approx {\Delta }_r\frac{{\tau }_F{f}_F\left(\frac{1}{p_0}-1\right)}{{\left(1+\overline{r}{\tau }_F{f}_F\right)}^2}=\frac{{\Delta }_r\left(\langle \overline{F}\rangle -1\right)}{\overline{r}\left(1+\overline{r}{\tau }_F{f}_F\right)}.$$ (A12)

From calculations similar to those leading to Eq. A9 (see Dayan and Abbott 2001 and also Lindner et al. 2009), the variance in F⁻ is given by

$$\text{Var}\left(F^{-}\right)=\frac{r{\tau }_F{f}_F^2{\left(\frac{1}{p_0}-1\right)}^2}{{\left(1+r{\tau }_F{f}_F\right)}^2\left[2+r{\tau }_F{f}_F\left(2-f_F\right)\right]};$$ (A13)

and thus the standard deviation in F⁻ can be written as

$$\sigma \left(F^{-}\right)=\frac{\langle F^{-}\rangle -1}{\sqrt{r{\tau }_F\left[2+r{\tau }_F{f}_F\left(2-f_F\right)\right]}},$$ (A14)

which becomes much smaller than the increase in synaptic response (proportional to 〈F⁻〉) if rτ_F is large compared with 1. Because the time constant for facilitation is on the order of hundreds of milliseconds, for typical firing rates needed to drive a decision-making circuit, the added variability in synaptic response due to variation in excitatory postsynaptic potential (EPSP) amplitudes is small compared with the boost in EPSP amplitude.

Substituting Eqs. A11, A12, and A14 into Eq. A8 and assuming Δ_r ≪ r̄ leads to the SNR for the difference in Poisson spike trains with facilitating synapses as

$$\text{SNR}={\Delta }_r{\tau }_s\frac{\left[1+\frac{\overline{r}{\tau }_F{f}_F\left(\frac{1}{p_0}-1\right)}{1+\overline{r}{\tau }_F{f}_F\left(1+\overline{r}{\tau }_F{f}_F/p_0\right)}\right]}{\sqrt{2\overline{r}{\tau }_s\left(\frac{1}{2}-\overline{r}{\tau }_s\right)+\frac{{\overline{r}}^2{\tau }_s{\tau }_F{f}_F^2{\left(\frac{1}{p_0}-1\right)}^2}{\left[2+\overline{r}{\tau }_F{f}_F\left(2-f_F\right)\right]{\left(1+\overline{r}{\tau }_F{f}_F/p_0\right)}^2}}}$$ (A15)

Dashed curves in Fig. 5D are plotted using Eq. A15 with f_F = 0.4 and f_F = 0 for facilitating and static synapses, respectively, demonstrating a boost to SNR by synaptic facilitation.

Depressing synapses, with a fraction D_frac of vesicles released per spike and recovery with time constant τ_D, lead to similar results, but the equivalent of 〈ΔF〉 is negative,

$$\langle \Delta D\rangle =\langle D_1\rangle -\langle D_2\rangle \approx -{\Delta }_r\frac{{\tau }_D{D}_{\text{frac}}}{{\left(1+{\overline{r}}_D{D}_{\text{frac}}\right)}^2},$$ (A16)

so depressing synapses reduce the signal as well as increasing the noise. Thus, for depressing synapses, the SNR is always reduced by an amount that can be calculated as

$$\text{Var}\left(D^{-}\right)=\frac{r{\tau }_D{D}_{\text{frac}}^2}{{\left(1+r{\tau }_D{f}_D\right)}^2\left[2+r{\tau }_D{D}_{\text{frac}}\left(2-D_{\text{frac}}\right)\right]},$$ (A17)

to give

$$\text{SNR}={\Delta }_r{\tau }_s\frac{\left[1-\frac{\overline{r}{\tau }_D{D}_{\text{frac}}}{\left(1+\overline{r}{\tau }_D{D}_{\text{frac}}\right)}\right]}{\sqrt{2\overline{r}{\tau }_s\left(\frac{1}{2}-\overline{r}{\tau }_s\right)+\frac{{\overline{r}}^2{\tau }_s{\tau }_D{D}_{\text{frac}}^2}{\left[2+\overline{r}{\tau }_D{D}_{\text{frac}}\left(2-D_{\text{frac}}\right)\right]},}}$$ (A18)

which is shown as the dashed red curve in Fig. 5D, with D_frac = 0.5 and τ_D = 200 ms.