Figure 1. Synaptic drive and spiking rate of neurons in a recurrent network can learn complex patterns.

(a) Schematic of network training. Blue square represents the external stimulus that elicits the desired response. Black curves represent target output for each neuron. Red arrows represent recurrent connectivity that is trained to produce desired target patterns. (b) Synaptic drive of 10 sample neurons before, during and after training. Pre-training is followed by multiple training trials. An external stimulus (blue) is applied prior to training for 100 ms. Synaptic drive (black) is trained to follow the target (red). If the training is successful, the same external stimulus can elicit the desired response. Bottom shows the spike rater of 100 neurons. (c) Top, The Pearson correlation between the actual synaptic drive and the target output during training trials. Bottom, The matrix (Fresenius) norm of changes in recurrent connectivity normalized to initial connectivity during training. (d) Filtered spike train of 10 neurons before, during and after training. As in (b), external stimulus (blue) is applied immediately before training trials. Filtered spike train (black) learns to follow the target spiking rate (red) with large errors during the early trials. Applying the stimulus to a successfully trained network elicits the desired spiking rate patterns in every neuron. (e) Top, Same as in (c) but measures the correlation between filtered spike trains and target outputs. Bottom, Same as in (c).

Figure 1—figure supplement 1. Learning arbitrarily complex target patterns in a network of rate-based neurons.

The network dynamics obey ${\displaystyle \tau {\dot{x}}_i=-x_i+\sum _{j=1}^N{W}_{ij}{r}_j+I_i}$ where ${\displaystyle r_j=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left(x_j\right)}$. The synaptic current $x_i$ to every neuron in the network was trained to follow complex periodic functions ${\displaystyle f\left(t\right)=A\mathrm{s}\mathrm{i}\mathrm{n}\left(2\pi \left(t-T_0\right)/T_1\right)\mathrm{s}\mathrm{i}\mathrm{n}\left(2\pi \left(t-T_0\right)/T_2\right)}$ where the initial phase $T_0$ and frequencies $T_1,T_2$ were selected randomly. The elements of initial connectivity matrix $W_{ij}$ were drawn from a Gaussian distribution with mean zero and standard deviation $\sigma /\sqrt{Np}$ where $\sigma =2$ was strong enough to induce chaotic dynamics; Network size $N=500$, connection probability between neurons $p=0.3$, and time constant $\tau =10$ ms. External input $I_i$ with constant random amplitude was applied to each neuron for 50 ms (blue) and was set to zero elsewhere. (a) Before training, the network is in chaotic regime and the synaptic current (black) of individual neurons fluctuates irregularly. (b) After learning to follow the target trajectories (red), the synaptic current tracks the target pattern closely in response to the external stimulus.

Figure 1—figure supplement 2. Training a network that has no initial connections.

The coupling strength of the initial recurrent connectivity is zero, and, prior to training, no synaptic or spiking activity appears beyond the first few hundred milliseconds. (a) Training synaptic drive patterns using the RLS algorithm. Black curves show the actual synaptic drive of 10 neurons and red curves show the target outputs. Blue shows the 100 ms external stimulus. (b) Correlation between synaptic drive and target function (top) and the Frobenius norm of changes in recurrent connectivity normalized to initial connectivity during training (botom). (c and d) Same as in (a) and (b), but spiking rate patterns are trained.