Fig 4. Learning of longer-term memory dependent computations with spiking neural networks.

(a-c): Delayed reaction and time interval estimation: The synaptic output of a CSN learns to generate a generic reaction several seconds after a short input. Upper panels show typical examples of input, desired and actual reactions (green, yellow and red traces). In the three panels, the desired reaction delay is the same (9sec), the networks (CSNs with saturating synapses) have different levels of recurrent connection strengths ((a), (b), (c): low, intermediate, high level). The generation of the reaction is best for the network with intermediate level of connection strength. The CSNs with lower or higher levels have not maintained sufficient memory due to their extinguished or noisy and likely chaotic dynamics (gray background lines: spike rates of individual neurons). The median errors of responses measured for different delays in ensembles of networks (levels of connection strength as in the upper panels), are given in the lower panels. The shaded regions represent the area between the first and third quartile of the response errors. Dashed lines highlight delay and error size of the examples in the upper panels. (d): Persistent retaining of instructions and switching between computations: The network receives (i) two random continuous operand inputs (upper sub-panel, yellow and purple traces), and (ii) two pulsed instruction inputs (middle sub-panel, blue and green; memory of last instruction pulse: red). The network has learned to perform different computations on the operand inputs, depending on the last instruction (lower subpanel): if it was +1 (triggered by instruction channel 1), the network performs a nonlinear computation, it outputs the absolute value of the difference of the operands (red trace (network output) agrees with blue); if it was -1 (triggered by channel 2), the values of the operands are added (red trace agrees with green trace).