Fig. 3.

Learning by forgetting. a, b Digit representations learnt with digits 0 through 2 shown sequentially to an Spiking Neural Network (SNN) (with nine excitatory neurons) trained with standard spike-timing dependent plasticity (STDP) (a) and adaptive synaptic plasticity (ASP) that integrates habituation (b). Presenting the digits one-by-one sequentially i.e., first all the images for digit 0 followed digit 1, and so on can be treated as a dynamic learning environment. No particular digit instance or class is re-shown to the network. SNN trained with STDP tried to learn the new digit representation (for instance, digit 1) while retaining a portion of the old data (for instance, digit 0). However, fixed network size and absence of data reinforcement (i.e., no old data or digit showing with the new data) resulted in accumulation causing new weight updates to coalesce with already learnt patterns rendering the network incapable of categorizing the digits. In sharp contrast, ASP-learnt SNN, with identical resource constraints in place, gracefully forgets old patterns and adapts to learn new inputs effectively without catastrophically erasing old data. Supplementary Fig. 10 shows the representations learnt for a larger network when all digits 0 through 9 are presented. The color intensity of the patterns are representative of the value of synaptic weights with lowest intensity (white) corresponding to a weight value of −0.5 and highest intensity (black) corresponding to 0.5