Figure 5.

Dynamic ASNN states (Figure 4) involved in an OC procedure. The Cue neuron (A) emits spikes in reaction to regular vibes pacing into the virtual world. The strong synaptic link between Cue and Action first commands the robot to do actions (C) implemented as sound emissions. Because the synapse between Cue and Action incorporates a habituation rule which decrease the synaptic efficiency (D), the PSP eventually fades enough to stop the action around cycle 100 (B). Simultaneously, Cue emits spikes toward the Predictor neuron, causing small PSPs (E). During the first 100 cycles, the synaptic weight (F) between Cue and Predictor is not strong enough to lead to a spike. The Reinforcer sensory neuron (G) spikes unconditionally from a light perception producing unvarying strong PSPs (H) toward the Predictor neuron (I). The reward comes from the virtual world in response to the action. The spiking pattern of Predictor results from the integration of growing PSPs from Cue/STDP and the PSP of the Reinforcer. Stopping the reward still produces the desired behavior in an expectative manner due to OC.