Fig. 2.

Comparison of basic learning rules. Eligibility traces are denoted as E, while x ₀ (or r respectively) represents the unconditioned input or reward and x ₁ the conditioned input. The amplifier symbol denotes the changing synaptic weight. All models use the derivative of the postsynaptic signal in order to control the weight change. Both Sutton and Barto models a, b use eligibility traces only in the learning pathway, whereas ISO Learning c and TD-Rephrased d introduce them also in the other pathway. The rephrased TD model d is mixture of models b and c. We introduce a multiplicative factor α to control the impact of u ₀ on the δ-value. Setting α → 0 we reobtain ISO Learning, setting ω ₀ = 0 we get TD Learning (with a filtered x ₀ path instead of an unfiltered)