Table 2.

Postsynaptic information required by deep synapses for optimal learning. $I_{ij}^h$ represents the signal carried by the deep learning channel and the postsynaptic term in the learning rules considered here. Different algorithms reveal the essential ingredients of this signal and how it can be simplified. In the last row, the function F can be implemented with sparse or adaptive matrices, carry low precision signals, or include non-linear transformations in the learning channel (see also [4]).

Information	Algorithm
$I_{ij}^h=I_{ij}^h(T,O,w_{rs}^l(l>h),f^{\prime }(l\ge h))$	General Form
$I_{ij}^h=I_i^h(T,O,w_{rs}^l(l>h),f^{\prime }(l\ge h))$	BP (symmetric weights)
$I_{ij}^h=I_i^h(T-O,w_{rs}^l(l>h),f^{\prime }(l\ge h))$	BP (symmetric weights)
$I_{ij}^h=I_i^h(T-O,w_{rs}^l(l>h+1),w_{ki}^{h+1},f^{\prime }(l\ge h))$	BP (symmetric weights)
$I_{ij}^h=I_i^h(T-O,r_{rs}^l(l\ge h+1),r_{ki}^h,f^{\prime }(l\ge h))$	RBP (random weights)
$I_{ij}^h=I_i^h(T-O,r_{ki}^h,f^{\prime }(l\ge h))$	SRBP (random skipped weights)
$I_{ij}^h=I_i^h(T-O,r_{ki}^h,f^{\prime }(l=h))$	SRBP (random skipped weights)
$I_{ij}^h=I_i^h(F(T-O),f^{\prime }(l=h))$	F sparse/low-prec./adaptive/non-lin.