Fig. 6. Information flow between the forward and backward pathways.

a Both layers are trained with the rule $\mathcal{F}\left(\mathit{\Theta }\right)={\theta }_0{\mathcal{F}}^0$ via feedback alignment. In this case, information from B_2,1 is transmitted to W_0,1 through ${\mathcal{F}}^0$ (①) and then propagated forward to W_1,2 (②). b The first layer is updated with the rule $\mathcal{F}\left(\mathit{\Theta }\right)={\theta }_0{\mathcal{F}}^0$ via feedback alignment, while the second layer uses ${\mathcal{F}}^{\mathrm{eHebb}}\left(\mathit{\Theta }\right)={\theta }_0{\mathcal{F}}^0+{\theta }_2{\mathcal{F}}^2$. Using ${\mathcal{F}}^0$, information from B_2,1 is communicated to W_1,2 (①,③); meanwhile, the presence of ${\mathcal{F}}^2$ sets up a new channel to directly communicate information from B_2,1 to W_1,2 (②). The blue arrows depict information propagation through the forward and backward paths. The communications between feedback and feedforward pathways are represented with red arrows.