Figure 2.

Backpropagation in one neuron and opinion update of weight and output. The backpropagation process in neural network training first compares the true label and output given by the neuron, then back propagates the difference to net, and adjusts the weight accordingly to minimize the error. The weight opinion update process mimics the backpropagation: (i) At current episode, the opinion of neuron is the combined opinion of forward opinion and backward opinion, which are based on current ${\overline{W}}_{weight}$, and current ${\overline{W}}_{output|label}$, respectively. (ii) Then in the next episode, the opinion of neuron will be recalculated by taking updated ${\overline{W}}_{weight}$ and ${\overline{W}}_{output|label}$ into consideration.