Table 1:
BP | FA | DFA | SF | KP | WM | PAL | PFA | |
---|---|---|---|---|---|---|---|---|
| ||||||||
No need to transport weight sign | ✗ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ |
No need to transport weight magnitude | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
No separate feedback weight learning phase | ✓ | ✓ | ✓ | ✛ | ✓ | ✗ | ✓ | ✓ |
No explicit weight symmetry after training | ✗ | ✽ | ✽ | ✽ | ✗ | ✗ | ✽ | ✓ |
Accurate approximation to BP (path alignment) | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ |
BP-level task performance | ✓ | ✗ | ✗ | ✦ | ✓ | ✓ | ✦ | ✓ |
BP: backpropagation. FA: feedback alignment. DFA: direct feedback alignment. SF: sign-concordant feedback. KP: Kollen-Pollack algorithm. WM: weight mirror. PAL: phaseless alignment learning. PFA: product feedback alignment.
: It is unclear how the feedback weights in SF can be learned in a biologically plausible way.
: these algorithms reduce, but do not fully eliminate explicit weight symmetry.
: These algorithms significantly outperform FA and DFA, but still underperform compared to BP in more challenging tasks (CIFAR10 for PAL and ImageNet for SF).