Algorithm 1.

Difference Target Propagation with pooling layers (single-step)

1: Forward functions fl = conv + pooling, l = 1, …, L

2: Backward functions gl = strided deconv, l = 1, …, L − 1

3: Input x0

4:

5: for l = 1 to L do

6:   xl = fl(xl − 1)

7: end for

8:

9: for l = L to 2 do

10:   ${\mathcal{L}}_l^{inv}=\parallel g_l(f_l({\text{x}}_{l-1}))-{\text{x}}_{l-1}{\parallel }_2^2$

11:   Update parameters in gl by minimizing ${\mathcal{L}}_l^{inv}$ using SGD

12: end for

13:

14: Set the first target: ${\widehat{\text{x}}}_L={\text{x}}_L-{\eta }_L\frac{\partial {\mathcal{L}}_L}{\partial {\text{x}}_L}$

15: for l = L to 2 do

16:   ${\widehat{\text{x}}}_{l-1}={\text{x}}_{l-1}-g_l({\text{x}}_l)+g_l({\widehat{\text{x}}}_l)$

17: end for

18:

19: for l = 1 to L do

20:   ${\mathcal{L}}_l=\parallel f_l({\text{x}}_{l-1})-{\widehat{\text{x}}}_l{\parallel }_2^2$ if l < L

21:   ${\mathcal{L}}_l=\mathcal{L}$ if l = L

22:   Update parameters in fl by minimizing ${\mathcal{L}}_l$ using SGD

23: end for