Algorithm 2 The reverse propagation algorithm of CNN

1  //process of the reverse propagation

2 Input: The error $E$ of the output layer calculated by the loss function, the learning rate $\gamma ,\gamma \in (0,1)$.

3  Process:

4  for $(l=L;l\ge 1;l=l-1)$ do

5    if (${\delta }_n^{l+1}$ is fully connected layer) then

6    get ${\delta }_n^l$ according to ${\delta }_n^l={(w^{l+1})}^T{\delta }_n^{l+1}\odot f^{\prime }(w^l{a}_n^{l-1}+b^l)$

7    end

8    if (${\delta }_n^{l+1}$ is the convolutional layer) then

9    get ${\delta }_n^l$ according to ${\delta }_n^l={\delta }_n^{l+1}\frac{\partial [f(z^l)\times w^{l+1}+b^{l+1}]}{\partial z^l}\times rot180(w^{l+1})\odot f^{\prime }(w^l\times a_n^l+b^l)$

10   end

11   if (${\delta }_n^{l+1}$ is the pooling layer) then

12    get ${\delta }_n^l$ according to ${\delta }_n^l=upsample({\delta }_n^{l+1})\odot f^{\prime }(pool(a_n^l))$

13   end

14  end

15  for $(l=2;l\le L;l=l+1)$ do

16   if($h^l$ is fully connected layer) then

17    $w^l=w^l-\gamma {\displaystyle \sum _{n=1}^N{\delta }_n^l{(a_n^{l-1})}^T}$

18    $b^l=b^l-\gamma {\displaystyle \sum _{n=1}^N{\delta }_n^l}$

19   end

20   if($h^l$ is the convolutional layer) then

21    $w^l=w^l-\gamma {\displaystyle \sum _{n=1}^N{\delta }_n^l\times rot180(a_n^{l-1})}$

22    $b^l=b^l-\gamma {\displaystyle \sum _{n=1}^N{\displaystyle \sum _{u,v}{({\delta }_n^l)}_{u,v}}}$

23   end

24  end

25 Output: Updated values for $w$ and $b$.