Algorithm 1 The forward propagation algorithm of CNN

1 //process of the forward propagation

2 Input: training set $D={\left\{(x_n,y_n)\right\}}_{n=1}^N$; the number of CNN layers is L, each layer denoted as $h_l$; $a_n^l$ represents the $n\mathrm{th}$ input sample corresponding to the output of layer $l$; expected output $t_n^v$.

3 Process:

4 Initialization: Initialize all layers of convolutional kernel $w^l$ and offset term $b^l$.

5 for all $(x_n,y_n)\in D$ do

6  for $(l=1;l\le L;l+1)$ do

7   if ($h_l$ is the convolutional layer) then

8     for (all $a_n^l$) do

9     get $a_n^l$ according to (1)

10    end

11   end

12   if ($h_l$ is the pooling layer) then

13    for (all $a_n^l$) do

14     get $a_n^l$ according to (7)

15    end

16   end

17   if ($h_l$ is fully connected layer)

18    for(all $a_n^l$) do

19     get $a_n^l$ according to (8)

20    end

21   end

22   if ($h_L$ is output layer) then

23    get $a_n^L$ or $o_n^v$ according to (9)

24   end

25  end

26 end

27 Output: Calculate the error $E$ of the output layer by the loss function, according to (11).