|
Algorithm 1 Flow of the proposed AL-PP algorithm |
-
Input:
Training dataset , where n denotes the number of training samples; fixed pre-trained baseline network ; pruning network ; discriminator ; sparsity factor ; polarization parameter ; maximum number of training epochs N.
-
1:
Phase 1: Sparsity training with performance recovery
-
2:
Load the parameters of the pre-trained baseline network .
-
3:
Initialize the pruning network parameters and the soft masks m, as well as the discriminator parameters .
-
4:
for to N do
-
5:
Fix baseline network and update discriminator
-
6:
Feed the training samples into the baseline network and the pruning network to obtain the outputs and .
-
7:
Update the discriminator parameters by maximizing the following objective using stochastic gradient descent (SGD):
-
8:
Fix discriminator and update pruning network
-
9:
Update the pruning network parameters and the soft masks m by minimizing the following objective using SGD:
-
10:
end for
-
11:
Phase 2: Structured channel pruning
-
12:
Evaluate the importance of each channel according to the learned soft mask values.
-
13:
Retain channels with large mask values and prune channels with small mask values to obtain the compact pruned model.
-
Output:
The compact model after structured pruning.
|