Algorithm 1 Original Stepwise Weight Pruning Algorithm (SWPA) [43]

Input: training data $X\in {\mathbf{R}}^{n\times d}$, training labels Y, base network $f_{\theta }(.)$, Drop-in Layer W, Step Counter $n\in {\mathbf{Z}}_{⩾1}$. Selection factor $f\in [0,1]$ for $count$ in $1,\cdots ,n+1$ do     $O\leftarrow \{w_1{x}_1,\cdots ,w_d{x}_d\}$     if $count$ > 1 then         k$\leftarrow \frac{(1-f)·d}{n}$         Sort the weights W of the Drop-in Layer based on their absolute value.         Set the least k of them to 0.     Train the base network on O Take the features corresponding to the top f fraction of the weights in W based on their absolute value and train them on the base network.