Algorithm 1 Teacher Network Pre-training

INPUT: Teacher network ${\mathcal{T}}_{\theta }$, training dataset $\mathbf{X}$, the number of transformation K

OUTPUT: Pre-trained teacher network ${\mathcal{T}}_{\theta }$

1:Initialize the teacher network ${\mathcal{T}}_{\theta }(·)$  2:Initialize the dataset ${\mathbf{X}}_T$  3:for$i=0,1,\cdots K$do  4:  Calculate the k-th transformed dataset ${\mathbf{X}}_k$ using Equation (2)  5:  Add ${\mathbf{X}}_k$ to ${\mathbf{X}}_T$  6:end for  7:Optimize the loss function described in Equation (5)