Algorithm1. Training procedure of the SPA model

$\mathrm{Input}: \mathrm{Original} \mathrm{input} \mathrm{frame} \mathrm{clips} x^g$

Output: $\theta$ of SPA model

$1: \mathrm{Initialize} x^p$, ${\theta }_0$, $t=0$

$2: f_{global}=Feature\_extractor\left(x^g\right)$

$3: \mathbf{while} k\le K$do

$4: \hspace{1em}\mathbf{while} t\le T$do

$5: \hspace{1em}\hspace{1em}{f}_{saliency}=Feature\_extractor\left(x_t^p\right)$

6:     Get logits through $f_{global}$, $f_{saliency}$

7:      $s_t^r=cat\left(f_{global}, f_{saliency},logits\right)$

$8: \hspace{1em}\hspace{1em}\mathrm{Get} \left\{a_m^t\right\}$ from Policy $\pi \left(a_t|s_t^r;\theta \right)$

$9: \hspace{1em}\hspace{1em}\mathrm{Crop} x^g$ by $\left\{a_m^t\right\}$ getting $x_{t+1}^p$

$10: \hspace{1em}\hspace{1em}{r}_t=reward\left(x^g,x_t^p,x_{t+1}^p\right)$

$11: \hspace{1em}\hspace{1em}\mathrm{Estimate} \mathrm{advantage} {\widehat{A}}_t$

12:     $t=t+1$

13:   end while

$14: \hspace{1em}\mathbf{while} j\le N_{batch}$do

$15: \hspace{1em}\hspace{1em}\mathrm{Calculate} {ℒ}_{{\theta }_k}^{CLIP}\left({\theta }_k^j\right)$

$16: \hspace{1em}\hspace{1em}\mathrm{Update} {\theta }_k^j$ with gradient $\nabla {ℒ}_{{\theta }_k}^{CLIP}\left({\theta }_k^j\right)$

17:   end while

18:   $k=k+1$

19: end while