Algorithm 1 Motion vectors estimation, updating, and refinement.

Require: Low-light Surveillance Video $\mathbf{I}\in {\mathbb{R}}^{m\times n\times t}$, Block Size $Bs=8$, Padding Pixel $Pp=16$, Overlap Pixel $Op=4$, Search Window Size $Sws=6$, Vector Updating Number $num=0$, Rationality Matrix $\mathbf{R}$, Reasonable Degree $\beta =0.96$, and Small Search Window Size $SSws=2$. Ensure: The Final Vector $\mathbf{V}$ and Rationality Matrix $\mathbf{R}$. 1:   Initialization: $M={\displaystyle \frac{m}{Bs}}$, $N={\displaystyle \frac{n}{Bs}}$, Zero Matrix $\mathbf{V}\in {\mathbb{R}}^{M\times N\times (t-1)}$, and $\mathbf{R}\in {\mathbb{R}}^{M\times N\times (t-1)}$ 2:   for each $k\in [2,t]$ do 3:       for each $a\in [1,M]$ do 4:           for each $b\in [1,N]$ do 5:              Get the areas $A_{ab}$, $S{A}_{ab}$, and $SS{A}_{ab}$. 6:              Solve Equation (2) to get the vector $(\Delta x,\Delta y)$. 7:              Repeat: Compute $A{L}_{ab}$, $P{L}_{ab}$ using Equation (3). 8:              Update $R_{ab}$, ${\mathbf{V}}_{ab}$ using Equations (4) and (5). 9:              $num=num+1$, and $\mathbf{R}(a,b,k-1)=R_{ab}$. 10:             While: ${\displaystyle \frac{{\sum }_{a=1}^M{\sum }_{b=1}^N{R}_{ab}}{MN}}\ge \beta$ or $num=15$. 11:            Get the updated vector $(\Delta x^{\prime },\Delta y^{\prime })$. 12:            Solve Equation (8) to get the best vector. 13:            $\mathbf{V}(a,b,k-1)=(\Delta x^{\prime }+V{C}_X,\Delta y^{\prime }+V{C}_Y)$. 14:            Update $R_{ab}$ using Equation (5) for each vector. 15:            $\mathbf{R}(a,b,k-1)=R_{ab}$. 16:         end for 17:     end for 18: end for 19: return The final vector V and rationality matrix R;