Algorithm 1 Estimating Homography for a Foreground in Current Frame

Input: Infrared foreground: $B^{IR}$. visible foreground: $B^{VIS}$. Moving state: $F^{IR}$.

Reservoir group: $\left\{R^{cur}\right\}$. Previous homography group: $\left\{M{q}^{t-1}\right\}$. Global matrix: $M{g}_t$

Output: current frame-wide homography group: $\left\{M{q}^t\right\}$

Proceduce Homography estimation

If $\mathrm{size}\left(\left\{R^{cur}\right\}\right)=1$

$R^{cur}\to H{q}^{cur}$, $E^{cur}=1-\mathsf{\Gamma }\left(B^{IR},H{q}^{cur}\right){{\displaystyle \cap }}^{\text{}}{B}^{VIS}∕\mathsf{\Gamma }\left(B^{IR},H{q}^{cur}\right){{\displaystyle \cup }}^{\text{}}{B}^{VIS}$

If $F^{IR}=\mathrm{tracking}\mathrm{normally}$ $M{q}_{ref}^{t-1}=M{q}^{t-1}$ end if;

If $F^{IR}=\mathrm{fragmentation}$ $M{q}_{ref}^{t-1}={\displaystyle \sum }_{i=1}^{N}M{q}_i^{t-1}∕N$ end if;

If $F^{IR}=\mathrm{entering}\mathrm{or}\mathrm{lost}\mathrm{tracking}$ $M{q}_{ref}^{t-1}=M{g}_t$ end if;

$E_{ref}^{t-1}=1-\mathsf{\Gamma }\left(B^{IR},M{q}_{ref}^{t-1}\right){{\displaystyle \cap }}^{\text{}}{B}^{VIS}∕\mathsf{\Gamma }\left(B^{IR},M{q}_{ref}^{t-1}\right){{\displaystyle \cup }}^{\text{}}{B}^{VIS}$

$H{q}^{cur},E^{cur},M{q}_{ref}^{t-1},E_{ref}^{t-1}\to M{q}^t$

Else

$B^{IR}\to B_1^{IR},B_2^{IR},\dots ,B_M^{IR}$

For $R_j^{cur}\in R_1^{cur},R_2^{cur},\dots ,R_M^{cur}$ do

$R_j^{cur}\to H{q}_j^{cur}$, $E_j^{cur}=1-\mathsf{\Gamma }\left(B_j^{IR},H{q}_j^{cur}\right){{\displaystyle \cap }}^{\text{}}{B}^{VIS}∕\mathsf{\Gamma }\left(B_j^{IR},H{q}_j^{cur}\right){{\displaystyle \cup }}^{\text{}}{B}^{VIS}$

$M{q}_{ref}^{t-1}=M{q}_j^{t-1}$, $E_{ref}^{t-1}=1-\mathsf{\Gamma }\left(B_j^{IR},M{q}_{ref}^{t-1}\right){{\displaystyle \cap }}^{\text{}}{B}^{VIS}∕\mathsf{\Gamma }\left(B_j^{IR},M{q}_{ref}^{t-1}\right){{\displaystyle \cup }}^{\text{}}{B}^{VIS}$

$H{q}_j^{cur},E_j^{cur},M{q}_{ref}^{t-1},E_{ref}^{t-1}\to M{q}_j^t$

End for

End if

End proceduce