Algorithm 1: Training details of FusionADA

Parameter definitions

$N_G$, $N_D$: The numbers of steps for training G, D.

${\mathcal{L}}_{max}$, ${\mathcal{L}}_{min}$ and ${\mathcal{L}}_{Gmax}$ are applied to determine a range when training.

${\mathcal{L}}_{max}$ and ${\mathcal{L}}_{min}$ mean the adversarial losses of G and D.

${\mathcal{L}}_{Gmax}$: the total loss of G.

We set ${\mathcal{L}}_{max}=1.387$, ${\mathcal{L}}_{min}=1.386$, and ${\mathcal{L}}_{Gmax}=0.1$ in the first batch empirically in our work. 1Initialize ${\theta }_G$ for G; ${\theta }_D$ for D. 2For each training iteration: 3 Train DiscriminatorD: Sample n $VI$ patches {$V^1,\cdots ,V^n$} and n corresponding $IR$ patches {$I^1,\cdots ,I^n$}; Acquire generated data {$F^1,\cdots ,F^n$} Update Discriminator parameters ${\theta }_D$ by GradientDescentOptimizer to minimize ${\mathcal{L}}_D$ in Equation (9); (step I) While ${\mathcal{L}}_D>{\mathcal{L}}_{max}\mathrm{and}{N}_D<10$, repeat step I. $N_D\leftarrow N_D+1$; Train GeneratorG: Sample n $VI$ patches {$V^1,\cdots ,V^n$} and n corresponding $IR$ patches {$I^1,\cdots ,I^n$}; Acquire generated data {$F^1,\cdots ,F^n$} Update parameters ${\theta }_G$ by RMSPropOptimizer for minimizing ${\mathcal{L}}_G$ in Equation (2); (step II) While ${\mathcal{L}}_D<{\mathcal{L}}_{min}\mathrm{and}{N}_G<10$, repeat step II. $N_G\leftarrow N_G+1$; While ${\mathcal{L}}_G>{\mathcal{L}}_{Gmax}\mathrm{and}{N}_G<10$, repeat step II. $N_G\leftarrow N_G+1$;