Algorithm 1: Pseudocode of Adamax

$\eta$: Rate of Learning ${\beta }_1$, ${\beta }_2\in$ $[0$, 1): Exponential decomposed values to moment candidate $C\left(w\right)$: Cost function with $w$ variable $w_0$: parameter vector $m_0\leftarrow 0$ $u_0\leftarrow 0$ $i\leftarrow 0$ (Implement time step) while $w$ does not converge do               $i\leftarrow i+1$        $m_i\leftarrow {\beta }_1\times m_{i-1}+(1-{\beta }_1)\times \frac{\partial C}{\partial w}\left(w_i\right)$          $u_i\leftarrow \mathrm{m}\mathrm{a}\mathrm{x}\left({\beta }_2\times u_{i-1},\left|\frac{\partial C}{\partial w}\left(w_i\right)\right|\right)$ $w_{i+1}\leftarrow w_i-(\eta /(1-{\beta }_1^i\left)\right)\times m_i/u_i$ end while show $w_i$ (end parameter)