Algorithm 1. Pseudocode of Adamax

$\eta$: Rate of Learning ${\beta }_1$, ${\beta }_2\in$ $[0$,1): Exponential decomposing value to moment candidate $C\left(w\right)$: The cost function with variable $w$ $w_0$: Primary parameter vector $m_0\leftarrow 0$ $u_0\leftarrow 0$ $i\leftarrow 0$ (Apply time step) while $w$ does not converge apply $i\leftarrow i+1$ $m_i\leftarrow {\beta }_1·m_{i-1}+\left(1-{\beta }_1\right)·\frac{\partial C}{\partial w}\left(w_i\right)$ $u_i\leftarrow \max \left({\beta }_2·u_{i-1}, \left|\frac{\partial C}{\partial w}\left(w_i\right)\right|\right)$ $w_{i+1}\leftarrow w_i-\left(\eta /\left(1-{\beta }_1^i\right)\right)·m_i/u_i$ end while displaying $w_i$ (end variable)