Table 1.

The parameter settings of HyAdamC and the compared first-order optimization methods.

Algorithms	Parameter Settings
HyAdamC	$\alpha ={10}^{-3}$, ${\beta }_1=0.9$, ${\beta }_2=0.99$, $\epsilon ={10}^{-8}$
SGD	$\alpha ={10}^{-3}$
RMSProp	Learning rate = ${10}^{-2}$, $\alpha =0.99$, $\epsilon ={10}^{-8}$
Adam	$\alpha ={10}^{-3}$, ${\beta }_1=0.9$, ${\beta }_2=0.99$
AdamW	$\alpha ={10}^{-3}$, ${\beta }_1=0.9$, ${\beta }_2=0.99$
Adagrad	$\alpha ={10}^{-2}$, ${\beta }_1=0.9$, $\epsilon ={10}^{-10}$
AdaDelta	$\alpha =1.0$, $\rho =0.9$, $\epsilon ={10}^{-6}$
Rprop	$\alpha ={10}^{-2}$, ${\eta }_{-}=0.5$, ${\eta }_{+}=1.2$, step sizes $=[{10}^{-6},50]$
Yogi	$\alpha ={10}^{-2}$, ${\beta }_1=0.9$, ${\beta }_2=0.99$, $\epsilon ={10}^{-3}$
Fromage	$\alpha ={10}^{-2}$
TAdam	$\alpha ={10}^{-3}$, ${\beta }_1=0.9$, ${\beta }_2=0.99$, $v=d$, $k_v=1.0$
diffGrad	$\alpha ={10}^{-3}$, ${\beta }_1=0.9$, ${\beta }_2=0.99$