Table 6.

Loss comparison of optimization algorithms.

Optimizers	Loss
	500	1,000	1,500	2,000
RMSprop	0.5655	0.5450	0.5125	0.4932
Adam	0.5544	0.5094	0.4986	0.4900
RAdam	0.5430	0.4829	0.4693	0.4531
Lookahead	0.5394	0.4639	0.4529	0.4226
R-Lookahead	0.5211	0.4590	0.4213	0.3928