Figure - PMC

Skip to main content

View full-text article in PMC

. Author manuscript; available in PMC: 2023 Apr 7.

Published in final edited form as: Neurocomputing (Amst). 2022 Jan 21;481:333–356. doi: 10.1016/j.neucom.2022.01.014

Figure B.2.5: — The range of A-LR: $1 / s o f t p l u s (\sqrt{v_{t}})$ , $v_{t} = m a x \{v_{t - 1}, {\tilde{v}}_{t}\}$ over iterations for SAMSGrad on MNIST with different choice of β. The maximum ranges in all figures are compressed to a reasonable smaller value compared with those of AMSGrad on MNIST