Table 7.
The test accuracies of the ResNet-18 and 101 trained by the optimization methods for the CIFAR-10 image dataset classification task. “W (Win)”, “T (Tie)”, and “L (Loss)” refer to the number of the compared methods for which HyAdamC-Basic (or HyAdamC-Scale) achieved better, equivalent, and worse test accuracies, respectively. The first and second best results are highlighted in red and orange, respectively.
| ResNet-18 | ResNet-101 | |||
|---|---|---|---|---|
| Methods | Batch 64 | Batch 128 | Batch 64 | Batch 128 |
| SGD | 0.881 | 0.860 | 0.854 | 0.825 |
| RMSProp | 0.924 | 0.920 | 0.912 | 0.911 |
| Adam | 0.931 | 0.923 | 0.934 | 0.929 |
| AdamW | 0.923 | 0.927 | 0.922 | 0.928 |
| Adagrad | 0.914 | 0.910 | 0.924 | 0.918 |
| AdaDelta | 0.932 | 0.931 | 0.931 | 0.936 |
| Rprop | 0.498 | 0.533 | 0.102 | 0.302 |
| Yogi | 0.929 | 0.927 | 0.928 | 0.931 |
| Fromage | 0.894 | 0.921 | 0.911 | 0.912 |
| TAdam | 0.934 | 0.932 | 0.939 | 0.936 |
| diffGrad | 0.926 | 0.928 | 0.933 | 0.938 |
| HyAdamC-Basic | 0.934 | 0.935 | 0.940 | 0.939 |
| HyAdamC-Scale | 0.935 | 0.938 | 0.939 | 0.937 |
| HyAdamC-Basic: W/T/L | 10/1/0 | 11/0/0 | 11/0/0 | 11/0/0 |
| HyAdamC-Scale: W/T/L | 11/0/0 | 11/0/0 | 10/1/0 | 10/0/1 |