Table 6.
The test accuracies of the VGG-16 and 19 trained by the optimization methods for the CIFAR-10 image dataset classification task. “W (Win)”, “T (Tie)”, and “L (Loss)” refer to the number of the compared methods for which HyAdamC-Basic (or HyAdamC-Scale) achieved better, equivalent, and worse test accuracies, respectively. The first and second best results are highlighted in red and orange, respectively.
| VGG-16 | VGG-19 | |||
|---|---|---|---|---|
| Methods | Batch 64 | Batch 128 | Batch 64 | Batch 128 |
| SGD | 0.820 | 0.674 | 0.790 | 0.688 |
| RMSProp | 0.100 | 0.100 | 0.100 | 0.100 |
| Adam | 0.100 | 0.871 | 0.100 | 0.100 |
| AdamW | 0.100 | 0.100 | 0.100 | 0.100 |
| Adagrad | 0.746 | 0.738 | 0.740 | 0.742 |
| AdaDelta | 0.100 | 0.100 | 0.100 | 0.100 |
| Rprop | 0.123 | 0.223 | 0.149 | 0.166 |
| Yogi | 0.100 | 0.100 | 0.100 | 0.100 |
| Fromage | 0.883 | 0.897 | 0.859 | 0.882 |
| TAdam | 0.875 | 0.889 | 0.871 | 0.887 |
| diffGrad | 0.875 | 0.886 | 0.100 | 0.878 |
| HyAdamC-Basic | 0.894 | 0.902 | 0.885 | 0.900 |
| HyAdamC-Scale | 0.899 | 0.900 | 0.890 | 0.899 |
| HyAdamC-Basic: W/T/L | 11/0/0 | 11/0/0 | 11/0/0 | 11/0/0 |
| HyAdamC-Scale: W/T/L | 11/0/0 | 11/0/0 | 11/0/0 | 11/0/0 |