Table 3. Test loss values obtained as a result of experiments to determine the best AFs.
| Model | Exp | Epoch | AF | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Tanh | ReLU | ELU | SELU | Mish | SiLU | GELU | |||
| VGG16 | Exp1 | 1 | 2.3076 | 0.9476 | 1.0678 | 1.2563 | 1.2892 | 1.1551 | 1.1570 |
| 2 | 0.6696 | 0.6680 | 0.7820 | 0.8432 | 0.8062 | 0.7725 | |||
| 3 | 0.5962 | 0.5724 | 0.6139 | 0.6929 | 0.6398 | 0.7045 | |||
| 4 | 0.5256 | 0.5992 | 0.6085 | 0.6095 | 0.6129 | 0.6445 | |||
| 5 | 0.5696 | 0.6102 | 0.6128 | 0.5439 | 0.5598 | ||||
| 6 | 0.5554 | 0.5163 | 0.5340 | 0.5436 | |||||
| Exp2 | 1 | 2.3142 | 1.0733 | 0.9366 | 1.4203 | 1.2050 | 1.2574 | 1.3257 | |
| 2 | 2.3071 | 0.6606 | 0.7246 | 0.8865 | 0.8317 | 0.7698 | 0.9488 | ||
| 3 | 0.6901 | 0.5796 | 0.6664 | 0.5938 | 0.6802 | 0.6997 | |||
| 4 | 0.5445 | 0.5255 | 0.5771 | 0.5877 | 0.5580 | 0.6504 | |||
| 5 | 0.5327 | 0.6197 | 0.5706 | 0.5932 | 0.5556 | 0.5919 | |||
| 6 | 0.5106 | 0.5274 | 0.5679 | 0.5578 | |||||
| Exp3 | 1 | 2.3181 | 1.3439 | 2.3096 | 1.2635 | 1.2560 | 1.2025 | 1.6888 | |
| 2 | 2.3079 | 0.8153 | 2.3157 | 0.8118 | 0.7729 | 0.7723 | 0.8539 | ||
| 3 | 0.7513 | 2.3073 | 0.6305 | 0.6168 | 0.6512 | 0.9304 | |||
| 4 | 0.6177 | 1.3916 | 0.6437 | 0.5809 | 0.5713 | 0.6721 | |||
| 5 | 0.5600 | 0.7189 | 0.5747 | 0.5858 | 0.5833 | ||||
| 6 | 0.5151 | 0.6646 | 0.5580 | 0.6320 | |||||
| 7 | 0.5391 | 0.5170 | |||||||
| 8 | 0.5236 | ||||||||
| DenseNet121 | Exp1 | 1 | 2.5334 | 0.6554 | 1.5114 | 1.1965 | 1.1738 | 1.1499 | 0.9080 |
| 2 | 1.3031 | 0.5978 | 1.1989 | 1.1150 | 1.0227 | 0.9996 | 0.6984 | ||
| 3 | 1.5998 | 1.2462 | 1.0469 | 0.8134 | 0.9571 | 0.6139 | |||
| 4 | 1.1421 | 0.8565 | 1.0753 | 1.1066 | 0.7643 | 0.6911 | |||
| 5 | 1.1080 | 0.9678 | 0.9509 | 0.9248 | 1.0973 | 0.7137 | |||
| 6 | 1.0704 | 0.8023 | 0.8480 | 0.6475 | 0.7673 | 0.5965 | |||
| 7 | 0.9837 | 0.7692 | 0.8905 | 0.7938 | |||||
| 8 | 0.8963 | 0.8283 | 0.8551 | 0.7441 | |||||
| 9 | 0.8924 | 0.7902 | 0.8649 | 0.8058 | |||||
| 10 | 1.2865 | 0.7778 | 0.9231 | 0.7149 | |||||
| 11 | 0.8584 | 0.6953 | 0.8685 | 0.6174 | |||||
| 12 | 20.381 | 0.9441 | |||||||
| 13 | 0.9130 | 0.7886 | |||||||
| 14 | 0.9475 | 0.8488 | |||||||
| 15 | 1.5578 | 0.7172 | |||||||
| 16 | 0.7754 | ||||||||
| Exp2 | 1 | 2.6434 | 0.6864 | 1.2152 | 1.2862 | 1.3581 | 1.1517 | 0.8766 | |
| 2 | 1.4907 | 0.6845 | 1.0475 | 1.0604 | 1.4940 | 1.1341 | 0.8051 | ||
| 3 | 1.8102 | 0.6161 | 0.8702 | 1.1703 | 0.7974 | 0.8132 | 0.6477 | ||
| 4 | 0.9409 | 1.0633 | 1.1219 | 0.8704 | 0.8092 | 0.6355 | |||
| 5 | 1.4932 | 0.9105 | 1.0914 | 0.7391 | 0.7236 | 0.8993 | |||
| 6 | 1.3614 | 0.7997 | 0.9500 | 0.7348 | 0.7522 | 0.5716 | |||
| 7 | 0.9439 | 0.8416 | 1.0224 | 0.6762 | 0.7218 | ||||
| 8 | 1.0116 | 0.8869 | 1.2598 | 0.7274 | 0.7383 | ||||
| 9 | 0.9864 | 0.9702 | 1.0666 | 0.7168 | 0.7116 | ||||
| 10 | 1.2859 | 0.7945 | 1.2473 | 0.7997 | 0.6661 | ||||
| 11 | 0.7728 | 0.7792 | 0.8475 | 0.6685 | 0.7161 | ||||
| 12 | 1.0680 | 0.7670 | 0.6738 | ||||||
| 13 | 0.7520 | 0.6274 | |||||||
| 14 | 0.7260 | ||||||||
| Exp3 | 1 | 2.8821 | 0.6683 | 1.4735 | 1.2436 | 1.0554 | 1.0850 | 0.8528 | |
| 2 | 1.2687 | 1.0291 | 0.9248 | 1.1282 | 1.1075 | 0.9101 | 0.9093 | ||
| 3 | 1.1558 | 0.5604 | 0.8414 | 1.0731 | 0.7743 | 0.8047 | 0.8000 | ||
| 4 | 1.1121 | 0.8957 | 1.1002 | 0.8894 | 0.7906 | 0.7144 | |||
| 5 | 0.9807 | 1.0537 | 0.9099 | 0.6880 | 0.7124 | 0.6606 | |||
| 6 | 0.9404 | 0.7695 | 0.8878 | 0.6593 | 0.7021 | 0.6444 | |||
| 7 | 1.1520 | 1.1104 | 0.8715 | 0.8545 | 0.7681 | 0.6788 | |||
| 8 | 0.9789 | 0.8887 | 0.8372 | 0.5865 | 0.7868 | 0.6399 | |||
| 9 | 0.9592 | 0.8042 | 0.9499 | 0.6442 | |||||
| 10 | 0.8842 | 0.7549 | 0.8418 | ||||||
| 11 | 1.2925 | 0.7156 | 1.1430 | ||||||
| 12 | 0.7164 | 0.8619 | 0.9685 | ||||||
| 13 | 1.0142 | 0.9562 | |||||||
| 14 | 0.7758 | 1.1994 | |||||||
| 15 | 0.6857 | 0.8176 | |||||||
| 16 | 0.9022 | ||||||||
| 17 | 0.9555 | ||||||||
| 18 | 0.8060 | ||||||||