Table 6. Consecutive loss increase numbers (determination of p1 value).
Model | Experiment | Previous best AFs | Next AF | Average | |||||
---|---|---|---|---|---|---|---|---|---|
ReLU | ELU | SELU | Mish | SiLU | GELU | ||||
VGG16 | Exp1 | SELU | 0 | 1 | 0 | 0 | 0 | 0.20 | |
ReLU | 0 | 0 | 0 | 0 | 0 | 0.00 | |||
Mish | 0 | 0 | 0 | 0 | 0 | 0.00 | |||
Exp2 | ELU | 0 | 0 | 0 | 0 | 0 | 0.00 | ||
SELU | 0 | 0 | 0 | 0 | 0 | 0.00 | |||
ReLU | 0 | 0 | 0 | 0 | 0 | 0.00 | |||
Exp3 | ReLU | 0 | 0 | 0 | 0 | 0 | 0.00 | ||
GELU | 0 | 0 | 0 | 0 | 0 | 0.00 | |||
ELU | 0 | 0 | 0 | 0 | 0 | 0.00 | |||
Overall average | 0.02 | ||||||||
DenseNet121 | Exp1 | GELU | 0 | 2 | 3 | 1 | 0 | 1.20 | |
ReLU | 4 | 2 | 2 | 3 | 2 | 2.60 | |||
SiLU | 0 | 0 | 0 | 0 | 0 | 0.00 | |||
Exp2 | GELU | 0 | 1 | 4 | 0 | 0 | 1.00 | ||
ReLU | 2 | 3 | 4 | 1 | 2 | 2.40 | |||
SiLU | 0 | 2 | 3 | 0 | 0 | 1.00 | |||
Exp3 | ReLU | 3 | 3 | 4 | 2 | 2 | 2.80 | ||
Mish | 0 | 1 | 0 | 0 | 0 | 0.20 | |||
GELU | 0 | 4 | 0 | 0 | 2 | 1.20 | |||
Overall average | 1.38 |