Table 2:
Trial Numbers for Figure 2, Comparing Nonlinear Activation Functions for the Dendrite Nodes in the k-tree For Each Dataset.
MNIST | ||||||
---|---|---|---|---|---|---|
| ||||||
1-tree | 2-tree | 4-tree | 8-tree | 16-tree | 32-tree | |
ReLU | 8 | 9 | 10 | 9 | 10 | 10 |
LReLU | 8 | 10 | 10 | 10 | 10 | 10 |
Sigmoid | 1 | 5 | 0 | 2 | 0 | 0 |
Swish | 10 | 10 | 10 | 10 | 10 | 10 |
NaCaK | 10 | 10 | 10 | 10 | 10 | 10 |
| ||||||
FMNIST | ||||||
| ||||||
1-tree | 2-tree | 4-tree | 8-tree | 16-tree | 32-tree | |
| ||||||
ReLU | 7 | 10 | 8 | 10 | 10 | 10 |
LReLU | 7 | 10 | 10 | 10 | 10 | 10 |
Sigmoid | 2 | 3 | 0 | 0 | 0 | 1 |
Swish | 4 | 7 | 8 | 9 | 10 | 10 |
NaCaK | 9 | 10 | 10 | 10 | 10 | 10 |
| ||||||
KMNIST | ||||||
| ||||||
1-tree | 2-tree | 4-tree | 8-tree | 16-tree | 32-tree | |
| ||||||
ReLU | 8 | 9 | 9 | 10 | 10 | 10 |
LReLU | 7 | 7 | 10 | 10 | 10 | 10 |
Sigmoid | 1 | 1 | 1 | 0 | 0 | 0 |
Swish | 10 | 10 | 10 | 10 | 10 | 10 |
NaCaK | 9 | 10 | 10 | 10 | 10 | 10 |
| ||||||
EMNIST | ||||||
| ||||||
1-tree | 2-tree | 4-tree | 8-tree | 16-tree | 32-tree | |
| ||||||
ReLU | 10 | 10 | 10 | 10 | 10 | 10 |
LReLU | 10 | 10 | 10 | 10 | 10 | 10 |
Sigmoid | 8 | 3 | 1 | 2 | 0 | 0 |
Swish | 10 | 10 | 10 | 10 | 10 | 10 |
NaCaK | 10 | 10 | 10 | 10 | 10 | 10 |
| ||||||
SVHN | ||||||
| ||||||
1-tree | 2-tree | 4-tree | 8-tree | 16-tree | 32-tree | |
| ||||||
ReLU | 0 | 0 | 0 | 1 | 0 | 1 |
LReLU | 1 | 1 | 1 | 1 | 1 | 4 |
Sigmoid | 0 | 0 | 0 | 0 | 0 | 0 |
Swish | 4 | 2 | 2 | 3 | 3 | 3 |
NaCaK | 2 | 3 | 4 | 4 | 3 | 2 |
| ||||||
USPS | ||||||
| ||||||
1-tree | 2-tree | 4-tree | 8-tree | 16-tree | 32-tree | |
| ||||||
ReLU | 6 | 8 | 10 | 10 | 10 | 10 |
LReLU | 6 | 9 | 10 | 10 | 10 | 10 |
Sigmoid | 3 | 3 | 2 | 0 | 0 | 0 |
Swish | 10 | 10 | 10 | 10 | 10 | 10 |
NaCaK | 8 | 6 | 7 | 10 | 10 | 10 |
| ||||||
CIFAR10 | ||||||
| ||||||
1-tree | 2-tree | 4-tree | 8-tree | 16-tree | 32-tree | |
| ||||||
ReLU | 2 | 4 | 4 | 5 | 5 | 6 |
LReLU | 2 | 1 | 5 | 3 | 7 | 6 |
Sigmoid | 0 | 0 | 0 | 0 | 0 | 0 |
Swish | 1 | 4 | 3 | 2 | 4 | 6 |
NaCaK | 10 | 9 | 8 | 8 | 9 | 8 |