Table 2.

Experiment with hyperparameters optimization and settings.

Hyperparameters and Settings
Loss Function	Categorical Smooth Loss				Categorical Cross-Entropy
Optimizers	Adam		SGD		Adam		SGD
Learning rate	0.0001	0.001	0.0001	0.001	0.0001	0.001	0.0001	0.001
Batch size	8		8		8		8
Reduce Learning Rate	0.2		0.2		0.2		0.2
Epsilon	0.001		0.001		0.001		0.001
Patience	10		10		10		10
Verbose	1		1		1		1
Es-Callback (Patience)	10		10		10		10
Clip Value	0.2		0.2		0.2		0.2
Epoch	100		100		100		100
Patch Size	(2, 2)		(2, 2)		(2, 2)		(2, 2)
Drop Rate	0.01		0.01		0.01		0.01
Number of Heads	8		8		8		8
Embed_dim	64		64		64		64
Num_MLP	256		256		256		256
Window Size	Window Size//2		Window Size//2		Window Size//2		Window Size//2
Input Size	(224 x 224)		(224 x 224)		(224 x 224)		(224 x 224)