Table 5. Test loss values obtained as a result of experiments to determine the next best AFs.

Model	Experiment	Previous best AFs	Epoch	Next AF
				ReLU	ELU	SELU	Mish	SiLU	GELU
VGG16	Exp1	SELU	1	0.5176	0.5686		0.5154	0.5488	0.5974
			2		0.5863
			3		0.5659
		ReLU	1		0.5669	0.8879	0.5360	0.5024	0.5180
			2		0.5193	0.6527
			3			0.6220
			4			0.5375
		Mish	1	0.5685	0.6889	0.7069		0.5997	0.5727
			2		0.6410	0.5970
			3		0.5883
	Exp2	ELU	1	0.5168		0.5886	0.5549	0.5565	0.5648
			2					0.5424
		SELU	1	0.5955	0.5296		0.6120	0.5394	0.5688
			2	0.5555			0.5665	0.5379	0.5490
		ReLU	1		0.7453	0.9466	0.5607	0.5521	0.5671
			2		0.5835	0.6856
			3		0.5711	0.5991
			4			0.5508
	Exp3	ReLU	1		0.6359	0.6902	0.5166	0.5393	0.5562
			2		0.5564	0.5660		0.5208
		GELU	1	0.5698	0.8151	1.0327	0.5688	0.5495
			2		0.5771	0.6736	0.5681
			3		0.5500	0.6185
			4			0.5921
		ELU	1	0.5722		0.5233	0.5318	0.5265	0.5046
			2	0.5171
DenseNet121	Exp1	GELU	1	0.5478	0.9620	1.2328	0.9561	0.6372
			2		0.7391	1.1520	0.7870
			3		0.8254	1.3142	0.6893
			4		1.0896	1.0417	0.7205
			5		0.7031	1.2832	0.6803
			6			1.2325
			7			1.5440
			8			0.8618
		ReLU	1		1.5544	1.2104	0.9347	0.8831	0.8520
			2		0.9110	1.0741	0.7624	0.7925	0.7353
			3		0.9521	1.0381	0.7525	0.6902	0.6069
			4		1.1389	1.2686	1.0301	0.6127	0.7015
			5		0.9409	1.2387	0.8243	0.7614	0.6703
			6		0.9092	0.9237	0.6854	0.7430	0.5824
			7		0.8348	1.0289	0.7646	0.6335
			8		0.8329	0.8830	0.6800	0.6071
			9		0.7013	1.0159	0.6325	0.5949
			10		0.8792	0.7960
			11		0.8068
			12		0.8731
			13		0.7015
			14		0.6589
		SiLU	1	0.6180	1.1109	1.7046	0.7340		0.6514
			2		0.7586	1.2093	0.7088
			3		0.6890	0.9131
	Exp2	GELU	1	0.5539	0.8230	1.3235	0.7502	0.6129
			2		1.0572	1.1054	0.6783
			3		0.8220	1.0143	0.6193
			4		0.8454	0.9445
			5		0.7590	0.8853
			6		0.7906	0.9024
			7		0.7254	0.9238
			8			1.1919
			9			1.1035
			10			0.8324
		ReLU	1		1.0134	1.2213	0.9434	0.8702	0.9008
			2		0.9379	1.1288	0.7861	0.8505	0.7791
			3		1.0422	0.9406	0.6726	0.9180	0.6313
			4		0.9784	1.0536	0.9586	0.6715	0.6604
			5		0.8998	1.1468	0.7218	0.5984	0.7107
			6		0.8033	0.9460	0.7172		0.6131
			7		0.8312	0.8926	0.8329		0.6058
			8		0.7211	0.9610	0.6197		0.5496
			9			0.8678
		SiLU	1	0.6232	0.9390	1.5974	0.7148		0.6366
			2		0.8063	1.1322
			3		1.0008	0.9104
			4		0.7552	0.9664
			5		0.8774	1.0150
			6		0.8153	1.1679
			7		0.7441	0.8454
	Exp3	ReLU	1		1.1103	1.1455	0.9927	1.1415	0.9124
			2		1.3042	1.1690	0.8607	0.8905	0.6170
			3		0.8980	1.1342	0.7389	0.6987	0.7598
			4		0.8322	0.9823	0.6869	0.6999	0.6880
			5		0.8283	0.9458	0.6260	0.7668	0.5713
			6		1.0211	1.3660	0.6776	0.5890
			7		0.8523	0.9618	0.6410
			8		0.8341	1.2932	0.7156
			9		0.8244	0.9192	0.8070
			10		0.8018	0.7769	0.6258
			11		0.6888
			12		0.7590
			13		0.6685
		Mish	1	0.5551	0.8781	0.9893		0.6514	0.6946
			2		0.9072	0.9175			0.6533
			3		0.8157
			4		0.7380
		GELU	1	0.5971	0.8531	1.3561	0.6342	0.9523
			2		0.8778	1.0365		0.7361
			3		0.9276			0.7168
			4		0.8790			0.7327
			5		0.9445			0.7413
			6		0.7341			0.6974