TABLE IV.

Summary of Refinement on S-I: Uniformly Flip Patches

Dataset	Tumor/tissue	Method		F1	IoU	PPV	TPR	TNR	NPV
CAME LYON16	29.7 % (± 17.0 %)	Coarse annotations		0.592 ± 0.195	0.416 ± 0.189	0.545 ± 0.246	0.733 ± 0.153	0.745 ± 0.137	0.869 ± 0.139
		DkNN		0.686 ± 0.313	0.570 ± 0.297	0.798 ± 0.308	0.649 ± 0.343	0.971 ± 0.038	0.889 ± 0.146
		Rank Pruning		0.879 ± 0.113	0.772 ± 0.147	0.915 ± 0.050	0.865 ± 0.148	0.965 ± 0.033	0.956 ± 0.040
		LC-MIL	LC-MIL-miNet	0.871 ± 0.104	0.758 ± 0.144	0.836 ± 0.151	0.934 ± 0.062	0.912 ± 0.147	0.971 ± 0.037
			LC-MIL-atten	0.882 ± 0.068	0.770 ± 0.117	0.863 ± 0.110	0.915 ± 0.077	0.938 ± 0.059	0.967 ± 0.042
PAIP2019	38.0 % (± 18.9 %)	Coarse annotations		0.665 ± 0.182	0.514 ± 0.197	0.641 ± 0.237	0.747 ± 0.151	0.758 ± 0.157	0.819 ± 0.144
		DkNN		0.790 ± 0.291	0.711 ± 0.297	0.916 ± 0.149	0.767 ± 0.317	0.961 ± 0.057	0.905 ± 0.140
		Rank Pruning		0.907 ± 0.083	0.823 ± 0.121	0.911 ± 0.114	0.915 ± 0.070	0.952 ± 0.056	0.950 ± 0.039
		LC-MIL	LC-MIL-miNet	0.875 ± 0.156	0.811 ± 0.144	0.875 ± 0.156	0.942 ± 0.058	0.919 ± 0.164	0.954 ± 0.054
			LC-MIL-atten	0.907 ± 0.098	0.828 ± 0.133	0.895 ± 0.138	0.939 ± 0.066	0.934 ± 0.130	0.957 ± 0.046
PAIP2020	43.3 % (± 13.0) %	Coarse annotations		0.592 ± 0.195	0.503 ± 0.169	0.545 ± 0.246	0.697 ± 0.153	0.750 ± 0.138	0.765 ± 0.157
		DkNN		0.662 ± 0.372	0.588 ± 0.362	0.901 ± 0.220	0.626 ± 0.387	0.965 ± 0.058	0.830 ± 0.168
		Rank Pruning		0.841 ± 0.176	0.744 ± 0.182	0.900 ± 0.197	0.807 ± 0.165	0.933 ± 0.130	0.867 ± 0.116
		LC-MIL	LC-MIL-miNet	0.811 ± 0.181	0.704 ± 0.195	0.820 ± 0.198	0.836 ± 0.202	0.881 ± 0.120	0.882 ± 0.124
			LC-MIL-atten	0.823 ± 0.167	0.718 ± 0.185	0.833 ± 0.191	0.848 ± 0.175	0.886 ± 0.128	0.882 ± 0.131