TABLE VI.

Summary of Refinement on Hand-Drawn Annotations

Dataset	Tumor/tissue	Method		F1	IoU	PPV	TPR	TNR	NPV
CAME LYON16	29.7 % (± 17.0 %)	Coarse annotations		0.682 ±0.149	0.537 ± 0.173	0.656 ±0.219	0.830 ±0.208	0.980 ± 0.034	0.992 ± 0.015
		DkNN		0.756 ±0.182	0.639 ± 0.214	0.832 ±0.167	0.776 ±0.252	0.992 ± 0.020	0.990 ± 0.017
		Rank Pruning		0.830 ±0.121	0.725 ± 0.154	0.868 ±0.131	0.836 ±0.168	0.995 ± 0.009	0.993 ± 0.013
		LC-MIL	LC-MIL-miNet	0.840 ±0.096	0.736 ± 0.136	0.819 ±0.141	0.898 ±0.129	0.994 ± 0.010	0.994 ± 0.014
			LC-MIL-atten	0.848 ±0.096	0.748 ± 0.133	0.828 ±0.140	0.901 ±0.114	0.994 ± 0.012	0.995 ± 0.012
PAIP2019	38.0 % (± 18.9 %)	Coarse annotations		0.742 ±0.182	0.618 ± 0.195	0.647 ±0.202	0.934 ±0.174	0.848 ± 0.108	0.982 ± 0.039
		DkNN		0.808 ±0.181	0.708 ± 0.203	0.753 ±0.202	0.920 ±0.179	0.912 ± 0.082	0.979 ± 0.040
		Rank Pruning		0.830 ±0.168	0.737 ± 0.197	0.799 ±0.194	0.913 ±0.155	0.926 ± 0.086	0.975 ± 0.044
		LC-MIL	LC-MIL-miNet	0.833 ±0.167	0.741 ± 0.197	0.785 ±0.202	0.932 ±0.131	0.933 ± 0.070	0.978 ± 0.030
			LC-MIL-atten	0.833 ±0.163	0.742 ± 0.203	0.805 ±0.213	0.919 ±0.114	0.938 ± 0.075	0.971 ± 0.034
PAIP2020	43.3 % (± 13.0 %)	Coarse annotations		0.796 ± 0.114	0.675 ± 0.147	0.705 ± 0.152	0.951 ± 0.092	0.925 ± 0.075	0.990 ± 0.022
		DkNN		0.898 ± 0.095	0.826 ± 0.126	0.882 ± 0.105	0.936 ± 0.112	0.977 ± 0.034	0.990 ± 0.022
		Rank Pruning		0.867 ± 0.110	0.779 ± 0.141	0.908 ± 0.091	0.853 ± 0.152	0.984 ± 0.027	0.978 ± 0.028
		LC-MIL	LC-MIL-miNet	0.869 ± 0.081	0.776 ± 0.115	0.860 ± 0.112	0.897 ± 0.103	0.979 ± 0.020	0.981 ± 0.027
			LC-MIL-atten	0.863 ± 0.077	0.766 ± 0.111	0.902 ± 0.097	0.844 ± 0.114	0.986 ± 0.019	0.971 ± 0.035