TABLE V.

Summary of Refinement on S-II: Omit Small Lesions

Dataset	Tumor/tissue	Method		F1	IoU	PPV	TPR	TNR	NPV
CAME LYON16	29.7% (± 17.0%)	Coarse annotations		0.561 ± 0.136	0.402 ± 0.129	0.485 ± 0.147	0.927 ± 0.177	0.987 ± 0.012	0.984 ± 0.020
		DkNN		0.597 ± 0.266	0.471 ± 0.247	0.774 ± 0.188	0.861 ± 0.233	0.998 ± 0.004	0.981 ± 0.023
		Rank Pruning		0.758 ± 0.144	0.630 ± 0.175	0.844 ± 0.147	0.869 ± 0.174	0.998 ± 0.003	0.986 ± 0.019
		LC-MIL	LC-MIL-miNet	0.840 ± 0.091	0.735 ± 0.127	0.773 ± 0.142	0.922 ± 0.123	0.996 ± 0.005,	0.993 ± 0.014
			LC-MIL-atten	0.849 ± 0.084	0.747 ± 0.117	0.827 ± 0.137	0.906 ± 0.119	0.995 ± 0.008	0.994 ± 0.012
PAIP2019	38.0% (± 18.9%)	Coarse annotations		0.687 ± 0.168	0.547 ± 0.186	0.586 ± 0.179	0.989 ± 0.029	0.892 ± 0.076	0.930 ± 0.093
		DkNN		0.722 ± 0.242	0.612 ± 0.254	0.713 ± 0.193	0.971 ± 0.051	0.947 ± 0.051	0.929 ± 0.093
		Rank Pruning		0.768 ± 0.215	0.666 ± 0.245	0.766 ± 0.195	0.955 ± 0.055	0.962 ± 0.043	0.931 ± 0.094
		LC-MIL	LC-MIL-miNet	0.842 ± 0.119	0.743 ± 0.165	0.747 ± 0.208	0.961 ± 0.053	0.955 ± 0.040	0.956 ± 0.069
			LC-MIL-atten	0.834 ± 0.128	0.734 ± 0.173	0.781 ± 0.219	0.934 ± 0.077	0.951 ± 0.049	0.955 ± 0.067
PAIP2020	43.3% (± 13.0%)	Coarse annotations		0.570 ± 0.076	0.403 ± 0.070	0.600 ± 0.130	0.979 ± 0.045	0.961 ± 0.030	0.918 ± 0.058
		DkNN		0.548 ± 0.118	0.386 ± 0.104	0.834 ± 0.116	0.968 ± 0.048	0.991 ± 0.010	0.909 ± 0.058
		Rank Pruning		0.607 ± 0.147	0.452 ± 0.152	0.867 ± 0.101	0.901 ± 0.129	0.993 ± 0.009	0.916 ± 0.062
		LC-MIL	LC-MIL-miNet	0.786 ± 0.106	0.659 ± 0.138	0.832 ± 0.115	0.923 ± 0.057	0.985 ± 0.014	0.950 ± 0.055
			LC-MIL-atten	0.718 ± 0.138	0.576 ± 0.157	0.884 ± 0.100	0.875 ± 0.075	0.988 ± 0.014	0.933 ± 0.064