Table 2.
The weighted F1-score of the SVM classifier trained on features generated by different methods, with various percentages of annotations.
| Annotations | 1% | 3% | 5% | 10% | 20% | 30% | 100% |
|---|---|---|---|---|---|---|---|
| Ovarian T cells | |||||||
| No. of cells | 18 | 54 | 91 | 182 | 365 | 548 | 1828 |
| Supervised classifier | – | – | – | 0.518 (0.03) | 0.803 (0.026) | 0.797 (0.027) | 0.856 |
| SimCLR | 0.707 (0.069) | 0.782 (0.038) | 0.789 (0.032) | 0.831 (0.012) | 0.839 (0.007) | 0.850 (0.014) | 0.863 |
| MoCo | 0.527 (0.098) | 0.671 (0.068) | 0.718 (0.052) | 0.767 (0.014) | 0.776 (0.006) | 0.785 (0.007) | 0.800 |
| Debiased | 0.795 (0.053) | 0.802 (0.084) | 0.844 (0.042) | 0.857 (0.017) | 0.858 (0.008) | 0.855 (0.014) | 0.847 |
| SANDI | 0.820 (0.028) | 0.817 (0.077) | 0.829 (0.064) | 0.845 (0.014) | 0.853 (0.019) | 0.849 (0.012) | 0.846 |
| LUSC T cells | |||||||
| No. of cells | 24 | 72 | 120 | 240 | 481 | 722 | 2407 |
| Supervised classifier | – | – | – | 0.338 (0.005) | 0.905 (0.042) | 0.918 (0.016) | 0.935 |
| SimCLR | 0.716 (0.064) | 0.757 (0.03) | 0.806 (0.017) | 0.857 (0.041) | 0.880 (0.027) | 0.898 (0.005) | 0.910 |
| MoCo | 0.664 (0.031) | 0.725 (0.035) | 0.776 (0.012) | 0.808 (0.036) | 0.827 (0.03) | 0.854 (0.013) | 0.898 |
| Debiased | 0.867 (0.012) | 0.869 (0.045) | 0.872 (0.03) | 0.903 (0.02) | 0.929 (0.009) | 0.927 (0.007) | 0.935 |
| SANDI | 0.883 (0.019) | 0.886(0.013) | 0.887(0.014) | 0.898 (0.016) | 0.916 (0.022) | 0.922 (0.008) | 0.934 |
| DCIS FOXP3 | |||||||
| No. of cells | 114 | 343 | 572 | 1145 | 2291 | 3437 | 11,459 |
| Supervised classifier | – | – | – | 0.938 (0.052) | 0.981 (0.008) | 0.985(0.001) | 0.988 |
| SimCLR | 0.965 (0.019) | 0.979 (0.005) | 0.977 (0.009) | 0.976 (0.006) | 0.981 (0.001) | 0.982 (0.003) | 0.987 |
| MoCo | 0.983 (0.006) | 0.982 (0.005) | 0.982(0.006) | 0.980 (0.005) | 0.985(0.001) | 0.986(0.001) | 0.989 |
| Debiased | 0.986 (0.001) | 0.986 (0.003) | 0.983 (0.004) | 0.985 (0.002) | 0.986 (0.001) | 0.986 (0.002) | 0.988 |
| SANDI | 0.986(0.003) | 0.982 (0.008) | 0.984(0.006) | 0.984(0.003) | 0.985(0.001) | 0.985(0.001) | 0.986 |
| Myeloma | |||||||
| No. of cells | 32 | 98 | 163 | 326 | 653 | 980 | 3269 |
| Supervised classifier | – | – | – | 0.958 (0.003) | 0.961 (0.007) | 0.955 (0.011) | 0.965 |
| SimCLR | 0.879 (0.016) | 0.917 (0.04) | 0.949 (0.016) | 0.953 (0.008) | 0.962 (0.008) | 0.968 (0.005) | 0.982 |
| MoCo | 0.844 (0.024) | 0.876 (0.043) | 0.912 (0.02) | 0.928 (0.002) | 0.949 (0.006) | 0.967 (0.007) | 0.985 |
| Debiased | 0.894 (0.024) | 0.900 (0.042) | 0.918 (0.016) | 0.935 (0.010) | 0.940 (0.006) | 0.940 (0.012) | 0.946 |
| SANDI | 0.912(0.028) | 0.942(0.011) | 0.952(0.008) | 0.965(0.002) | 0.975(0.004) | 0.977(0.004) | 0.983 |
| IMC CD4-CD8 | |||||||
| No. of cells | 39 | 118 | 197 | 395 | 790 | 1186 | 3954 |
| Supervised classifier | – | – | – | 0.696 (0.073) | 0.795 (0.103) | 0.777 (0.115) | 0.958 |
| SimCLR | 0.867 (0.038) | 0.908 (0.008) | 0.916 (0.009) | 0.925 (0.006) | 0.933 (0.009) | 0.940 (0.003) | 0.947 |
| MoCo | 0.777 (0.018) | 0.820 (0.017) | 0.844 (0.007) | 0.859 (0.014) | 0.875 (0.009) | 0.878 (0.007) | 0.892 |
| Debiased | 0.910 (0.038) | 0.933 (0.011) | 0.935 (0.004) | 0.940 (0.007) | 0.942 (0.008) | 0.945 (0.006) | 0.958 |
| SANDI | 0.921(0.014) | 0.928 (0.01) | 0.932 (0.007) | 0.933 (0.005) | 0.942(0.005) | 0.942 (0.005) | 0.957 |
Results for annotation percentages ranging from 1% to 30% are the average over 5 trials with different random samplings. Standard deviations are shown inside the parentheses.
Bold values are within 0.003 lower than the best.