Table 1.
Comparison of leading methods to our approach on our assessment data sets. For each method and data set, we report the classification accuracy on the cells that are assigned labels, and in parentheses the proportion of cells that were assigned labels. We also report the ratio of accuracy among assigned cells between the external and withheld test sets
Accuracy ( assigned) in external test set |
Ratio of accuracy in external to withheld test sets | |||||||
|---|---|---|---|---|---|---|---|---|
| Lung | Colon | Brain | PBMC | Lung | Colon | Brain | PBMC | |
| Barcode | 0.918 (84) | 0.998 (100) | 1.000 (73) | 0.986 (100) | 0.95 | 1.01 | 1.01 | 1.02 |
| SingleR | 0.824 (100) | 0.879 (100) | 0.964 (99) | 0.955 (100) | 0.84 | 0.88 | 0.96 | 1.00 |
| Garnett | 0.823 (85) | 0.905 (72) | 1.000 (100) | 1.000 (100) | 0.89 | 0.94 | 1.00 | 1.05 |
| CellAssign | 0.940 (49) | 0.994 (97) | NA (0) | 0.985 (99) | 0.97 | 1.01 | NA | 1.08 |
| CaSTLe | 0.633 (NA) | 0.677 (NA) | 0.136 (NA) | 0.984 (NA) | 0.65 | 0.68 | 0.14 | 1.00 |
| CHETAH | 0.968 (32) | 0.647 (72) | 1.000 (96) | 0.991 (97) | 0.98 | 0.66 | 1.00 | 1.00 |
| SingleCellNet | 0.640 (100) | 0.650 (100) | 1.000 (100) | 0.984 (100) | 0.66 | 0.65 | 1.00 | 1.02 |
| SVM | 1.000 (25) | 0.517 (96) | 1.000 (74) | 0.962 (22) | 1.01 | 0.52 | 1.00 | 0.97 |
| scmap | 0.988 (11) | 0.877 (82) | 1.000 (0.31) | 0.991 (78) | 1.02 | 0.88 | 1.00 | 1.02 |