Table 2: Human Baseline:
performance of models on joint training experiments is compared to the human baseline. The analysis is restricted to the 45 tasks used for evaluating humans. ResNet 50 approaches human-level performance only after SSL pre-training and finetuning on all task rules with 1000 samples per rule. Which is 50 times higher than the number of samples needed by humans.
| N training samples | 20 | 1000 | ||
|---|---|---|---|---|
| ResNet-50 | 28.0 | 0 | 57.9 | 14 |
| ViT-small | 29.3 | 1 | 32.7 | 3 |
| SCL | 26.4 | 0 | 44.9 | 11 |
| WReN | 27.5 | 0 | 42.4 | 10 |
| SCL-ResNet 18 | 26.8 | 0 | 64.1 | 18 |
|
| ||||
| ResNet-50 SSL | 45.7 | 7 | 78.3 | 25 |
| ViT-small SSL | 38.7 | 6 | 60.3 | 17 |
|
| ||||
| Humans | 78.7 | 26 | - | - |