Table 2.
OOD detection performance for different baselines. Near-OOD represents label shift, with samples from the unseen classes of the same dataset. Far-OOD samples are from a separate dataset. The numbers are averaged over five runs.
| Train Dataset | Method | Near-OOD (Wild) | Far-OOD (CIFAR10) | Far-OOD (CelebA) | |||
|---|---|---|---|---|---|---|---|
| AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | ||
| AFHQ | Baseline | 0.88±0.04 | 47.40±5.2 | 0.95±0.04 | 73.59±9.4 | 0.95±0.03 | 70.69±8.9 |
| Baseline+TS [17] | 0.88±0.03 | 45.53±9.8 | 0.95±0.04 | 71.77±8.9 | 0.95±0.03 | 65.89±8.3 | |
| Baseline+TS+ODIN [38] | 0.87±0.05 | 45.02±1.51 | 0.95±0.05 | 69.42±2.38 | 0.95±0.03 | 67.18±2.16 | |
| Baseline+energy [42] | 0.88±0.03 | 47.77±1.10 | 0.94±0.05 | 72.68±2.69 | 0.96±0.04 | 74.75±2.89 | |
| Mixup [70] | 0.86±0.06 | 53.83±6.8 | 0.82±0.11 | 57.01±8.6 | 0.88±0.13 | 70.51±9.8 | |
| DUQ [65] | 0.78±0.05 | 20.98±2.0 | 0.67±0.59 | 16.23±1.5 | 0.66±0.55 | 15.34±2.6 | |
| DDU [47] | 0.83±0.02 | 23.19±2.6 | 0.90±0.02 | 32.98±10 | 0.75±0.02 | 10.32±5.6 | |
| Baseline+ACE | 0.89±0.03 | 51.39±4.4 | 0.98±0.02 | 88.71±5.7 | 0.97±0.03 | 88.87±9.8 | |
|
| |||||||
| MC-Dropout [9] | 0.84±0.09 | 30.78±2.9 | 0.94±0.02 | 73.59±2.1 | 0.95±0.02 | 71.23±1.9 | |
| 5-Ensemble [32] | 0.99±0.01 | 65.73±1.2 | 0.97±0.02 | 89.91±0.9 | 0.99±0.01 | 92.12±0.7 | |
| Near-OOD (Digits 7–9) | Far-OOD (SVHN) | Far-OOD (fMNIST) | |||||
| AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | ||
|
| |||||||
| Dirty MNIST | Baseline | 0.86±0.04 | 28.23±2.9 | 0.75±0.15 | 51.98±0.9 | 0.87±0.02 | 58.12±1.5 |
| Baseline+TS [17] | 0.86±0.01 | 30.12±2.1 | 0.73±0.07 | 48.12±1.5 | 0.89±0.01 | 61.71±2.8 | |
| Baseline+TS+ODIN [38] | 0.83±0.04 | 34.13±12.07 | 0.77±0.13 | 21.59±19.62 | 0.89±0.02 | 46.43±4.31 | |
| Baseline+energy [42] | 0.87±0.04 | 40.30±1.05 | 0.86±0.12 | 43.92±2.30 | 0.91±0.02 | 62.10±5.17 | |
| Mixup [70] | 0.86±0.02 | 35.46±1.0 | 0.95±0.03 | 65.12±3.1 | 0.94±0.05 | 66.00±0.8 | |
| DUQ [65] | 0.78±0.01 | 15.26±3.9 | 0.73±0.03 | 45.23±1.9 | 0.75±0.03 | 50.29±3.1 | |
| DDU [47] | 0.67±0.07 | 10.23±0.9 | 0.68±0.04 | 39.31±2.2 | 0.85±0.02 | 53.76±3.7 | |
| Baseline+ACE | 0.94±0.02 | 37.23±1.9 | 0.98±0.02 | 67.88±3.1 | 0.97±0.02 | 70.71±1.1 | |
|
| |||||||
| MC-Dropout [9] | 0.97±0.02 | 40.89±1.5 | 0.95±0.01 | 62.12±5.7 | 0.93±0.02 | 65.01±0.7 | |
| 5-Ensemble [32] | 0.98±0.02 | 42.17±1.0 | 0.82±0.03 | 55.12±2.1 | 0.94±0.01 | 64.19±4.2 | |
| Near-OOD (Kids) | Far-OOD (AFHQ) | Far-OOD (CIFAR10) | |||||
| AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | ||
|
| |||||||
| CelebA | Baseline | 0.84±0.02 | 1.25±0.1 | 0.86±0.03 | 88.57±0.9 | 0.79±0.02 | 29.01±5.1 |
| Baseline+TS [17] | 0.82±0.04 | 1.24±0.1 | 0.87±0.06 | 88.75±0.9 | 0.78±0.04 | 29.01±5.1 | |
| Baseline+TS+ODIN [38] | 0.65±0.01 | 8.75±2.21 | 0.55±0.01 | 23.03±0.16 | 0.54±0.01 | 5.00±0.07 | |
| Baseline+energy [42] | 0.76±0.51 | 9.40±0.01 | 0.94±0.08 | 32.08±1.70 | 0.85±0.76 | 17.10±0.72 | |
| Mixup [70] | 0.82±0.08 | 22.18±2.7 | 0.95±0.02 | 82.96±2.5 | 0.79±0.13 | 30.54±1.3 | |
| DUQ [65] | 0.80±0.03 | 14.68±3.1 | 0.72±0.07 | 26.62±7.7 | 0.86±0.04 | 28.70±4.1 | |
| DDU [47] | 0.73±0.15 | 7.9±0.4 | 0.74±0.13 | 8.18±0.4 | 0.81±0.15 | 25.45±1.4 | |
| Baseline+ACE | 0.87±0.03 | 34.37±2.5 | 0.96±0.01 | 96.35±2.5 | 0.92±0.05 | 63.51±1.5 | |
|
| |||||||
| MC-Dropout [9] | 0.70±0.10 | 25.62±1.7 | 0.86±0.1 | 91.72±7.5 | 0.74±0.12 | 64.79±1.8 | |
| 5-Ensemble [32] | 0.93±0.03 | 10.35±0.2 | 0.99±0.0 | 98.31±1.2 | 0.92±0.10 | 61.88±1.2 | |
| Near-OOD (other lesions) | Far-OOD (CelebA) | Far-OOD (Skin-texture) | |||||
| AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | AUC-ROC | TNR@TPR95 | ||
|
| |||||||
| Skin Lesion | Baseline | 0.67±0.04 | 8.70±2.5 | 0.66±0.06 | 10.00±3.6 | 0.65±0.10 | 5.91±2.8 |
| Baseline+TS [17] | 0.67±0.05 | 8.69±2.0 | 0.63±0.06 | 9.24±4.3 | 0.68±0.07 | 5.70±3.2 | |
| Baseline+TS+ODIN [38] | 0.68±0.01 | 9.43±0.33 | 0.67±0.07 | 11.32±4.66 | 0.68±0.07 | 6.60±0.29 | |
| Baseline+energy [42] | 0.70±0.04 | 10.85±0.08 | 0.70±0.14 | 7.90±0.29 | 0.65±0.20 | 2.83±1.33 | |
| Mixup [70] | 0.67±0.01 | 8.52±2.8 | 0.64±0.08 | 10.21±4.0 | 0.72±0.05 | 5.26±3.1 | |
| DUQ [65] | 0.67±0.04 | 3.12±1.8 | 0.89±0.09 | 11.89±2.5 | 0.64±0.03 | 4.8±1.5 | |
| DDU [47] | 0.65±0.03 | 3.45±1.9 | 0.75±0.04 | 15.45±2.9 | 0.71±0.05 | 4.19±1.3 | |
| Baseline+ACE | 0.72±0.04 | 10.99±2.8 | 0.97±0.02 | 66.77±1.4 | 0.96±0.03 | 95.83±5.0 | |
|
| |||||||
| MC-Dropout [9] | 0.67±0.05 | 9.45±3.9 | 0.80±0.07 | 30.00±3.2 | 0.56±0.03 | 10.87±2.3 | |
| 5-Ensemble [32] | 0.88±0.01 | 11.23±1.7 | 0.91±0.03 | 27.89±5.9 | 0.76±0.02 | 17.89±3.5 | |