Table 2.
Justified Trust and Explanation Satisfaction Results of CX-ToM and baselines on ResNet-50
| XAI framework | Justified trust (±std) | Explanation satisfaction (±std) |
||||
|---|---|---|---|---|---|---|
| Confidence | Usefulness | Appropriate detail | Understandability | Sufficiency | ||
| Non-expert pool | ||||||
| Grad-CAM (Selvaraju et al., 2017a) | 21.6 ± 2.8% | 3.2 ± 1.5 | 3.2 ± 1.6 | 2.7 ± 2.8 | 3.0 ± 2.0 | 2.9 ± 0.9 |
| LIME (Ribeiro et al., 2016) | 26.9 ± 3.5% | 3.3 ± 2.5 | 3.1 ± 2.1 | 3.7 ± 1.9 | 3.1 ± 1.8 | 4.0 ± 1.3 |
| TCAV (Kim et al., 2018) | 42.2 ± 2.6% | 4.1 ± 2.7 | 3.2 ± 2.4 | 3.8 ± 1.9 | 4.0 ± 1.5 | 3.5 ± 1.8 |
| CVE (Goyal et al., 2019) | 38.1 ± 3.5% | 2.7 ± 2.5 | 2.6 ± 1.5 | 3.0 ± 2.0 | 3.2 ± 1.1 | 3.2 ± 1.9 |
| Fault-lines without ToM | 54.2 ± 2.4% | 6.1 ± 1.7 | 5.9 ± 1.2 | 6.6 ± 1.5 | 6.4 ± 0.9 | 6.2 ± 1.1 |
| CX-ToM (fault-lines with ToM) | 58.3 ± 1.8% | 6.3 ± 1.8 | 6.2 ± 1.6 | 6.9 ± 1.1 | 7.2 ± 0.8 | 7.2 ± 1.6 |
| Expert pool | ||||||
| Grad-CAM (Selvaraju et al., 2017a) | 20.1 ± 1.8% | 2.5 ± 2.2 | 2.5 ± 1.8 | 1.7 ± 1.9 | 3.0 ± 1.9 | 3.0 ± 1.2 |
| LIME (Ribeiro et al., 2016) | 25.4 ± 2.7% | 3.0 ± 1.6 | 3.2 ± 2.9 | 3.8 ± 2.1 | 2.6 ± 1.0 | 2.5 ± 2.9 |
| TCAV (Kim et al., 2018) | 46.0 ± 2.4% | 3.5 ± 1.4 | 3.8 ± 1.7 | 3.6 ± 2.2 | 3.8 ± 2.1 | 4.0 ± 1.9 |
| CVE (Goyal et al., 2019) | 43.1 ± 3.1% | 3.2 ± 2.3 | 3.2 ± 0.9 | 3.0 ± 1.8 | 3.0 ± 1.3 | 3.4 ± 1.8 |
| Fault-lines without ToM | 54.9 ± 1.6% | 6.2 ± 2.1 | 6.0 ± 1.2 | 5.3 ± 1.6 | 6.0 ± 1.5 | 5.9 ± 1.5 |
| CX-ToM (fault-lines with ToM) | 56.0 ± 1.9% | 5.8 ± 1.6 | 6.1 ± 1.0 | 6.1 ± 1.0 | 7.0 ± 1.5 | 7.0 ± 1.2 |