Table 3.
Justified Trust and Explanation Satisfaction Results of CX-ToM and baselines on PACNet
| XAI framework | Justified trust (±std) | Explanation satisfaction (±std) |
||||
|---|---|---|---|---|---|---|
| Confidence | Usefulness | Appropriate detail | Understandability | Sufficiency | ||
| Non-expert pool | ||||||
| Grad-CAM (Selvaraju et al., 2017a) | 15.2 ± 1.5% | 2.4 ± 1.8 | 2.6 ± 1.2 | 2.5 ± 1.5 | 2.9 ± 1.7 | 3.0 ± 1.2 |
| LIME (Ribeiro et al., 2016) | 22.2 ± 2.4% | 3.1 ± 2.2 | 2.7 ± 2.0 | 3.5 ± 1.9 | 2.7 ± 1.2 | 3.8 ± 1.6 |
| TCAV (Kim et al., 2018) | 40.1 ± 2.2% | 3.9 ± 1.7 | 3.6 ± 1.1 | 4.1 ± 2.5 | 4.0 ± 1.2 | 3.6 ± 1.8 |
| CVE (Goyal et al., 2019) | 41.5 ± 3.2% | 3.1 ± 1.5 | 3.3 ± 1.0 | 3.8 ± 2.1 | 3.8 ± 2.0 | 3.9 ± 1.2 |
| Fault-lines without ToM | 53.8 ± 1.9% | 6.3 ± 2.0 | 5.6 ± 1.1 | 6.1 ± 1.9 | 5.9 ± 0.6 | 6.6 ± 1.6 |
| CX-ToM (fault-lines with ToM) | 54.8 ± 2.0% | 6.2 ± 2.0 | 6.5 ± 1.8 | 6.2 ± 1.0 | 7.0 ± 1.9 | 6.8 ± 1.9 |
| Expert pool | ||||||
| Grad-CAM (Selvaraju et al., 2017a) | 16.8 ± 1.9% | 2.3 ± 1.2 | 2.9 ± 1.4 | 2.0 ± 1.9 | 3.1 ± 1.5 | 3.2 ± 2.2 |
| LIME (Ribeiro et al., 2016) | 23.7 ± 2.0% | 2.9 ± 1.3 | 3.2 ± 2.5 | 3.0 ± 2.1 | 2.5 ± 1.6 | 2.9 ± 2.0 |
| TCAV (Kim et al., 2018) | 38.6 ± 3.1% | 3.9 ± 1.3 | 3.2 ± 1.5 | 3.9 ± 2.0 | 4.0 ± 1.0 | 3.7 ± 1.1 |
| CVE (Goyal et al., 2019) | 39.1 ± 2.0% | 3.5 ± 2.2 | 3.7 ± 1.6 | 3.2 ± 1.2 | 3.9 ± 1.1 | 3.0 ± 1.5 |
| Fault-lines without ToM | 57.0 ± 1.8% | 6.0 ± 1.5 | 6.2 ± 1.7 | 5.8 ± 1.9 | 5.5 ± 1.1 | 6.1 ± 1.9 |
| CX-ToM (fault-lines with ToM) | 59.8 ± 1.6% | 6.3 ± 1.1 | 6.5 ± 1.7 | 7.0 ± 1.5 | 6.7 ± 1.7 | 6.5 ± 1.0 |