Table 3.
Metrics categorized according to the fairness definition associated, with frequency of papers using the metric.
| Fairness definition | Metric | Frequency |
|---|---|---|
| Demographic Parity (DP) | Earth Mover's Distance (EMD) | 3 |
| Mann-Whitney U test (MWU) | 3 | |
| Normalized KL Divergence(NDKL) | 2 | |
| User Group Fairness (UGF) | 1 | |
| Salary Association Test (SAT) | 1 | |
| Set Difference Rate (SDR) | 1 | |
| List Difference Rate (LDR) | 1 | |
| Min/Max Skew (MS) | 1 | |
| Kendall's Tau (KT) | 1 | |
| Kruskal-Wallis H test (KWH) | 1 | |
| Total Difference between the Recall Curves (TDRC) | 1 | |
| Equal opportunity (EO) | True Positive Rate Parity (TPRP) | 4 |
| Adverse Imact Rate (AIR) | 2 | |
| Counterfactual Fraction (CF) | 1 | |
| True Negative Rate (TNR) | 1 | |
| Infeasible Index (II) | 1 | |
| Infeasible Count (IC) | 1 | |
| Exposure | 1 | |
| Individual Fairness (IF) | Individual Fairness Certification (IFC) | 1 |
| Effect Coefficient (EC) | 1 | |
| Proportional Fairness (PF) | Violations | 1 |
| — | User Acceptance Test (UAT) | 1 |
| Between Group Skill Difference (BGSD) | 1 | |
| Disparate Impact (DI) | 1 | |
| Statistical Parity (SP) | 1 |