Table 7.
Comparison of the performance of the proposed dynamic fairness re-weighting method (DFW) against three other effective methods applied to the XGBoost model: Grid Search (GS), Exponentiated Gradient Reduction (EGR), and Correlation Remover (CR) on the test dataset. Additionally, we compare the performance of DFW with our baseline model, which incorporates Demographic Parity Loss (DPL). Each row: specifies the definition of the sensitive attribute; Each column: specifies the fairness metric to achieve. The disparity is defined as the largest difference in the fairness metric among different subgroups. A smaller disparity is better (bold).
| GS | EGR | CR | DPL | DFW | GS | EGR | CR | DPL | DFW | GS | EGR | CR | DPL | DFW | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AUROC | Recall | Negative predictive value (NPV) | |||||||||||||
| Ethnicity | |||||||||||||||
| Hispanic | 0.8869 | 0.8264 | 0.8856 | 0.8424 | 0.8347 | 0.6604 | 0.6415 | 0.6604 | 0.8289 | 0.7632 | 0.9104 | 0.9073 | 0.9122 | 0.9350 | 0.9127 |
| Non-hispanic | 0.9125 | 0.8287 | 0.9120 | 0.8607 | 0.8345 | 0.6585 | 0.6690 | 0.6655 | 0.8294 | 0.7640 | 0.8929 | 0.8948 | 0.8943 | 0.9211 | 0.9151 |
| Disparity(Max-Min) | 0.0256 | 0.0023 | 0.0264 | 0.0183 | 0.0002 | 0.0019 | 0.0275 | 0.0051 | 0.0005 | 0.0008 | 0.0175 | 0.0125 | 0.0179 | 0.0139 | 0.0024 |
| Gender | |||||||||||||||
| Female | 0.9205 | 0.8592 | 0.9259 | 0.8549 | 0.8109 | 0.6987 | 0.6923 | 0.6859 | 0.7652 | 0.7174 | 0.9056 | 0.9028 | 0.9026 | 0.9261 | 0.9092 |
| Male | 0.9063 | 0.8205 | 0.8925 | 0.8396 | 0.8172 | 0.6087 | 0.6413 | 0.6467 | 0.7956 | 0.7336 | 0.8841 | 0.8920 | 0.8934 | 0.9179 | 0.9050 |
| Disparity(Max-Min) | 0.0142 | 0.0387 | 0.0334 | 0.0153 | 0.0063 | 0.0900 | 0.0510 | 0.0392 | 0.0304 | 0.0162 | 0.0215 | 0.0108 | 0.0092 | 0.0082 | 0.0042 |
| Race | |||||||||||||||
| Asian | 0.9394 | 0.9167 | 0.9646 | 0.9722 | 0.8667 | 0.7778 | 0.8889 | 0.8889 | 0.7889 | 0.8333 | 0.9130 | 0.9545 | 0.9524 | 0.9200 | 0.9310 |
| White | 0.8717 | 0.8301 | 0.8905 | 0.7905 | 0.8074 | 0.5532 | 0.5957 | 0.6277 | 0.6547 | 0.7589 | 0.8783 | 0.8869 | 0.8955 | 0.8490 | 0.9326 |
| African | 0.8918 | 0.8280 | 0.9169 | 0.7935 | 0.7856 | 0.5000 | 0.6071 | 0.6964 | 0.7194 | 0.8023 | 0.8519 | 0.9091 | 0.9012 | 0.8554 | 0.8970 |
| Other | 0.8889 | 0.8388 | 0.9015 | 0.8502 | 0.8518 | 0.5680 | 0.6748 | 0.6553 | 0.6878 | 0.8155 | 0.8455 | 0.8797 | 0.8737 | 0.8571 | 0.9314 |
| Disparity(Max-Min) | 0.0677 | 0.0887 | 0.0741 | 0.1817 | 0.0811 | 0.2778 | 0.2932 | 0.2612 | 0.1342 | 0.0744 | 0.0675 | 0.0748 | 0.0787 | 0.0710 | 0.0356 |