Table 2.
Model performance measures across subpopulations: before and after fairness algorithm processing
PCE |
FRAX |
|||
---|---|---|---|---|
Performance measure | Before | After | Before | After |
Expectation across subpopulations [CITL] | 1.247 (1.227–1.277) | 1.039 (1.025–1.058) | 1.076 (1.065–1.089) | 1.001 (0.992–1.012) |
Variance across subpopulations (CITL) | 0.500 (0.450–0.633) | 0.006 (0.006–0.013) | 0.070 (0.064–0.082) | 0.004 (0.003–0.006) |
CITL 20th percentile | 0.805 (0.787–0.824) | 0.983 (0.965–0.992) | 0.865 (0.856–0.882) | 0.962 (0.945–0.967) |
CITL 50th percentile | 1.068 (1.043–1.087) | 1.019 (1.005–1.036) | 1.004 (0.991–1.018) | 0.997 (0.987–1.006) |
CITL 80th percentile | 1.492 (1.453–1.565) | 1.090 (1.071–1.119) | 1.249 (1.237–1.285) | 1.034 (1.028–1.058) |
Expectation across subpopulations [CS] | 1.739 (1.698–1.781) | 0.876 (0.855–0.898) | 1.414 (1.377–1.449) | 0.906 (0.884–0.927) |
Variance across subpopulations (CS) | 1.179 (1.048–1.382) | 0.019 (0.018–0.030) | 0.484 (0.434–0.560) | 0.014 (0.014–0.023) |
CS 20th percentile | 0.884 (0.846–0.907) | 0.772 (0.738–0.798) | 0.835 (0.800–0.851) | 0.819 (0.786–0.843) |
CS 50th percentile | 1.280 (1.235–1.322) | 0.883 (0.866–0.910) | 1.238 (1.151–1.276) | 0.913 (0.898–0.933) |
CS 80th percentile | 2.885 (2.683–2.952) | 0.969 (0.956–1.01) | 2.027 (1.924–2.135) | 0.982 (0.973–1.014) |
Area Under the Receiver Operating Curve | 0.730 (0.727–0.733) | 0.736 (0.734–0.739) | 0.712 (0.710–0.715) | 0.714 (0.712–0.716) |
Area Under the Precision-Recall Curve | 0.109 (0.107–0.111) | 0.116 (0.114–0.119) | 0.180 (0.177–0.183) | 0.183 (0.180–0.185) |
Brier Score | 0.043 (0.043–0.044) | 0.043 (0.042–0.043) | 0.068 (0.068–0.069) | 0.068 (0.067–0.068) |
Note: Values are point estimates and 95% confidence intervals, derived via the percentile bootstrap method with 200 repetitions.
Abbreviations: CITL, calibration in the large; CS, calibration slope; FRAX, fracture risk assessment tool; NA, not applicable; PCE, pooled cohort equations.